Ranking the Unrankable

It is often said in football and basketball that referees should not decide the outcome of the game. There is good reason for that truism, and its truth makes Olympic judging controversies inevitable. If spectators cannot tell who won the gold until the judges reveal their scores, it is not possible for everyone to agree on the winners and losers. When beauty is in the eyes of the beholder, scoring ultimately is highly subjective, and reasonable people will view things differently.

And the inevitability of disagreement is aggravated by inescapable biases. In some cases, referees succumb to the “home team” bias, as apparently happened earlier this week in the women’s figure skating final. In other cases, referees will shade their scores based on reputation bias. Judges give higher marks when they have a more positive view of the competitor’s reputation.

All in all, it is not clear why we even try to identify the best skaters, gymnasts, or other athletes whose scores are determined by judges. While some of these athletes are clearly better than other athletes, we cannot draw the fine distinctions that Olympic medaling purports to draw. We can say who runs or swims faster, who jumps higher or farther, and which team scores more points, but we cannot confidently say who is the best figure skater, snowboarder, or diver anymore than we can say who is the best painter, sculptor, or actor.

We should be satisfied enjoying the entertainment provided by ice skaters and gymnasts, just as we enjoy the entertainment of Cirque du Soleil, but we should not try to rank the unrankable.

[cross-posted at orentlicher.tumblr.com]

Comments

[Howard Wasserman]

I frame this as the difference between sport and non-sport.

[Urska]

As a former gymnast, I have to disagree with both David and Howard. To competitors, it's quite obvious who is the better athlete. They might disagree with their own score, think judges are biased against them, but ultimately, the best do win. A new scoring system can mess things up, but it often gets adjusted over time to better reflect who's the best.

[Barry]

"It is often said in football and basketball that referees should not decide the outcome of the game. "

And it's often said that referees who are enforcing the rules are 'deciding the outcome of the game'.