May 14, 2018

The problem with forced rankings

What’s the best college in the US?

What about the best car?

Best stereo speakers? Best pizza?

The answer is always the same: It depends.

People hate that. “It depends” puts you on the hook, requires you to have priorities and a point of view. 

A forced ranking is freeing. It tells you exactly what to expect, and if things don’t work out, well, blame the system. A forced ranking brings status along with it, because, apparently, if you care enough or are rich enough to have the best, then you must be the best.

When we compress 100 variables into just one linear measure, we add enormous amounts of editorial tweaking and lose a ton of nuance. If you want to study aeronautical engineering, Harvard isn’t going to be a good choice. If you’re gluten-free or diabetic, that pizza place might not work out so well for you. And if your definition of a good car includes safety, fuel efficiency or the ability to move your family around, that McLaren isn’t going to make you happy.

Forced rankings abandon multiple variables, and they magnify differences that aren’t statistically significant. “Well, there has to be one winner,” they say, but of course, this isn’t true. It’s not a linear race, and the very concept of a single winner is forced.

When the US News college list started to get traction, plenty of college presidents spoke out in opposition. Over time, though, they discovered that being well ranked was profitable, and in an industry that touches billions of dollars a year, status leads to money and money leads to more status… Today, many colleges are intentionally gaming the system by changing what they originally stood for simply to move up. 

High rankings do more than distort the behavior of those that seek to move up. High rankings attract the sort of people who don’t want to discover their own ‘best’. Who want to be around others that care about high rankings. Who will run to the next high rank the moment the world changes. And those that are attracted to the winner of a forced ranking change the very tenor of the place they chose. So now, that restaurant that used to be special is merely crowded. Now the company that only keeps its top performers is a horrible place to work.

The biggest problem with a forced ranking is that it’s forced.