FIFA is no stranger to alleged biases and corruption (see Sepp Blatter, the Russian and Qatari World Cup selection controversies). But to what extent does bias permeate The Best FIFA Men’s Player award, the annual award given to the most outstanding player in men’s football?

The winner of this annual award is selected via a ranked-ballot vote by an international jury of representatives and media from each national men’s team. It should come as no surprise that a significant amount of bias permeates this process – but precisely how much, and along what lines is what professors Ian McCarthy and Michael Johnson set out to uncover.

It was revealed some years ago that the English captain, Steven Gerrard, included Luis Suarez, at the top of his 2013 ballot. This revelation was the catalyst that inspired McCarthy and Johnson to investigate potential bias in award voting. At the time, Suarez was arguably a top 10 player, but he certainly wasn’t the best player. So why did Gerrard vote this way?

Luckily, FIFA publishes the voting results, making them easy to study. Each member country has three voting members: the coach and captain from the men’s national team and a member of the media who all vote on a shortlist of 23 candidates. Each voting member submits a ranked ballot with three votes—worth five, three and one point(s) respectively – the player with the highest point total wins the award.

McCarthy and Johnson analyzed the voting records against the best players’ statistics for each season from 2010 to 2016, comparing the number of points a voter awarded to a player against the average points that same player received. Unsurprisingly, they discovered a significant amount of voting bias, which they described in three grouped factors: cultural similarity, in-group bias, and impartiality of the voters’ countries.

Cultural similarity can be decomposed into three factors (cultural distance, cultural clusters and collectivism) while in-group factors can be broken down into six (nationality, club, league, geography, ethnicity, religion and language). Impartiality of the voter’s country refers to whether or not that country’s government treats all individuals in an impartial manner regardless of race, ethnicity, family/political ties, or social standing.

They found that some of the strongest determinants of voting bias were voter-player similarity in cultural distance, cultural clusters, and collectivism. That means that voters and players from neighboring countries, whose capital cities were geographically close, were more likely to be biased towards one another in their voting. Interestingly, McCarthy and Johnson found that voters from collectivist cultures were also more likely to demonstrate bias, while those from more diverse countries were less likely to do so.

Captains demonstrated the highest amount of voting bias, followed by coaches, then the media. The captain’s biases can be explained by their closeness to players and in-group bias. Back to the Gerrard/Suarez example—McCarthy and Johnson found that case explainable by an in-group bias. When voters and players share a nationality, club, league, or are from neighboring countries, then voting bias is positively correlated. In the Gerrard/Suarez example, Gerrard and Suarez played for the same club—Liverpool FC.

“In sporting awards from soccer to basketball to cricket, there is a whole range of data where you can empirically say, ‘This is the best player,’” McCarthy says. “What we're showing is that beyond the top three superstar outliers, there are significant levels of cultural bias in terms of where they actually rank.”

Luckily, the amount of bias that McCarthy and Johnson discovered wasn’t enough to jeopardize the integrity of the award. But given how common it is across many industries to select award winners in a similar manner, it is worthwhile to keep their findings in mind when examining the results.

It seems like these biases are part of human nature and are only suppressed by strong institutions in a relatively small number of countries. On the other hand, homophily, or the tendency of individuals to associate similar others, is common in cultures across the world – after all, birds of a feather flock together.