Statisticians Have Finally Cracked the Code to Resolve Conflicting Sports Rankings

Determining the definitive “best” in any field is rarely straightforward, as evidenced by the often contradictory lists produced by sports analysts. When media outlets offer vastly different top-tier rankings for professional athletes, identifying a true consensus becomes a statistical minefield. A new collaborative study from Rice University and Cornell University aims to cut through this noise with a sophisticated analytical framework designed to harmonize disparate data.

The research, published in the Journal of Quantitative Analysis of Sports, introduces a method called Bayesian Multivariate Rank Regression (BMRR). Unlike simple averaging, which can be skewed by missing entries or extreme outlier opinions, BMRR treats ranking data as a complex evidentiary problem. By utilizing a hierarchical Bayesian structure, the model accounts for incomplete lists, varying levels of agreement among sources, and fluctuations over time, ultimately providing a measure of certainty regarding the final ranking.

“Rankings look simple on the surface, but statistically they’re actually very complicated,” says Rose Graves, a doctoral student at Rice and the study’s corresponding author. “Two experts may rank different numbers of items, disagree about the order, or even change their opinions over time. We wanted to create a way to bring all of that information together while accounting for uncertainty.”

To validate the model, the researchers applied it to four years of preseason top-100 lists for Major League Baseball (MLB) players, sourced from five prominent media outlets: ESPN, CBS, Bleacher Report, Yahoo Sports, and MLB.com. By focusing on 55 players who appeared in these lists consistently between 2021 and 2024, the team was able to map not only the consensus on player value but also the specific trajectories of individual careers.

The analysis revealed that while media rankings are heavily influenced by a player’s Wins Above Replacement (WAR) stat, they also lean significantly on age and salary. This suggests that these rankings serve as a hybrid metric, blending recent on-field production with predictive expectations about future performance. The model successfully highlighted notable career arcs, such as the rapid climb of Shohei Ohtani, who went from being largely omitted from top-100 lists in 2021 to a perennial consensus top-10 selection by 2024.

Beyond identifying the “top” players, the BMRR framework offers unique insights into the behavior of the rankers themselves. The researchers found that some outlets, such as Yahoo Sports, showed a higher tendency to favor younger, up-and-coming talent, whereas others aligned more closely with the overall aggregate. This granular view allows for a deeper understanding of how specific biases influence public perception.

The practical applications of this research extend far beyond the ballpark. Because ranking systems are used in everything from search engine algorithms and political polling to intelligence assessments and clinical decision-making, the ability to quantify uncertainty is crucial. In professional sports, where these rankings can influence high-stakes contract negotiations and trade decisions, knowing the degree of confidence behind a specific ranking is as valuable as the ranking itself.

“Rather than asking only who ranks first, this framework lets us ask how confident we are in that conclusion, why someone may be ranked highly, and how much agreement actually exists among the people doing the ranking,” says Marina Vannucci, a professor of statistics at Rice. “That kind of uncertainty quantification can be extremely important when rankings are being used to inform decisions.”

For researchers and data scientists interested in applying this approach to their own datasets, the team has made the BMRR code available on GitHub. As Dan Kowal, a former Rice faculty member now at Cornell University, notes, the challenges inherent in ranking data remain remarkably consistent across diverse industries, suggesting that this new tool could provide clarity wherever conflicting opinions vie for authority.