BenchMIRT: What are LLM benchmarks actually measuring?

AI2 Questions What LLM Benchmarks Actually Measure

The Allen Institute for AI (AI2) has released BenchMIRT, a new method for auditing large language model benchmarks at the level of individual prompts. The research demonstrates that many standard evaluations, often designed to measure a single capability like safety or reasoning, are actually testing a complex mix of underlying skills. This finding challenges the industry's reliance on single-score leaderboards, suggesting that a model’s high rank on a benchmark may not accurately reflect its specific intended capabilities.

Under the Hood: A Psychometrics Approach to AI

BenchMIRT applies multidimensional item response theory (MIRT), a technique from psychometrics, to analyze LLM performance. Researchers at AI2 trained the system on results from 100 LLMs across 16 benchmarks, covering more than 34,000 questions. Without being told what the benchmarks were for, the model independently identified two dominant dimensions driving performance: general reasoning and safety. The analysis revealed several counterintuitive findings:

• The BBQ benchmark, designed to test for social bias and typically grouped with safety evaluations, was found to align much more strongly with general reasoning.
• Similarly, WMDP, a test for dangerous dual-use knowledge, was also more closely associated with reasoning. Stronger reasoning ability correlated with lower (safer) scores, as models correctly refused to answer.
• HarmBench, a single benchmark, was shown to contain different signals; its standard and contextual prompts mapped to safety, while its copyright-related questions correlated more with general reasoning.

More Efficient and Transparent Evaluation

The primary impact of BenchMIRT is its potential to create smaller, more focused, and more transparent evaluations. The analysis can identify the most informative questions within a benchmark, allowing for significant streamlining. According to AI2, using just the top 10% of questions identified by BenchMIRT generally preserved the overall model rankings from the full benchmark. While this allows for more efficient testing, the authors note a key risk: the same tools could be used to identify and remove a benchmark's most challenging safety questions, creating a weaker test that an unsafe model could pass.

Strategic Takeaway: BenchMIRT's analysis is a critical reminder that single-metric leaderboards oversimplify complex model capabilities. Enterprises relying on these scores for model selection must look deeper, as a high rank in a 'safety' benchmark could be driven more by a model's reasoning skills than its actual alignment safeguards.