Will a text model achieve 100% performance on the MMLU in five years?
Mini
3
Ṁ402028
28%
chance
1D
1W
1M
ALL
This video predicts that a model will achieve 100% performance on the Massive Multitask Language Understanding Benchmark.
This will resolve YES if a model is listed as having achieved 100 percent performance on the MMLU on papers with code, in a technical document, or published article by 3:00 PM EST June 28, 2028. This will resolve NO if a model is not listed as ahving achieved 100 percent performance on the MMLU.
Get
1,000and
1.00
1,0001.00Related questions
Related questions
Which company has best Text-to-Image AI end of 2025? (Artificial Analysis Leaderboard)
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
60% chance
Will an open-source text-to-music LLM model be able to create an entire album as convincing as a human by end of 2025?
26% chance
Which company has best Text-to-Video AI end of 2025? (Artificial Analysis Leaderboard)
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
62% chance
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance
Will a team consisting primarily of ML models complete MIT Mystery Hunt by 2030?
45% chance
End of pre-training era for language models: Will an LM fine-tune for more FLOPs than it is pre-trained for, before 2026
44% chance