Will OpenAI's o4 get above 50% on humanity's last exam?
Plus
40
Ṁ73952027
13%
chance
1D
1W
1M
ALL
Resolves N/A if there is no o4 model. o4 is defined as any compute setting on the o4 model. Something like deepresearch (which is based on o3/o4) would also resolve yes.
Update 2025-04-17 (PST) (AI summary of creator comment): o4 mini Exclusion Clarification
o4 is defined as any compute setting on the o4 model.
o4 mini is explicitly excluded from being considered as o4.
Get
1,000and
1.00
1,0001.00Sort by:
They have to almost 4x the o4-mini score for this to happen, so definitely unlikely. However, given how much they were willing to spend on compute to get an unexpectedly high score on a similar high profile benchmark with o3 earlier it could happen, especially given a few months more of tinkering.
12% was simply a bit too low
Related questions
Related questions
What will be the best AI performance on Humanity's Last Exam by December 31st 2025?
When will OpenAI announce o4 (full)
Will OpenAI o1 (or any direct iteration like o3) get gold on any International Math Olympiad by the end of 2025?
29% chance
Will the first AI model that saturates Humanity's Last Exam be employable as a software engineer?
31% chance
Will xAI rank above OpenAI at EOY?
30% chance
Will OpenAI's next major LLM (after GPT-4) surpass 70% accuracy on the GPQA benchmark?
75% chance
Will OpenAI's next major LLM (after GPT-4) surpass 74% accuracy on the GPQA benchmark?
86% chance
Humanity's Last Exam score in 2025?
-
Will "OpenAI o1" make the top fifty posts in LessWrong's 2024 Annual Review?
8% chance
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
41% chance