Will Al achieve 95% or higher score on the FrontierMath benchmark before 2030?
Mini
3
Ṁ85
2029
73%
chance

From a recent arXiv preprint,

We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians. The questions cover most major branches of modern mathematics -- from computationally intensive problems in number theory and real analysis to abstract questions in algebraic geometry and category theory. Solving a typical problem requires multiple hours of effort from a researcher in the relevant branch of mathematics, and for the upper end questions, multiple days. FrontierMath uses new, unpublished problems and automated verification to reliably evaluate models while minimizing risk of data contamination. Current state-of-the-art AI models solve under 2% of problems, revealing a vast gap between AI capabilities and the prowess of the mathematical community. As AI systems advance toward expert-level mathematical abilities, FrontierMath offers a rigorous testbed that quantifies their progress.

This question resolves to YES if the state-of-the-art average accuracy score on the FrontierMath benchmark, as reported prior to January 1st 2030 is 85% or higher for any fully-automated computer method. Credible reports include but are not limited to blog posts, arXiv preprints, and papers. Otherwise, this question resolves to NO.

Clarification: There is no maximum inference budget for this benchmark - models can use any amount of compute to solve the problems.

Get
Ṁ1,000
and
S1.00