What will be true of Gemini 2?
Mini
19
Ṁ1213
2027
99%
It will be released in 2024
99%
It will be released before GPT-5
76%
It will score >70% on GPQA (diamond, pass@1)
72%
>20% (5-shot) on SWEBench (current sota ~ 6%?)
59%
It will get into at least one PR blunder, like the woke issue for Gemini 1 Ultra
58%
It will score better than o1-preview at AIME (44.6% in single pass)
54%
It (the strongest variant) will be better than the best OpenAI model at the moment of release
52%
It will score >= 85% on MMLU Pro
47%
It will score >=80% on MMMU
31%
It has input modalities other than text, audio, and video (MIDI/code count as text)

Get
Ṁ1,000
and
S1.00
Sort by:

huh, not sure how the benchmark questions should resolve now that 2.0 Flash is out. Maybe wait for info on 2.0 pro? Also relevant that it is called "2.0 Flash Experimental" for now

https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/#gemini-2-0-flash

Current leading benchmarks, excluding o1 (o1-preview gets 73% on GPQA)