Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
Mini
3
แน75Jan 1
25%
chance
1D
1W
1M
ALL
Will any company release a pure Mamaba LLM that scores over 82 points on the standard full MMLU (5-shot) using https://github.com/EleutherAI/lm-evaluation-harness with open source weights?
Get
1,000and
1.00
1,0001.00Related questions
Related questions
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
37% chance
๐ OpenLLMs: Will Any Open Source LLM on the HuggingFace OpenLLM Leaderboard Significantly Gain in Avg Score by YE 2024?
49% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
79% chance
Will there be an open source LLM as good as GPT4 by the end of 2024?
68% chance
What will be the maximum context length of any OpenAI LLM by EOY 2024?
Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?
20% chance
Will an open-source LLM beat or match GPT-4 by the end of 2024?
85% chance
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
What will be true of OpenAI's best LLM by EOY 2025?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance