Which GPT-4?

@firstuserhere does it need to be as good as the best GPT-4, or just as good as any of the GPT-4 models?

Would you have called Llama open source and would it have resolved this market to yes if it was as good as you want?

@Seeker Yes, i know LLaMA isn't truly open source but it would've qualified for the purposes of this market.

i find it interesting that this market is at ~27% although mine is at ~70% - mine focuses on the full year and only relies on one metric to resolve while this one is only till june and relies a bit on subjective definition so maybe this is why

The extended version of this market, for the entire 2024

Why does this close in January?

I think an Elo ranking (https://arena.lmsys.org/) could be used to determine the winner objectively. Interestingly, Mistral-Medium is on par with GPT 3.5 in terms of elo :O

'Mistral-Medium outperforms GPT-4 in Winogrande benchmark lmao'

https://twitter.com/yupiop12/status/1734137238177698106

@Dom95cc The Mistral/Mixtral models only seem good in very particular ways. Medium only scores 75 on the MMLU.

@firstuserhere GPT-4 as it is then or as it is at market open?

@TobiasH the benchmark results for GPT-4 from its report at the time of release, and qualitative baseline of today's GPT-4