There's this piece that was published recently and claims that, and generated a bunch of controversy on social media.
The claim itself is unclear, but since the article's argument is that Google has much more hardware than OpenAI, I will be ironmanning the author's argument. And interpret it as meaning that Gemini is trained on 5x as "effective compute". I define effective compute as compute resources used for training, accounting for differences in efficiency. This might be something like FLOPs, but accounting for the fact that I expect Bard to be trained on TPUs and not GPUs, examples seen since training, etc. I will try to judge as objectively as possible but some subjectivity is unavoidable so I will not bet on this market. When information that allows me to judge this is made publicly available, this will resolve. If that doesn't happen until the end of 2024 (since by then the models will have significantly changed), this market will resolve N/A.
Please note that 5x compute does not mean that it generates text that is 5x better, whatever that would mean.
1,000
1.00The description of this question doesn't make sense.
>I define effective compute as compute resources used for training, accounting for differences in efficiency. This might be something like FLOPs, but accounting for the fact that I expect Bard to be trained on TPUs and not GPUs, examples seen since training, etc.
You should just ask about the difference in FLOP.
@TamayBesiroglu I might have asked about FLOPs if we knew that... Sadly there is extremely scarce information about the training of top LLMs. Also I don't know what's the correct way to compare TPU vs GPU performance, I'm not sure if's that simple. I was trying to expand this such that any suitable criterion can be used.
@Shump TPUs and GPUs both perform floating point operations. You don't need to make any additional adjustments based on their computational performance.
@TamayBesiroglu So do CPUs. But comparing them to GPUs is still tricky. Actual running time depends on more than just that.
I don't understand your point. I suggest checking out some resources about this, such as this report.
Did anyone find relevant info for this? Couldn't find anything concrete in the technical report. Market seems underpriced rn.
Traders please note: I haven't gotten much feedback on the resolution criteria, so I am making a decision.
Since I don't trust that any of these metrics will be public at any point, and each of these have flaws, the resolution is changing and will be slightly subjective. I have therefore exited the market and will not bet on it. See above for the updated resolution criteria.