The question I am asking here is whether a "traditional" Large Languge Model resembling the GPTs will defeat the World Champion of Chess of that time (eg Magnus Carlsen), before 2035.
I define a traditional LLM as a neural net model primarily trained using unsupervised text completion (like GPT or BERT) and then optionally fine tuned afterwards as chatGPT and Bing were.
To qualify, the training corpus cannot be deliberately synthetically generated from the output of superhuman computer models not themselves meeting these criteria (eg AlphaZero or Stockfish)
Also, to qualify, this LLM must be a pure text generation model. It is not permitted to do any brute forcing of move trees (as AlphaZero does). The moves must simply be the completions given by the text AI when presented with the board state and history.
If the world champion declines to play the LLM, then the question will devolve down to the highest rated player who plays such a match, provided they are ranked in the Top 10 worldwide. If none are willing to play, this question will resolve to the best estimate I can get from prediction markets as to who would win if they did play.
1,000
1.00To be clear, it has come out that OpenAI specifically trained GPT-4 on chess using fine tuning after the main training, purposely removing games under a certain ELO and including a huge number of games that were not randomly in the base training data.
Based on the description, it seems this type of training is sufficient to allow this market to resolve yes, if the model achieves it.
@YaakovSaxon I actually had a similar, perhaps broader question (one that also relates to the question by @TaoLin ): how would we know that the LLM in question works according to the specification of this market? For example, OpenAI is anything but open, so no one really knows anymore neither how ChatGPT/Bing works, nor what goes into its training!