There are claims that have gone viral that an early GPT model, GPT-3.5-turbo-instruct, can beat the Stockfish program at chess when Stockfish is playing at an ELO 1800 rating. An ELO rating of 1800 signifies a highly skilled chess player.
If the claims are true, this would be astonishing considering that GPT was not trained to play chess, it was trained to predict the next word. There is no doubt that GPT has read a ton about chess, but playing at that level would require it to reason about positions it will have never seen or read about, and would mean that an advanced capability to search ahead and perform advanced inferenced emerged from the training.
Lonnie Chrisman put this to the test. Using the Analytica OpenAI API library, along with an Analytica chess library, he created a model to play chess against GPT.
Watch this video to see how well three different GPT models (GPT-3.5-turbo-instruct, GPT-3.5-turbo and GPT-4) did playing against him, and whether the claims have merit.
But here's the exciting part: Using the Analytica model, Chess with GPT.ana (from Analytica Enterprise), you too can challenge GPT in a game of chess.
Perhaps even more intriguing, you can look into the inner workings of the model, observing prompts for GPT and the internal reasoning and responses generated by GPT as it selects its moves.
This unique peek into prompt creation and the information used may offer valuable insights into integrating Large Language Models (LLMs) into other various applications.
