An important message from open source chess organizations.
A few days ago, ChessBase released Fat Fritz 2, described on their website as the “new number 1” chess engine “with a massive new neural network, trained by Albert Silver with the original Fat Fritz.” They advertise Fat Fritz 2 as using novel strong ideas compared to existing chess engines, but in reality Fat Fritz 2 is just Stockfish with a different neural network and minimal changes that are neither innovative nor appear to make the engine stronger.
As this is not the first time something like this has happened involving both ChessBase and A. Silver, we would like to share our impression of these releases.
[EDITORS NOTE: Since this article's publication Chessbase has removed the claim that Fat Fritz is "the new #1" from their website. You can see an archive of the old version here.]
In July 2018, Silver secretly sent the Leela Chess Zero engine with a custom neural network he had trained to participate in TCEC under the name DeusX. The network was trained using scripts from the Leela project, and the network architecture was the same as that used by the Leela engine (see this older Leela blog post for details). Training such a network is not unusual; members of the Leela project do so regularly to test ideas. But they have never pretended to have created a new engine by training a new network.
While an overwhelming part of DeusX’s strength is inherited from Leela, Silver downplayed the Leela work enormously in an interview, suggesting that he did in a few months what had taken other engine authors decades. Silver described himself as the engine author even though the engine itself was Leela without significant modification.
The following year, Silver released an updated version of the DeusX network under the name Fat Fritz, sold as a part of ChessBase’s Fritz package for €79.90. Once again, it used the Leela engine without functional changes (the changes made included modifying the name and author strings, and some default parameter values).
Fat Fritz was marketed as if it were an innovative engine, instead of being just a renamed Leela. As an example, the product description begins, “It’s a semi-secret development, an AlphaZero clone, engineered over the past nine months,” and doesn’t mention Leela. Probably the closest to what can be called an “attribution” is a brief mention in the middle of one of the articles, saying that Fat Fritz uses Leela “as a foundation.” In reality, Fat Fritz is Leela, but with a different net. Even this article begins by describing an “inspiring” talk given by a DeepMind employee to ChessBase programmers, supporting the false impression that ChessBase played a significant role in the development of the Fat Fritz code.
In ChessBase articles, the Fat Fritz “engine” was described in a way that implied it was stronger than Stockfish and Leela, but the evidence was questionable. Silver’s Stockfish comparison, for example, used an outdated version of Stockfish even though the development version was known to be considerably stronger. Similarly, when compared to Leela, the strongest configuration of Leela was not used.
If your idea of innovation in chess is charging 100 EUR for changing the parameters of an open source engine, you're going to have some problems competing with https://t.co/omt8vnPgDW and https://t.co/YYvW1XZRCj.
— Gian-Carlo Pascutto (@gcpascutto) February 9, 2021
Gian-Carlo Pascutto is an author of several strong Chess and Go engines and contributor to the Stockfish and Leela Chess Zero projects.
In 2020, Stockfish, Leela’s main competitor, started to support NNUE, fast neural networks that can run on a CPU. This feature improved Stockfish significantly, restoring its status as the strongest existing chess engine.
The Stockfish team had the same painful experience as the Leela team when Silver decided to jump on the hype train again, and released Fat Fritz 2, sold by ChessBase for €99.90. It is now Stockfish that has been copied instead of Leela, but the overall style is unchanged:
It is sad to see claims of innovation where there has been none, and claims of improvement in an engine that is weaker than its open-source origins. It is also sad to see people appropriating the open-source work and effort of others and claiming it as their own.
Everyone is permitted and encouraged to modify and improve code from Stockfish/Leela while giving credit; that is the intent of open-source software. Everyone is allowed to copy Stockfish/Leela and sell them, provided the terms of the Stockfish/Leela license are met. But don’t pretend that the product being sold is something it isn’t.