> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
I don't buy this. I've seen GMs playing online while verbalizing their thought process (on twitch / youtube videos where they livestream their play), and they consider way more branches than I do, while also going deeper than I ever would on some branches. My top rating in lichess blitz is about 1900, not bad but way lower than a GM.
Yeah I would like to see where that factoid comes from. Maybe it means they explore as many branches from the root position as an amateur, but they explore each branch way deeper and more accurately
It makes sense. The space of moves is a tree. If you're doing a breadth-first search and a grandmaster is doing a more depth-oriented search with heuristics for pruning low value branches, then the same total number of moves could be considered, but the grandmaster is reaching the better moves before you.
No, a grandmaster can also consider moves way faster than an amateur. Watch Hikaru Nakamura solve chess puzzles [1]. He has already solved the puzzle and moved on to the next one before an amateur can even see the pieces on the board.
1940s to 1980s papers (the source of the claims) might think "amateur" means something different from people today. (they probably mean club chess players)
It seemed to me that in the chess community, average club players are usually considered (much) stronger than the average online player. (Especially because of the much larger number of lower skill players online.)
So if you measure amateurs via club players (=what people might have considered chess amateurs before), that might give you a different result from the average online player (=what people might consider average amateurs today).
It is much easier to imagine the 1600 FIDE OTB player doing 20-40 positional evaluations than the 1200 random chess.com scrub (eg me) doing the same.
GMs also have access to a greater number of memorized situations. Regardless of whether they are conscious of it or not, they have previously analyzed so many moves that they can cut off branches they just know instinctively to be dead ends. Machines could do this, directly reference a massive dataset of precomputed board positions, but that isn't considered a proper chess engine... despite the fact that human GMs do use such techniques.
Cool exploration, and very much needed because depressingly few people who use these engines actually understand this.
Back in 2021 I got cfish and weiss working on a 386sx, though didn't test it much. cfish was doing 5k nps so some results can be extrapolated. NNUE complicated things due to RAM requirements. Though I did train a small one for lichess at some point and they can be surprisingly good.
Great project but I have a nit on the board displays: every board is displayed as <human username> vs Pentium Pro, even when the Pentium is playing White. The normal chess convention is <White name> vs <Black name>.
I spent some time trying to figure out how some of the humans seemed to have trapped the machine out of the opening, because it looked like the human was winning.
> because who would want to buy a Pentium Pro when the better-in-almost-every-way Pentium MMX
Maybe this was satirizing people running after every shiny new thing, but the MMX certainly wasn’t. The Pro was the first P6, a totally new, out-of-order microarchitecture that the Pentium 2 would be based on. The MMX was a standard P5 Pentium with a larger L1 and the MMX vector instruction set which, while the first of its kind in an x86, was ultimately not used much in applications, partly because it was integer-only. In any case, the Pros were expensive and meant for business use, while the MMXs were consumer processors.
Very interesting. I don't play chess but like to observe others playing and talking about it on YouTube.
I thought about something the other day... Modern chess sites run proprietary algorithms to detect cheating by modern engines. But would it be possible for an old engine, let's say from the 70s or 80s, running on modern hardware to deceive these algorithms?
They wouldn't win every tournament but still have a respectable performance.
From what I gather cheat detection uses a combination of factors to determine how "human-like" an account is: time usage, webpage interactions, move quality under time pressure and so on. I think you could probably try to deliberately deceive a specific website's cheat detection with software that tries to emulate how humans play, at which point you'd be in the classic attack vs. defense arms race.
Also notice that for events like Title Tuesday you must play on a dedicated client with at least 2 cameras, so it would be significantly harder to pull it off.
With AI gaining consciousness I can't wait for such "human-only" measures to be delegated to the trashcan of history, same as slavery. What happens when AI-human cyborgs start walking around? Will they also be treated as second class citizens?
We divide competitions into classes (by sex and age is common, for example "17 yo boys", and similar competition classes), so I don't see why it shouldn't be subject to the same kind of class distinction, maybe with an without augmentation.
> But yes, totally unsurprising that a megahertz machine with 100MB+ available ram has >2500 ELO.
I think this is underselling it a bit. If you'd told this to the IBM engineers at the time I think they would have taken great interest in how the performance was achieved.
I grew up with these machines around, and as a kid they seemed to so much more powerful than humans. It's kinda weird to think that I became 'smarter' than them eventually. I suspect they will be able to run superhuman AI one day too.
I don't think that real-time, superhuman AI performance is ever gonna happen with <1b parameter models.
Biology had a lot of time, attempts and pressure to make this work but even the most optimized lines (something like a magpie) need more complexity (almost a billion neurons) to get a fraction of superhuman intelligence.
Results with small artificial neural nets are very limited so far, too.
I'll go out on a limb and say that future AI models, running on today's smartphones, will be better at programming and formal math than today's frontier models running in datacenters. And by "future" I mean less than 5 years from now.
> (Also, uh, if anyone has any ideas for projects I could do with a Pentium Pro in a few weeks once I no longer need it to host this chess engine ... do let me know!)
Throw a 3dfx Voodoo in there and do some real-time 3D Gaussian Splatting!
When I tried this with a 6502 (Apple II), it did not work. It was just too underpowered for enough modern algorithms to kick in. It might also be an LLm-driving skill issue of course!
> As it turns out, the majority of the Elo gains (+720 of the +1335 gained) can be attributed to better programming, and not better hardware.
It's not so easy, a lot of that better programming can be attributed to better hardware, it wouldn't have been possible to implement NNUE or Fishtest without modern hardware.
In one sense, Knuth fans and really good engineers are correct to prioritize algorithms.
Unfortunately, most humans see big|new|expensive hardware as great status signifiers, barriers to lesser folks joining the Cool Club, and optimal sources of sales commissions.
I wish disclaimers like that would be at the start and not halfway trough the text.
> the average chess grandmaster during a classical game will analyze roughly the same number of moves as an amateur player, but they analyze only the most critical moves and do a better job with that analysis.
I don't buy this. I've seen GMs playing online while verbalizing their thought process (on twitch / youtube videos where they livestream their play), and they consider way more branches than I do, while also going deeper than I ever would on some branches. My top rating in lichess blitz is about 1900, not bad but way lower than a GM.
[1] https://youtu.be/4EfSymvT59M
So if you measure amateurs via club players (=what people might have considered chess amateurs before), that might give you a different result from the average online player (=what people might consider average amateurs today).
It is much easier to imagine the 1600 FIDE OTB player doing 20-40 positional evaluations than the 1200 random chess.com scrub (eg me) doing the same.
Back in 2021 I got cfish and weiss working on a 386sx, though didn't test it much. cfish was doing 5k nps so some results can be extrapolated. NNUE complicated things due to RAM requirements. Though I did train a small one for lichess at some point and they can be surprisingly good.
I spent some time trying to figure out how some of the humans seemed to have trapped the machine out of the opening, because it looked like the human was winning.
Maybe this was satirizing people running after every shiny new thing, but the MMX certainly wasn’t. The Pro was the first P6, a totally new, out-of-order microarchitecture that the Pentium 2 would be based on. The MMX was a standard P5 Pentium with a larger L1 and the MMX vector instruction set which, while the first of its kind in an x86, was ultimately not used much in applications, partly because it was integer-only. In any case, the Pros were expensive and meant for business use, while the MMXs were consumer processors.
I thought about something the other day... Modern chess sites run proprietary algorithms to detect cheating by modern engines. But would it be possible for an old engine, let's say from the 70s or 80s, running on modern hardware to deceive these algorithms?
They wouldn't win every tournament but still have a respectable performance.
https://c64online.com/c64-games/sargon-ii-de/
Also notice that for events like Title Tuesday you must play on a dedicated client with at least 2 cameras, so it would be significantly harder to pull it off.
Yes, along side you robosexuals
I think this is underselling it a bit. If you'd told this to the IBM engineers at the time I think they would have taken great interest in how the performance was achieved.
Biology had a lot of time, attempts and pressure to make this work but even the most optimized lines (something like a magpie) need more complexity (almost a billion neurons) to get a fraction of superhuman intelligence.
Results with small artificial neural nets are very limited so far, too.
I had a Pentium Pro 200 back in the day, to run 3DSMax 1.0 on Windows NT 3.5.
Throw a 3dfx Voodoo in there and do some real-time 3D Gaussian Splatting!
And, with more work, that's true of computers in general; they can all do the same computations once you somehow wire up enough memory: https://dmitry.gr/?r=05.Projects&proj=35.%20Linux4004
For chess though, there are time constraints. The 386 is a lot slower. Is it fast enough?
https://github.com/zellyn/8fish
It's not so easy, a lot of that better programming can be attributed to better hardware, it wouldn't have been possible to implement NNUE or Fishtest without modern hardware.
Unfortunately, most humans see big|new|expensive hardware as great status signifiers, barriers to lesser folks joining the Cool Club, and optimal sources of sales commissions.