Leela-chess: Remove history planes

[mooskagh]

That's not something to do immediately (probably, to avoid having too many moving parts), but I think that eventually we should do that.

Either remove all history planes (and encode en passant possibility some way),
or keep a history of one half-move.
Repetitions would be handled by MCTS.

Currently having the same position with different histories can change NN output a lot (I've seen probabilities to change by 15%), while they should be more or less the same.
Having unnatural history planes (like empty, or flipped) makes NN output look even weirder, sometimes seemingly totally random.

That is also the reason why enabling NN cache makes the engine play weaker. Disabling history planes would make it play at the same level while having benefits of the cache.

Also there are people who run LCZero to check on tactical puzzles which are fens with no history, and expect that to work.

(I wonder if lczero should reject position fen ... without moves with at least moves in it, that way more people will be aware that it's not supposed to work)

[Akababa]

Perhaps one thing to consider is that the history planes encode some information about the opponent, which isn't really part of "pure" chess but tends to be something humans use in practice. For example, repeating a position to see how the other side responds.

I'm interested in this too, but some way of encoding repetitions will probably be needed. One half-move with repetition count should be enough to retain the most important information in the history planes, but might also have the same drawbacks..

[luigio]

We should allow the possibility of Leela becoming strong without lookahead. I think history planes are necessary for that, as they prevent Leela accidentally incurring in repetitions in winning positions. In a perfect information game, players should have full knowledge of the game state, and, because of the repetition rule, the list of previous repeatable positions (i.e. starting from the last capture, pawn move or castling) is part of the game state just like the current position on the board.

Perhaps one thing to consider is that the history planes encode some information about the opponent, which isn't really part of "pure" chess

One could argue that the same is true for the current board position.

[luigio]

We should allow the possibility of Leela becoming strong without lookahead. I think history planes are necessary for that, as they prevent Leela accidentally incurring in repetitions in winning positions.

Counter-argument:

Let's say history planes aren't used and Leela plays with no lookahead at all. Position A, with Leela to move, is on the board. Leela thinks, rightfully, that it's a winning position and makes move X. 10 moves later, Leela finds itself in position B, where no good options exist except for move Y, which recreates position A. Move Y is made. Now, for Leela to avoid a second repetition and win, all it needs to know is the move it made in position A the first time so it can choose a different one this time. So I think the history planes could be replaced with a single plane that stores the move made in the previous occurrence of the current position, if there was one. If the current position hasn't appeared before, the plane remains empty.

With regard to repetitions, this results in exactly the same behavior as full history planes, as far as I can see, and it's better than the 8-ply history planes that we're using now. The first time Leela finds itself in position A, even full history planes would provide no indication that move X leads to a repetition, as the relevant history is the "history" after X, not before. Only the second time will it choose a different move, just like in the scenario with the alternative move-in-repeated-position plane.

Of course, even without either of those forms of history, the first time position A appears, an ideal net would know intuitively that move X leads to a repetition, and so it would choose a different move right away. But I think it's unrealistic to expect anything other than a really big net to reliably have good intuitions about this, so the move-in-repeated-position plane seems a good real-world compromise to me.

(As an aside, there is one case that can't be covered by either history planes or the move-in-repeated-position plane: a position A where moves X and Y lead back to position A and move Z wins instead. If Leela prefers X and Y over Z, it will inevitably end up drawing by threefold repetition. Only improving its intuitions to the point of preferring Z over X and Y or using a fourfold instead of threefold repetition rule will enable it to win in this case.)

[renouve]

I am in favor of removing it, it cause different evaluations for the same positions, but with different moves to reach it.

[killerducky]

• Removing history planes would be nice for doing analysis using FENs which have no history. But it does seem analyzing FENs with no history is not horribly broken.
• I think removing the history might make the engine weaker.
  
  
  
  • Deepmind proved that including history can make an engine stronger than SF8. There is no evidence what will happen when they are removed.
  
  
  • Mathematically the history does not change the eval (outside of repetitions). But that is not interesting because we cannot solve chess mathematically. We are trying to make a strong engine and imperfect heuristics are useful if they make the engine stronger.
  
  
  • Giving different evals for different histories of the same position does not prove that removing history completely will not make the engine weaker. It's just an annoying side effect of including history.
  
  
  • We should try to get as much evidence as possible before changing this.
  
  

I propose "someone" take the existing training data and train a net that does not use history. Then we can measure the Elo difference. This can be done by one person. From there we can decide if it's worth spending some RL cycles on it. Unfortunately that part is expensive because it will probably require a long time of our entire distributed effort to see what happens.

[gcp]

I propose "someone" take the existing training data and train a net that does not use history. Then we can measure the Elo difference.

I think this is a key part and good to do. For training go networks, the past moves provide context on the position that seems to boost the performance of networks (and not just capture ko) because they don't have to explicitly spend filters and layers recomputing it. The effect may be small to negligible in chess, but I wouldn't just assume that is the case without testing.

If you play chess, do you not analyze as "what did the last move change about the position"?

From there we can decide if it's worth spending some RL cycles on it.

I think if you retrain 2 networks from the same data, you get an answer directly without needing to do RL.

[Error323]

I think if you retrain 2 networks from the same data, you get an answer directly without needing to do RL.

Yes one can use the bootstrapping method and a sliding window to see how it evolves. The tricky part is writing the code that properly adapts the trainingdata to be history-less. It requires an en passant square to be distilled and the three-fold repetition square/piece in question.

I still think it's an interesting idea. It'll give the 3x3 filters more focus.

[killerducky]

See below some results rcostheta had in Discord back in 2018-07-18. Removing history (actually zeroing the planes out) made Training stats were worse, and Elo was worse.

https://discordapp.com/channels/425419482568196106/425419999096733706/469127055838150657
rcostheta 2018.07.18
With history: step 20000, policy=1.96913 training accuracy=55.1609%, mse=0.110399
No history: step 20000, policy=1.9728 training accuracy=54.5028%, mse=0.115069

https://discordapp.com/channels/425419482568196106/425419999096733706/469128193627127808
rcostheta 2018.07.18
I have 2 fairly similar networks: 1 with history and 1 with history planes zeroed each training step
http://webphactory.net/lczero/scs-nh2-256x20-20000.txt.gz - no history
http://webphactory.net/lczero/scs-wh1-256x20-20000.txt.gz - with history

https://discordapp.com/channels/425419482568196106/425419999096733706/469145414981910528
ghotrix 2018.07.18
@rcostheta Score of wh1 vs nh2: 55 - 27 - 18 [0.640] 100 Elo difference: 99.95 +/- 64.44 Finished match