Leela-chess: [Suggestion] Efficient opportunistic search option on root node

[ThomasCabaret]

Hi, I was thinking a lot about the impact of -v800 on the training, more or less convinced that trying to learn good data is better than successfully learn "bad" data and I came up with what seems to be an "quick win" idea, a way to do more than -v800 without doing it :).

The trick would be to force the search to keep going on root node if there is a direct child node better than the value of the current best + some %threshold.
By example, lets say the current best says 55% winrate and that there is another node, freshly discovered, with a winrate of 57% but really few visits and the end of the v800 is close. With a 1% threshold the idea would be to keep the search going because 57% > 55% + 1%, forcing the first move to explore this "opportunity" move _(and add another security threshold on the total visits to be sure we do not stays totally stuck)._
I think it can promote significantly good thoughts of the UCT, because sadly even very good moves seen during the search and not initially well scored by the policy are heavily penalized.

With a good threshold it will impact probably less than 10% of the moves but might make a huge difference in the elo _(and so produce better data)_ if avoiding a tactical trick that in fact had been seen but unfortunately not selected. Even if the search is 4 times slower on those specific moves it will not impact that much the training games playing rate.

What do you think about the idea?

[jjoshua2]

It sounds a lot like the puct/fpu tuning we are working on. If you make a PR, we can test that. Although it sounds like a bit of fpu and lowering puct is what you are talking about...

[ThomasCabaret]

I would say puct is more about convergence speed, and fpu is more about move not evaluated yet but I will have a close look, thx.

[jjoshua2]

They are both about exploiting vs exploring

[Akababa]

If I understand correctly this issue is more about time management than puct tuning. It proposes dynamically increasing the number of playouts based on the stability of the search, which has shown promise (among other things) in some MCTS-based Go programs (according to https://dke.maastrichtuniversity.nl/m.winands/documents/time_management_for_monte_carlo_tree_search.pdf)

Perhaps a simple way to implement this is by adding a safety margin for pruning, where candidate moves only get cut if they're trailing by a certain amount, and the search continues until only one move remains.

[oscardssmith]

The only potential downside is that it could keep the search going indefinitely if it gets into situations where two moves are legitimately of similar strength. Some cutoff would definitely be needed.