Stockfish: Crazyhouse move to avoid, 2 interesting positions

[Vinvin20]

5...Qb6?? : https://fr.lichess.org/N9N5mZ6H#9

23...Qh4?? https://fr.lichess.org/fBXBdtqF/black#45

[ianfab]

In the first game Stockfish overestimates the danger for the white king:

setoption name UCI_Variant value crazyhouse
position fen rnbqk1nr/pp3ppp/4p3/2bpP3/6Q1/8/PPP2PPP/RNB1KBNR/Pp b KQkq - 9 5 moves d8b6 g4g7 c5f2 e1d1 f2g1 g7h8
eval
info string variant crazyhouse startpos rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR[] w KQkq - 0 1
      Eval term |    White    |    Black    |    Total    
                |   MG    EG  |   MG    EG  |   MG    EG  
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  1.99  0.91 
      Imbalance |   ---   --- |   ---   --- | -0.60 -0.60 
          Pawns |   ---   --- |   ---   --- |  0.06 -0.00 
        Knights |  0.06  0.00 |  0.06  0.00 |  0.00  0.00 
         Bishop | -0.10 -0.29 | -0.16 -0.29 |  0.06  0.00 
          Rooks | -0.56  0.00 |  0.00  0.00 | -0.56  0.00 
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
       Mobility |  0.16  0.18 | -0.10  0.04 |  0.26  0.14 
    King safety | -6.90 -6.69 | -2.88 -2.75 | -4.02 -3.93 
        Threats |  1.00  0.64 |  0.00  0.23 |  1.00  0.41 
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Space |  0.34  0.00 |  0.30  0.00 |  0.04  0.00 
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- | -1.77 -3.11 

Total Evaluation: -2.28 (white side)

In the second game, it does not spot the mating sequence involving several sacrifices. This probably has to do with not extending checks with negative SEE. I tested a patch on that a few weeks ago and it failed, but I have just discovered a bug in it, so I will retry it with a fixed version.

[ianfab]

Due to the large branching factor, playing around with extensions in crazyhouse can result in drastic changes (like an order of magnitude in the time to reach same depth) and scaling should be checked very carefully. I thought that significant improvements should be possible, but so far the best results were something like 5-10 Elo at STC and I haven't tested at LTC yet.

Regarding king safety, I realized that we do not consider pieces in hand of the defender, which however are very important. Probably the most simple approach would be to modify kingDanger in a similar way as for KingDangerInHand, but with opposite sign (and different absolute values). I will try that, but I am also open for suggestions on a more sophisticated way to consider the defender's pieces in hand.

[ddugovic]

Reviewing evaluate_king, I am now curious whether kingDanger = ... can consider (and whether this would be an improvement):

• how many pieces the opponent has in hand (or whether the opponent has any in hand)
• whether the opponent has a queen (and/or a knight) in hand

EDIT: Preliminary results at 0.1+1:

Score of PATCH vs Stockfish 140117 64 BMI2: 263 - 225 - 12  [0.538] 500
Elo difference: 26.46 +/- 30.21

[Vinvin20]

2 more moves to avoid :
8...Nc2?? game : https://fr.lichess.org/w3XWSD1B#16
32.Nxd5?? : https://fr.lichess.org/iSnpTCwr/black#62

[ianfab]

@Vinvin20 I think the second one is this typical "multi-sacrifice checkmate" problem. It is easy to write patches that make Stockfish find such mates much more quickly, but so far none of these was an improvement in general play. I have no idea what the first one is about or why it is considered a blunder.

@ddugovic You are welcome to create a test for your patch on my fishtest server.

[Vinvin20]

About the second position :
It's always the same problem : it's an improvement against humans but it's a regression against previous SF. This is because humans are good to look at forced mates.

About the first position : 32.Nxd5?? is a blunder because white is mated in 6 moves (but eval still +6.7 for a long time), white is losing anyway but not that quickly.

[ianfab]

The problem is that there usually are a lot of possible checks with negative SEE and if we extend all of them, it dramatically slows down the search. So the question is how we can identify interesting checks.

E.g. in the position https://en.lichess.org/fBXBdtqF/black#46, there are 2 knight checks on f6 and e7, two checks each for bishop and queen on f7 and e6, and one pawn check on f7, which sums to 7 in total. So in positions like https://en.lichess.org/iSnpTCwr/black#63 and https://en.lichess.org/fBXBdtqF/black#46, what (that can easily be checked during the search) distinguishes the good from the bad sacrificial checks?

[Vinvin20]

As I pointed out already several times, Stockfish-matefinder from Joseph Hellis improves a lot the mates finding : https://github.com/jhellis3/Stockfish

[ddugovic]

I have attempted numerous times for crazyhouse to merge that and for crazyhouse it has always performed worse in terms of finding checkmates and in terms of overall performance.

[ianfab]

The difficult part is to improve mate finding without regressing playing strength. Currently, an attempt to extend moves within a SEE range depending on the number of pieces in hand performs well. It might require some tweaks, but I like the idea.

[MichaelB7]

Almost all of Joe's mate finding tricks cost ELO and usually the better it does , the more it costs.

[Vinvin20]

Almost all of Joe's mate finding tricks cost ELO

It would be good to specify : "cost ELO" -> Against other engines
We don't have numbers about how the mod behaves against humans.

[ianfab]

@Vinvin20 Yes, that is absolutely correct. However, unfortunately we can not guide development by performance against human opponents, since there is no way to generate statistically significant data. So I think the only reasonable way to consistently improve playing strength is to judge patches based on their performance in selfplay, which has proved to be effective for standard chess.

[ddugovic]

I just submitted a test. Either way I think this is my final attempt based on these two positions, so good luck Stockfish!

[ddugovic]

The best solution seems to be throwing more hardware at this problem, since attempts to patch for these positions greatly weaken the engine.