Stockfish: Pawnless positions are evaluated incorrectly because of a pawn hash failure

[atumanian]

Steps to reproduce

Start the program and run the following commands:

1. position fen 8/5K2/8/8/8/R4R2/5r1k/8 b - - 8 124
2. eval
3. bench
4. position fen 8/5K2/8/8/8/R4R2/5r1k/8 b - - 8 124
5. eval

The output of command 2:

      Eval term |    White    |    Black    |    Total    
                |   MG    EG  |   MG    EG  |   MG    EG  
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  4.96  5.81 
      Imbalance |   ---   --- |   ---   --- | -0.11 -0.11 
          Pawns |   ---   --- |   ---   --- |  0.00  0.00 
        Knights |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
         Bishop |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Rooks |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
       Mobility |  0.22  0.96 |  0.08  0.52 |  0.14  0.45 
    King safety | -0.54 -0.32 | -1.32 -0.40 |  0.78  0.08 
        Threats |  0.19  0.25 |  0.00  0.24 |  0.19  0.01 
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Space |  0.02  0.00 |  0.00  0.00 |  0.01  0.00 
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  5.96  5.66 

Total Evaluation: 5.58 (white side)

The output of command 5:

      Eval term |    White    |    Black    |    Total    
                |   MG    EG  |   MG    EG  |   MG    EG  
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  4.96  5.81 
      Imbalance |   ---   --- |   ---   --- | -0.11 -0.11 
          Pawns |   ---   --- |   ---   --- |  0.00  0.00 
        Knights |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
         Bishop |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Rooks |  0.36  0.16 |  0.18  0.08 |  0.18  0.08 
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
       Mobility |  0.22  0.96 |  0.08  0.52 |  0.14  0.45 
    King safety | -0.54 -0.32 | -1.32 -0.40 |  0.78  0.08 
        Threats |  0.19  0.25 |  0.00  0.24 |  0.19  0.01 
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Space |  0.34  0.00 |  0.31  0.00 |  0.03  0.00 
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  6.14  5.74 

Total Evaluation: 5.66 (white side)

Description

Pawnless positions are evaluated incorrectly near the start of the program. After the evaluation function has been executed several times, the position are evaluated correctly.
The reason for this bug is a pawn hash failure. The pawn hash manager treats 0 in the pawn key as a special value: the field in the pawn hash table is set to 0 at initialization and means that the entry is uninitialized. But all positions without pawns have this pawn key! After the program has done some work, the zero entry of the hash table has been written with another key, so the entry with the key of 0 is initialized and positions with this key are already evaluated correctly.
This is a quite old bug: it appears in Stockfish 7.

Fix
https://github.com/atumanian/Stockfish/tree/fix_pawn_key
At initialization we generate a new random key (Zobrist::noPawns). It's added to the pawn key of all positions.
Should I run a [-3, 1] test for this fix?

[MichaelB7]

Nice catch - hopefully your patch passes.

[atumanian]

@MichaelB7, thanks. I'm waiting for approval from the maintainers.

[mcostalba]

I had no time to check it properly, but please proceed.

On Monday, November 21, 2016, Aram Tumanian [email protected]
wrote:

@MichaelB7 https://github.com/MichaelB7, thanks. I'm waiting for
approval from the maintainers.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/official-stockfish/Stockfish/issues/882#issuecomment-262025495,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABDGAbEi5diTSOzEQP0Ltg3igRt47l_-ks5rAeOhgaJpZM4K4YTy
.

[joergoster]

And why is there space evaluation of 0.34 for white and 0.31 for black?
I thought we only evaluate space above a certain material limit?

[atumanian]

@joergoster, the space values are shown only in the MG column. The EG column shows 0.00. The final eval is a weighted average of the MG eval and the EG eval. The weights depend on the game phase.

[atumanian]

But it's strange that the final eval is less that both the MG and EG evals. The table obviously shows inconsistent results.

[joergoster]

@atumanian No, with only 3 rooks of non-pawn material left, space shouldn't be evaluated at all.
I see, the score for space evaluation is no longer correctly initialized to zero. This is a bug.
Edit: Now I got it, I guess. If we do trace, we always calculate space unlike to normal eval.

[vondele]

@atumanian actually, I think this might also resolve a problem I was seeing with the fact that the search was still not fully repeatable after an ucinewgame command. Could you check eval is reproducible after additionally

6 ucinewgame
7 position fen 8/5K2/8/8/8/R4R2/5r1k/8 b - - 8 124
8 eval

This should give the same result for step 2, 5, and 8, I think.

[atumanian]

@vondele, the new version gives the same output thrice.

[vondele]

@atumanian , thanks, as it should be.

However, I'm still wondering why it doesn't in the current version of the code. 8 gives the same as 5, while I would expect 8 to give the same as 2. So, what's the state that is not properly reset by the ucinewgame ?

It is somewhat orthogonal to your patch, but you might understand what is causing this.

[joergoster]

@vondele Doesn't Aram's patch exactly address this issue? Now the position state is correctly initialized even without pawns. Or do you mean something else?

[vondele]

@joergoster actually not quite, it avoids the bad behaviour by initialising the key differently, but the state is actually in the pawnstable (and materialstable). I've added #886 to clear those tables at ucinew search, and now 8 gives the same result as 2, also before the patch of @atumanian . His patch still makes sense, however.

[joergoster]

@vondele Ah, now I see what you mean. Definitely makes sense. :-)

[mcostalba]

Fixed upstream