Stockfish: SF settings for CCC

[anshulongithub]

@mcostalba @snicolet @Kiran @vondele @Vizvezdenec

@coolchess123 from CCC has reached out to me to provide the binary for CCC before the actual stage begins on coming Tuesday. They wanted to know what configuration settings SF shud play with there, here is the hardware that they have:
CPU engines:
CPUs: 2 x Intel Xeon Platinum 8168 @ 2.70 GHz 33 MB L3
Threads: 90 threads with HT on
16384 MB hash
RAM: 256GB DDR4-2666 ECC Registered RDIMM
SSD: 2x Crucial MX300 (1TB) in RAID1
OS: CentOS 7

Can one of you please confirm the configuration settings and the commit that they shud use to compile SF binary? They also wanted to know what flags shud be used for compiling the binary.

Shud the following settings be used?
Contempt=24
Hash=32gb
TB=6-men (7-men is available)
moveoverhead=2000ms

Please comment here and confirm the settings. Thank you.

[coolchess123]

the TC is 5+2 for 3 stages and for the finals which i expect SF to qualify its 10+5

[vondele]

I think it only matters little, my 2cents:

• TB .... largest size that fits the SSD, so 6.
• Hash, best determined by running from startpos on that hardware for a time that is representative for a move at the 10+5 TC, and going for largest possible value that yields a hash that is >~50% used, yet uses less than +- 40% of machine RAM.
• Version, latest one that has been tested on ncm without significant (> 5 Elo) regression.
• moveoverhead = 1000ms for the shorter TC
• building with make clean && make -j ARCH=x86-64-bmi2 profile-build

[Vizvezdenec]

CCC has SSD to use 7men TB AFAIK and is using it now for AB engines that support it.
I also think that "signifficant" ncm regression starts from like 20+ elo (when smth is seriously bugged), everything else is just random noise.

[vondele]

Ah... if 7men fits, we should try it out.

[MichaelB7]

Just FYI, we haven’t really tested 7 men syzygy under tournament conditions. If anybody has use the full set on SATA drives the slowdown, they would know it be an Elo killer ( even though I have the most commonly used TBs on SSD). All SSD drives would be different and certainly better than SATA. Of course if one wants to use CCC as a test bed for 7 men syzygy on SSD , then that’s the thing to do. And I’m in favor of using it regardless. Just giving a heads up , since so far the only experience I have and that I’m aware of is 7 men syzygy that is partial SSD and partial SSD and I have to modify the code slightly to make it usable under those conditions. Essentially I modified it only so it only probe 6 men on SSD during search and 7 men when there is just 7 men on the board since that search is just a lookup.

[coolchess123]

Its difficult for organizers to go figuring out which binary to pick or which commit to build when there are so many engines. I would like to know for CCC who i can email to get a reply as to following questions

1. Hash to use (based on tc, as TC keeps changing)
2. What commit to use for building
3. What flags to use for building (this is a one time thing)
4. Threads to use (with HT on)
5. 6men or 7 men TB
6. Contempt

Current settings

1. 8 GB [5 + 2]
2. Latest as of saturday
3. Not sure, i can find it out if required
4. 90 threads
5. 7 men TB
6. Contempt default

Configuration for CCC is

CPUs: 2 x Intel Xeon Platinum 8168 @ 2.70 GHz 33 MB L3
Threads: 90 threads with HT on
16384 MB hash
RAM: 256GB DDR4-2666 ECC Registered RDIMM
SSD: 2x Crucial MX300 (1TB) in RAID1
OS: CentOS 7

[coolchess123]

We will continue using the current settings unless a request for change is received

[fun8]

7 men TB need ~ 18 TB SSD,
CCCC CPU server have 2 TB SSD,
@coolchess123 CCCC not have full 7 man TB? Right?

[MichaelB7]

I was curious if they really had 18TB on SSD. That a serious investment. I found having less than the full set unsatisfactory, but other people were swearing by it. So I thought it was just me.

Sent from my iPhone

On Jun 18, 2019, at 10:41 PM, fun8 [e-mail] wrote:

7 men TB need ~ 18 TB SSD,
CCCC CPU server have 2 TB SSD,
@coolchess123 CCCC not have full 7 man TB? Right?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[MichaelB7]

@fun8 the key is the log - full set means it finds 1511 tablebases.

[anshulongithub]

@coolchess123 I think @syzygy1 has opened a PR for fixing a bug when SF uses 7 men TB. Until that is merged, we shud use 6 men. We shud also at least use 32 GB of hash if 256 GB is available on the machine. Also as @vondele mentioned, we shud use moveoverhead = 1000ms.

[fun8]

From wikipedia
RAID 1 consists of data mirroring, without parity or striping. Data is written identically to two drives, thereby producing a "mirrored set" of drives.
@coolchess123 lol you have only 1 TB SSD

[MichaelB7]

Not picking on anyone - but “shud” should be spelled “should”. When you see it 3 times , it starts becoming comical. And I understand English may or may not be your native tongue. l’m just trying to help out since in all other aspects everything appeared to be perfect and you are doing great work.

[anshulongithub]

@MichaelB7 I had advertently spelt it that way just to shorten it, it had become somewhat of a habit (also for the words, cud and wud). I totally appreciate your feedback and will take cognizance of it. Going forward, I will correct my written usage of all such words.
Thank you.

[LouisZulli]

@MichaelB7
A snippet from the cutechess logfile suggests that Stockfish is finding 621 tablebase files:

13 >Stockfish(1): uci
140 <Stockfish(1): Stockfish 160619 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
170 <Stockfish(1): id name Stockfish 160619 64 BMI2
170 <Stockfish(1): id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott
171 <Stockfish(1): option name Debug Log File type string default 
171 <Stockfish(1): option name Contempt type spin default 24 min -100 max 100
171 <Stockfish(1): option name Analysis Contempt type combo default Both var Off var White var Black var Both
171 <Stockfish(1): option name Threads type spin default 1 min 1 max 512
171 <Stockfish(1): option name Hash type spin default 16 min 1 max 131072
171 <Stockfish(1): option name Clear Hash type button
171 <Stockfish(1): option name Ponder type check default false
171 <Stockfish(1): option name MultiPV type spin default 1 min 1 max 500
171 <Stockfish(1): option name Skill Level type spin default 20 min 0 max 20
171 <Stockfish(1): option name Move Overhead type spin default 30 min 0 max 5000
171 <Stockfish(1): option name Minimum Thinking Time type spin default 20 min 0 max 5000
171 <Stockfish(1): option name Slow Mover type spin default 84 min 10 max 1000
171 <Stockfish(1): option name nodestime type spin default 0 min 0 max 10000
171 <Stockfish(1): option name UCI_Chess960 type check default false
171 <Stockfish(1): option name UCI_AnalyseMode type check default false
171 <Stockfish(1): option name SyzygyPath type string default <empty>
171 <Stockfish(1): option name SyzygyProbeDepth type spin default 1 min 1 max 100
171 <Stockfish(1): option name Syzygy50MoveRule type check default true
171 <Stockfish(1): option name SyzygyProbeLimit type spin default 7 min 0 max 7
171 <Stockfish(1): uciok
171 >Stockfish(1): setoption name Hash value 8192
171 >Stockfish(1): setoption name SyzygyPath value /data/tablebases
171 >Stockfish(1): setoption name Threads value 90
171 >Stockfish(1): isready

3854 <Stockfish(1): info string Found 621 tablebases
5865 <Stockfish(1): readyok
Started game 1 of 1 (Lc0 vs Stockfish)

@anshulongithub
At the very least, please avoid "cud"---unless referring to bovines.

[MichaelB7]

@anshulongithub. No worries and I appreciate your response and the fact that in today’s world , especially in texting , all kinds of shortcuts are being used. Obviously, if one makes people laugh unintentionally, the point you are trying to make is getting diluted. I once had a very close and dear friend trying to make a very serious point regarding something about the world. And as he’s telling me this , his earring , which was a small parrot hanging in the center of a ring , which was hanging from his ear , was spinning crazily as it was blowing in the wind. As he went on , I eventually just started laughing and told him it was really hard to take him seriously with this parrot spinning in the wind as he was talking, as my mind just seemed to focus on this spinning parrot. We still laugh about it today and it was many years ago , over 20 years.

[MichaelB7]

@LouisZulli Thanks - Using a 111 or so tablebases of the 7 man syzygy for of the most common endgames types , from stats I have seen published, will provide about 90% coverage in engine vs engine chess games and 95% coverage of human vs human games. The computer games will vary based now whether adjudication is being used or they are being played out to to the very end. The 111 was derived from the 621 you provided less the 510 tablebases for 6 man syzygy ( that's from memory , I could be off ).

[coolchess123]

Can i go with this settings for stage2, any objections kindly let me know

16 GB [5 + 2]
Latest build 
 make clean && make -j ARCH=x86-64-bmi2 profile-build
90 threads
6 men TB
Contempt default

[vondele]

seems good, until #2198 is merged (which will restore correct play with partial 7men).

[coolchess123]

Now that bug is fixed can we try the following

16 GB [5 + 2]
Latest build
make clean && make -j ARCH=x86-64-bmi2 profile-build
90 threads
7 men TB
Contempt default

[Alayan-stk-2]

Yes now that the TB bug has been fixed you can go back to using those partial 7-men TB.

[adentong]

SF actually blundered a win vs Ruby today so it's a good thing this got fixed so quickly.

[Nordlandia]

I remember Fire author said that CCC stored 7-piece TB on HDD. This might have chanced.

[snicolet]

Closing this issue now.