Stockfish: mvSF variants tests with slow fairySF repo

[ppigazzini]

Hello @ianfab and @ddugovic. If I'm not wrong there are some tests about mvSF variants using fairySF repo. The fairySF runs at 50% of mvSF speed, so there is a waste of worker resources. I'm missing something?
ps: It took some time to understand why my workers were running so slow :) I lowered by hand the threshold of 0.6 Mnps in games.py

[ianfab]

@ppigazzini Sorry for the inconvenience. Note that Fairy-SF also uses more memory, in case this is problematic. The tests using Fairy-SF can not be done with MV-SF, since the forks have different functionality, i.e., Fairy-SF is not just a slower re-implementation of MV-SF (in fact the code of MV-SF is not used there at all, only some ideas were ported). The tests are specifically to improve that fork, and only in a few cases can be ported to MV-SF.

In the last months, my focus was mainly on improving the new fork (mostly because it is a pleasure to be able to add new variants within minutes or at most a few hours), but I still try to keep contributing to MV-SF. Since MV-SF is used on lichess, and since I do not know how people value my new fork, I usually assign higher priority to tests for MV-SF in MV-fishtest than for Fairy-SF (as I did in the past with tests for the forks for Shatranj and Seirawan chess), but in recent times there are only very few tests for MV-SF, so most of the time the framework is still busy with tests for Fairy-SF. I hope this practice is fine with the CPU contributors, otherwise please let me know.

Hopefully, this makes the situation more transparent. If you want to know more about Fairy-SF, please see its readme and wiki, or directly ask me.

[ppigazzini]

@ianfab thank you for the answer. I was confused reading the variant names of the running tests, nearly all classical variants (crazyhouse, losers, giveaway). I supposed that the Fairy-SF repo was an extension of the MV-SF repo, so to me the efficient process was to test a branch (for classical variants) with MV-SF repo and then, if successful, merging it also in Fairy-SF repo.
Anyway consider to lower the Mnps threshold in games.py, your and @ddugovic workers are just up the 0.6 Mnps limit.

[ianfab]

The tests are mostly for the classical variants partly because I am currently focusing on shogi (variants), which is not supported by cutechess, so I can only test the patches' performance in crazyhouse on fishtest and test the shogi variants locally. Furthermore, some of the other variants are not supported by the cutechess version that is currently used on fishtest, since they were added only in the last months. If you have time to provide updated binaries, that would be very nice, since compiling the windows and linux 32bit versions is difficult, because I do not have corresponding systems at hand.

I did not run into issues yet, but lowering the nps threshold of course makes sense, I'll do that soon.

[ppigazzini]

@ianfab I started the building of updates cutechess binaries with the wiki script, give me a couple of days.
Anyway I run Ubuntu 14.04 32 and 64 bit using lxd/lxc on the latest Ubuntu LTS:

sudo snap install lxd
sudo lxd init   # accept all defaults
sudo usermod -aG lxd <username>  # restart the shell to activate the group change

lxc launch ubuntu:14.04 u1404
lxc exec u1404 -- bash   # this start a bash in the u1404 container, here lauch the cutechess wiki script
lxc file pull u1404/root/cutechess-cli-linux-XXbit.zip cutechess-cli-linux-XXbit.zip
lxc stop u1404

lxc launch ubuntu:14.04/i386 u1404i386
...

# launch another disto (here centos7) to test the cutechess build
lxc launch images:centos/7 c7

# list the remote distro images
lxc image list ubuntu: | more
lxc image list images: | more

# list containers and local images
lxc list
lxc image list

# delete container, delete local image
lxc delete u1404
lxc image delete <image_hash>

[ianfab]

@ppigazzini Thanks a lot.

[ianfab]

@ppigazzini Can you give an update on the status of the builds?

[ppigazzini]

@ianfab the build on ubuntu 14 triggers a gcc bug, with ubuntu 16 and later are missing some libraries for the QT static build. In the latest weeks I cared for my mother (health issue) so I had very little free time.

[ppigazzini]

@ianfab @ddugovic
I successfully built cutechess with the wiki script using these sets:

• windows 10 -> hyper-v (dynamic memory off, otherwise gcc bugs) -> ubuntu 19.04 -> lxd -> ubuntu 14.04
• ubuntu 18.04 -> lxd -> ubuntu 14.04

With very high probability the random gcc bugs were triggered by the chain of too many dynamic memory managements of my previous set:

• windows 10 -> vmware -> ubuntu 18.04 -> lxd -> ubuntu 14.04

ps: I updated the previous comment adding some useful lxd commands.