Leela-chess: is the goal of the project to produce data for your network? semi-open project?

[hsntgm]

Hello,

I spent a week to be adapted to the project like a detective because there is no information what are you doing exactly here.

1.Is supervised learning disabled? If so, why?

--supervise flag always throws an exception "Need to update SL flow."

This issue opened 15 days ago by @dkappe and flagged Help Wanted Are not you fully aware of the codes on the project or you want to go semi-open project?

2-All info on the wiki based on "use our client and send data to our server" so is the goal of the project to produce data for your network? Because of that supervised learning disabled and no info about the server side operations?

3.Where can i find the parse.py, leela_to_proto.py and supervised_parse.py files ?All these files is needed to server side tensor operations according to old topics.I found some of them from @Error323 (this guy is gold for the project) but all of them gives errors.So these files are secret?

On github project info has the below code:

cd training/tf
./parse.py configs/your-config.yaml

4-There is no enough information about the principles about defined new yaml file config.

batch_size:
learning_rate:
decay_rate:
decay_step:
policy_loss_weight:
value_loss_weight:

And the list goes on.

Sorry but i feel this project isn't totally open-source and it is nearly impossible to adapt.Just a few people have been playing here.Maybe it is better to go commercial.

Sincerely,

[Error323]

Hi @hsntgm,

First of all, the documentation is not great on training. I'm very sorry for that, time is limited and it's a matter of choosing to write new code or documentation.

1. The supervised flag allows one to generate trainingdata from existing pgn files. This training data is the format that our training pipeline takes. The pgn files need to be cleaned with pgn-extract first to make sure they're sane. Our current pgn parser is quite strict. It might be broken at the moment, a lot is currently in flux...

2. The full training pipeline code is located in training/tf. The trainingserver runs the start.sh script located in the glinscott/live branch.

3. All those have been replaced with train.py with the idea to simplify things, apparently this is not yet the case :) See start.sh on howto use it, or examine the code for now.

4. See training/tf/configs/example.yaml

I can assure you that this project is fully open source in the truest sense of the word. None of the code is hidden in any shape, way or form. The yaml config files for training are updated on our discord server, as this is where most of us reside.

Open source, however, does not imply solid documentation on anything... Not unless someone takes the time and effort todo so.

[hsntgm]

@Error323 Thanks for the key informations.

In short:

What is wrong here is primarily to provide the necessary documentation for people to prepare the environment in which they can train on their own machine or maybe on their own HPC system or on their own gpu farm.Only after that you deserve to want people gpu times for producing data for your main tensor server.

if you do not share this information with people properly i call this "they are worried about sharing" and this is not soul of open-source.

[brianprichardson]

FWIW I was able to get training running on my lowly PC after some fiddling.
Some cautions:
Self play on your own will require a very large hardware footprint.
Even just training the networks takes about a week on a medium fast gpu.

As far a sharing, there are about 600 people contributing per below.
The good news is that the project is moving very quickly.
The bad news is that polished documentation is lagging behind (sound familiar?)

See lczero.org:
Active Users
583 users in the last day have played 198927 games

[Dawiev]

Hsntgm, you most certainly have the right to your opinion and to voice it. However, it remains exactly that ? To refer to "people" and to presume to speak on behalf of others, would deny them their voice. Feel free to join the Discord chat and perhaps from there you can get to know the community and even help to create the documentation (you, not "the people") would like to see.

[hsntgm]

even help to create the documentation.

@Dawiev I've already done it.

Find my How to use intel mkl for blas api? documentation below.I published 3 days ago.
https://github.com/glinscott/leela-chess/issues/467

Now my turn, I would like someone share detailed training tutorial so i can train leela-chess on my own machine, because it is not possible with current training information on github.I am an ordinary system engineer and curious guy not much.

Maybe i have nvidia DGX2 or i will use it on univercity HPC system so i can get better results then @brianprichardson machine.

Thank you in advance all.

[brianprichardson]

@hsntgm To train the net a large set of samples are needed. Currently, the pgn parser is somewhat inflexible in what it will accept, so you can try to use other chess pgn utilities to clean up a file with a fairly large number of games. So, what is a good number? The self-play training uses the most recent 500,000 games, but you wouldn't need that many to understand the process before scaling things up. The "supervise" phase will convert the games from pgn format to position samples (with 8 half moves of history) in a compact format that serves as input to the actual NN training.

The way I got it to work was a hack. I ran a few self-play games and just duplicated the resulting sample output files many times over. I have not looked in a while, but it used to be one file per game, IIRC. You will need enough games to have a couple of "chunks". I think I also edited the code to reduce the chunk size and the yaml config file to reduce the other large values. Then you can run train.py. Of course, with so little input data the net will "train", but not improve (the old GIGO), but you can see how the process works.

If the above is not helpful, you might try a less computationally demanding game from the Other Projects links at the bottom of the main page or from here:
https://github.com/suragnair/alpha-zero-general

I used Othello on a 6x6 board to start with trying to use the Alpha Zero approach. That was after trying to understand Giraffe and getting that to work. Before moving from Othello to chess, I got the Zeta36 approach working, somewhat. The Leela projects (Go and Chess) are much more complicated being distributed.

Your technical skill are likely far better than mine as you got the Intel MKL for BLAS thing working. In any case, I'll try to help if you have other questions.

[killerducky]

We will try to improve the documentation when and where we can. For the specifics of SL learning, please continue discussion in the PR for that.