Leela-chess: Client crashing during game upload

[jkiliani]

I had similar cases to this output happening a couple of times now:

Created dirs data-1
PGN
1. d4 a6 2. a3 Nf6 3. Ra2 Ra7 4. Be3 b5 5. Kd2 Ng4 6. d5 Nxf2 7. Bb6 Nxd1
8. e3 h6 9. Be2 Rb7 10. Bxd1 d6 11. Bf3 Rg8 12. g4 h5 13. Nc3 f6 14. h4 Rh8
15. Be4 Be6 16. Bh7 Rxh7 17. Nce2 c6 18. dxe6 Nd7 19. Ng3 Ne5 20. Nxh5 Rxh5
21. a4 Rg5 22. Ba5 Nc4+ 23. Kd3 c5 24. e4 Nxb2+ 25. Kc3 Qxa5+ 26. Kxb2
bxa4+ 27. Kc1 Ra7 28. e5 Kd8 29. c3 Rf5 30. gxf5 Rc7 31. c4 Kc8 32. exf6
gxf6 33. Rb2 d5 34. Ne2 Rc6 35. Rg1 d4 36. h5 Qd8 37. Rb7 Kxb7 38. Kd1 Qc8
39. Rg3 d3 40. Nc1 Qc7 41. Kd2 Ka8 42. Kxd3 Qxg3+ 43. Ke4 a5 44. h6 Bxh6
45. Nb3 Qxb3 46. Kd5 Qf3+ 
END
Score: -1
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Error in gzip output
2018/03/04 20:53:08 signal: abort trap

I this a known bug, or are there fixes or workarounds for it yet? I don't have a trace unfortunately, it seems to happen occasionally only. Most of the time the game upload is successful, but sometimes I got such client crashes like this.

I ran two clients simultaneously, could this become a problem if both try to upload at the same time?

[Error323]

Hi jkiliani,

I think you were running multiple clients? If you update your client, this
should be fixed.

On Mon, Mar 5, 2018 at 12:07 PM, jkiliani [e-mail] wrote:

I had similar cases to this output happening a couple of times now:

Created dirs data-1
PGN

1. d4 a6 2. a3 Nf6 3. Ra2 Ra7 4. Be3 b5 5. Kd2 Ng4 6. d5 Nxf2 7. Bb6 Nxd1
2. e3 h6 9. Be2 Rb7 10. Bxd1 d6 11. Bf3 Rg8 12. g4 h5 13. Nc3 f6 14. h4 Rh8
3. Be4 Be6 16. Bh7 Rxh7 17. Nce2 c6 18. dxe6 Nd7 19. Ng3 Ne5 20. Nxh5 Rxh5
4. a4 Rg5 22. Ba5 Nc4+ 23. Kd3 c5 24. e4 Nxb2+ 25. Kc3 Qxa5+ 26. Kxb2
   bxa4+ 27. Kc1 Ra7 28. e5 Kd8 29. c3 Rf5 30. gxf5 Rc7 31. c4 Kc8 32. exf6
   gxf6 33. Rb2 d5 34. Ne2 Rc6 35. Rg1 d4 36. h5 Qd8 37. Rb7 Kxb7 38. Kd1 Qc8
5. Rg3 d3 40. Nc1 Qc7 41. Kd2 Ka8 42. Kxd3 Qxg3+ 43. Ke4 a5 44. h6 Bxh6
6. Nb3 Qxb3 46. Kd5 Qf3+
   END
   Score: -1
   libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Error in gzip output
   2018/03/04 20:53:08 signal: abort trap

I this a known bug, or are there fixes or workarounds for it yet? I don't
have a trace unfortunately, it seems to happen occasionally only. Most of
the time the game upload is successful, but sometimes I got such client
crashes like this.

I ran two clients simultaneously, could this become a problem if both try
to upload at the same time?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/glinscott/leela-chess/issues/97, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEpObtG6yBYhoQ2yn9-BenPTpYkFRKOks5tbRx_gaJpZM4ScGuY
.

--

Kind regards,

Folkert Huizinga

[jkiliani]

Cool thanks, I'll update in that case. If it somehow should happen again I'll reopen the issue.

[jkiliani]

@Error323, @glinscott: I updated my client, to fix this issue... unfortunately now I have a much more serious one: Some recent commit removed the CPU-only compatibility. I know from Leela Zero already that on my Mac, the OpenCL drivers are busted, i.e. I will always get self-check errors if I use the GPU. For this reason I only ran lczero on the CPU. But after the update, calling the client gets:

libc++abi.dylib: terminating with uncaught exception of type boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::program_options::unknown_option> >: unrecognised option '--gpu=0'

I can no longer run Leela Chess until this is fixed unfortunately.

[jkiliani]

I suspect https://github.com/glinscott/leela-chess/commit/ca41078cc87c35b6ace9c2c02d3bedc2cbfa1649 no longer supports CPU-only clients...

[Error323]

Yeah I think the --gpu option should be a no-op for CPU clients only. Right now the option is not baked into the binary, but the go client does expect it to be there.

[jkiliani]

Is there any way I can override the client passing on --gpu=0 with the command line? Or do I need to change the client source code?

If I hack the client to remove gpu_id from the lczero call, will this fix the problem, or could there be other problems?

[Error323]

diff --git a/src/main.cpp b/src/main.cpp
index c4b73b6..2afe5f0 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -76,9 +76,9 @@ static std::string parse_commandline(int argc, char *argv[]) {
         ("noponder", "Disable thinking on opponent's time.")
         ("start", po::value<std::string>(), "Start command {train, bench}.")
         ("supervise", po::value<std::string>(), "Dump supervised learning data from the pgn.")
-#ifdef USE_OPENCL
         ("gpu",  po::value<std::vector<int> >(),
                 "ID of the OpenCL device(s) to use (disables autodetection).")
+#ifdef USE_OPENCL
         ("full-tuner", "Try harder to find an optimal OpenCL tuning.")
         ("tune-only", "Tune OpenCL only and then exit.")
 #endif

That might do it, untested

[jkiliani]

Seems to work, thanks! I'll update if I notice any unintended consequences...

[jkiliani]

@Error323 Had my client running for a day now without noticing any ill effects. Could you commit this please, so I can add CPU-only instructions to https://github.com/glinscott/leela-chess/issues/88?

[jkiliani]

@Error323, @glinscott Could one of you please push this fix to master, so CPU only builds work with the client again without modifying the source code? Thank you.

[glinscott]

@jkiliani Oh, yes, I've fixed this issue with the match game update that will be rolling out. You can now pass --gpu -1 into the client and it will run in CPU mode. It's not quite ready yet. If you create a PR for this change I can apply in the meantime, and then I'd revert it as part of the match game PR.

[jkiliani]

Great, then I guess it's not urgent. I can continue to use the fix, and when client v2 comes out, we can advertise the CPU capability for people who want to contribute without a good GPU (or with broken OpenCL drivers, like me...)

[Error323]

I still added the patch to master. It's blocking for people.