Stockfish is non-compliant with the UCI spec

[gcp]

• if the engine or the GUI receives an unknown command or token it should just ignore it and try to
  parse the rest of the string in this line.
  Examples: "joho debug on\n" should switch the debug mode on given that joho is not defined,
  "debug joho on\n" will be undefined however.

Currently, Stockfish won't try to continue parsing an unknown command, but will throw an error:
https://github.com/official-stockfish/Stockfish/blob/9d791386825ef9797e62706654df8af8d36b6268/src/uci.cpp#L240

I think a reasonable argument can be made that the behavior requested by the UCI spec is not very sensible though.

[lucasart]

"I think a reasonable argument can be made that the behavior requested by the UCI spec is not very sensible though."
Exactly!

[gcp]

Actually I can see one clear case in which this would be desirable: if one wants to extend the UCI spec, and probe if the engine supports the extension, then one would presumably would want to send something like "uci2", and not have the engine crash/exit when it happens. So it makes sense to define that unknown commands can happen and should not lead to this.

But returning a (standardized!) error would have been far better, because right now (if the engine just ignores it) the interface won't be able to tell if the engine hung or just ignored the command, so one is forced to define an arbitrary timeout and assume that no reply means the engine doesn't support the extension.

[beyondCritics]

It claims to be an "UCI engine", but it doesn't follow the protocol. It's a clean bug.
There are several places, where stockfish writes a message unsynced. My suggested clean up would be as follows:
1) Extend the logging facility, so that it captures also logs to stderr.
2) Implement a "sync_cerr" method, analogous to "sync_cout"
and finally
3) Find all places, where stockfish writes to stderr and replace them with "sync_cerr".
4) Replace all faulty writes to cout with writes using "sync_cerr".

[MichaelB7]

Just curious as to the to the initial event that even triggered this issues. I have run hundreds of thousand games without issues

[gcp]

It's mostly an issue for people writing new UCI interfaces. The existing ones are sure to already work around any divergence in Stockfish's implementation.

[protonspring]

If I'm looking for something to do, should I work on this? It might keep me from testing every half-witted thought that flies through my head.

[gcp]

I found another protocol violation:

setoption name PONDER value TRUE

Stockfish correctly detects that PONDER is Options["Ponder"], but does a case-sensitive match against the value. The spec says the option value should be case-insensitive: "The name of the option in should not be case sensitive and can inludes spaces like also the value."

I think the code correctly handles it for combo options but the boolean one was overlooked.

[Rocky640]

"The name of the option in should not be case sensitive and can inludes spaces like also the value."

I'm not a native English speaker, but when I read this,
I only read that the name should not be case sensitive
and that both name and value might contains spaces.

http://wbec-ridderkerk.nl/html/UCIProtocol.html
According to this old source,

options of type check can be true or false

Maybe this must be read literally as "true" or "false" (always lowercase)

[gcp]

Maybe this must be read literally as "true" or "false" (always lowercase)

This is inconsistent with "var" variables. Note that Stockfish does handle those as case insensitive.

That is, even though it says option name Analysis Contempt type combo default Both var Off var White var Black var Both, I believe setoption name analysis contempt value oFF works correctly. It's only broken for booleans.