Currently the score estimator always uses the Chinese ruleset. This differs from the final game scoring which does respect the ruleset.
For example, for a Japanese rules game: https://online-go.com/game/28155811
Final score:

Estimated score:

This is because the current score estimator always estimates in Chinese, regardless of the games actual ruleset.
I didn't know that, makes sense. I've updated the issue to reflect this.
Worth noting that @anoek is currently working on some AI scoring upgrades, so hopefully this issue will be resolved with that rollout
I hope the scoring _estimator_ is not getting improved by AI. As has been discussed in the past, it's widely seen as a good thing that it is "approximate".
Are you able to provide an rough metric on exactly how wrong the score estimate needs to be? If you have it at all the damn thing might as well know how to count. As long as it can be disabled at game creation I cant see an issue with it being a full strength KataGo number (I agree there is a strong argument for not showing the shadow count on the board).
Rough metric: it's about right now.
If we make it more accurate, it needs to be able to be disabled independently of analysis, and should be disabled by default.
Not exactly quantifiable, but here is the criteria that I would look for in judging an in-game score estimator:
1) The score estimator should speed up my counting.
2) The score estimator should not tell me anything I couldn't have figured out if given more time.
Kata would break rule 2 in the case of tricky life and death. In my opinion, the in-game estimator should be dumber but more controllable than it is now (i.e. when you click the dots, things don't get all wacky).
I agree there is a strong argument for not showing the shadow count on the board
This does fix many of the too-much-help issues... Still, I prefer a score estimator that feels more like a tool than an oracle. Maybe there is room for both options?
Most helpful comment
Not exactly quantifiable, but here is the criteria that I would look for in judging an in-game score estimator:
1) The score estimator should speed up my counting.
2) The score estimator should not tell me anything I couldn't have figured out if given more time.
Kata would break rule 2 in the case of tricky life and death. In my opinion, the in-game estimator should be dumber but more controllable than it is now (i.e. when you click the dots, things don't get all wacky).
This does fix many of the too-much-help issues... Still, I prefer a score estimator that feels more like a tool than an oracle. Maybe there is room for both options?