The background of the ToggleButton is not set to transparent even though the ToggleButton, within it's Resources in XAML, defines all related resource style keys to transparent. I expect this is related to the style system re-write.

Related XAML is here: https://github.com/microsoft/microsoft-ui-xaml/blob/276611504457f340e4ce7b62b5eaca5a7cfcef32/dev/ColorPicker/ColorPicker.xaml#L348
Able to set a transparent background.

This is visible in the style of the ColorPicker 'MoreButton'. See PR #3955.
I'm sure one exists in this case but none have been tried.
Nuget Package:
Nuget Package Version(s):
Affected platform(s):
IDE:
Relevant plugins:
@davidjohnoliver Do you have any suggestions on where to start with this one? It would be good for me to learn how XAML is processed behind the scenes although I expect this issue goes beyond my current abilities with Uno.
There may be multiple layers to this one, but the first problem I can see is that we're not using the right resource keys in the vanilla ToggleButton default style:
https://github.com/unoplatform/uno/blob/master/src/Uno.UI/UI/Xaml/Style/Generic/Generic.xaml#L2873
It seems to be directly using the system resources instead of ToggleButtonBackground etc.
One quick way to check if that's the (only) issue is to use the Fluent styles, which are very close (in most cases identical) in Uno to the WinUI versions. They can be enabled in the samples app with the 'FL' button in the top right:

If you enable them and navigate to your ColorPicker sample, is the ToggleButton customized correctly?
GitHub
Build Mobile, Desktop and WebAssembly apps with C# and XAML. Today. Open source and professionally supported. - unoplatform/uno
Checking the FL checkbox and then retesting added rounded corners to the ToggleButton; however, the background colors/behavior is unchanged.
The good news though is the fluent styles fixed the slider thumb issues #3998
Then there's still something with the resource resolution that's not supporting this case properly.
So what should be happening is that the style setter (eg <Setter Property="Background" Value="{ThemeResource ToggleButtonBackground}" />) should be seeing the overidden resource value in the ToggleButton (to which the style is being applied), but seemingly it's not.
The Setter.ThemeResourceName is set from Xaml-generated code and then reacted to here when the style is applied.
I haven't dug into this much yet and am currently focusing on other things. However, PR #4135 partially fixes this which I suspect you might already know. The style is now correct until the button is pressed and correct again when not-pressed. Remaining issue may be in the ToggleButton itself.

Ahh I didn't realise that #4135 would help here but that makes sense!
The remaining problem when it's toggled may be a specific interaction of ThemeResources with visual states/animations.