When I try to set the application setting for a app service, the characters ^ and ! are ignored which are part of the string, I have tried using the quotes to no effect.
to replicate, try calling
call az webapp config appsettings set "serverpassword"="abcde!xyz"
I am using Microsoft Azure Command Prompt v2.9 on Windows 8.1 Pro build 6.3.9600
This sounds like an escaping issue. Did you try escaping the special characters with \! ?
That didn't work, if I use abcde!xyz it saves abcde\"xyz
If you run this with --debug you can see what string is actually being sent to the service.  If it is sending a mangled string, then it is likely an escaping issue with your chosen shell (but could also be the CLI).  If it is sending the correct string, then the issue is with the service doing some kind of string sanitization.
Note: please don't paste a debug trace in GitHub :)
The String sent in the request body does not contain the ! character, so the filtering is happening on the shell itself (Windows Command prompt cmd.exe). The command arguments themselves omit the character.
That helped me to narrow down the issue to be escape characters for ! in command prompt. https://ss64.com/nt/syntax-esc.html helped me that it needs to be ^^!
I ran into the same issue with a close brackets ) character, and also had to escape it with ^^. Spent a very long time trying to get to the bottom of this. Any chance the docs could be updated to warn of this issue? 
It might also be helpful if appsettings set provided an alternative way to get settings in that didn't suffer from this issue
Another scenario in which az webapp config appsettings set is almost unusable is trying to use the new WEBSITE_RUN_FROM_PACKAGE feature. To successfully escape the & character in the SAS URI I had to replace them all with ^^^&
Most helpful comment
Another scenario in which
az webapp config appsettings setis almost unusable is trying to use the newWEBSITE_RUN_FROM_PACKAGEfeature. To successfully escape the&character in the SAS URI I had to replace them all with^^^&