Aspnetcore: Random: The antiforgery token could not be decrypted.

[Eilon]

_From @zulander1 on Tuesday, 18 September 2018 23:44:52_

Issue Title

Hello,

We are getting random: The antiforgery token could not be decrypted.

Site is running on IIS on Windows 2016, the authentication is Azure AD, Application in running on NET CORE 2.1.4. No error generated on the development site.

I am not too sure where to start...
Thank's

fail: Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgery[7] An exception was thrown while deserializing the token. Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted. ---> System.Security.Cryptography.CryptographicException: The key {9009f561-0974-426e-8d3a-00fb5ce76e95} was not found in the key ring. at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData) at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken) --- End of inner exception stack trace --- at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken) at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgery.GetCookieTokenDoesNotThrow(HttpContext httpContext)

_Copied from original issue: dotnet/core#1947_

[Eilon]

_From @Petermarcu on Wednesday, 19 September 2018 02:32:45_

@Eilon

[Eilon]

@blowdart - thoughts?

[Tratcher]

Is the site load balanced across multiple instances? Or does this happen sometimes after a recent deployment?

[zulander1]

Hi,

Well, yes and non (not a true load balancer)

We have, Azure traffic manager... Which acts like a load balancer.

It's like this:

Azure traffic manager (works in priority mode)
1) Primary server windows
2) secondary (web application with Azure) (failover)

Mssql with Azure (with failover)

[Tratcher]

Data protection requires configuration when distributed across multiple machines.

[zulander1]

@Tratcher I understand; however, my concerns are that the current setup does not share data between servers. Each distribution is independent. Therefore, each request is forwarded to the same server all the time (un less the primary server has failed). Therefore, for me it does not make sense. On a typical day we would see 17 failure regarding this problem, the azure token gets re-authenticated every hour or so and we have about 15 to 35 users on the app, shouldn’t the we see more error, on each connection/authentication?

[Tratcher]

If the failover is not active then you're right, the cross machine data protection would not be relevant. That's easy to confirm if you verify your secondary server is not receiving any traffic.

With so little traffic does your site go idle and shut down? Do you have warnings in your logs on startup about your data protection keys not being persisted? There are some cases where encryption keys are lost across restarts.

[zulander1]

Hi,

Failover is never active, we get notification (and call's) if it was:
_Does your site go idle and shut down_: Yes, most likely at night when people go home after work
? _Do you have warnings in your logs on startup about your data protection keys not being persisted_: Non

we have just used the default data protection, nothing is configured for data protection

[Tratcher]

That covers all of the usual errors. @blowdart any other ideas?

[blowdart]

The question about warnings still stands. It may be dropping to in memory. During app startup Data Protection will log its configuration, we need that part of the log.

[zulander1]

@blowdart

There you go

[20:04:53 INF] Starting web host
[20:04:57 WRN] Using an in-memory repository. Keys will not be persisted to storage.
[20:04:57 WRN] Neither user profile nor HKLM registry available. Using an ephemeral key repository. Protected data will be unavailable when application exits.
[20:04:57 INF] Creating key {d8b848b0-20d2-4e61-9280-aa8873fd1c10} with creation date 2018-09-21 00:04:57Z, activation date 2018-09-21 00:04:57Z, and expiration date 2018-12-20 00:04:57Z.
[20:04:57 WRN] No XML encryptor configured. Key {d8b848b0-20d2-4e61-9280-aa8873fd1c10} may be persisted to storage in unencrypted form.
Hosting environment: Production

i am guessing the solution is at this blog:

https://cypressnorth.com/programming/solved-using-memory-repository-keys-will-not-persisted-storage-asp-net-core-iis/

[blowdart]

Yea, there's the problem

[20:04:57 WRN] Using an in-memory repository. Keys will not be persisted to storage.
[20:04:57 WRN] Neither user profile nor HKLM registry available. Using an ephemeral key repository. Protected data will be unavailable when application exits.

The solution can be different, if you don't want the overhead of loading the profile you can run a powershell script, as described in the deployment docs or you pick a file system directory and config it manually.

[zulander1]

Greatly appreciated!!

[zulander1]

@blowdart
still getting the error thing morning:

[11:31:38 INF] Starting web host
[11:31:42 INF] User profile not available. Using 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET\4.0.30319.0\AutoGenKeys\S-1-5-82-119119927-3655580819-890001450-1487895255-3634842929\DataProtection' as key repository and Windows DPAPI to encrypt keys at rest.

Hosting environment: Production

[09:43:25 ERR] An exception was thrown while deserializing the token.
Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted. ---> System.Security.Cryptography.CryptographicException: The key {b876deca-1593-40bf-97a6-eb5b6466f40d} was not found in the key ring.
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked)
at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken)
--- End of inner exception stack trace ---
at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken)
at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgery.GetCookieTokenDoesNotThrow(HttpContext httpContext)

[blowdart]

OK, can I see your code configuration for data protection? (If there's a chance this ended up in another server than the original one then this is expected, unless you are sharing the keyring)

[zulander1]

@blowdart there is no code for that, everything is at default, i didn't see any instruction on implementation for HKLM,

For you question "chance this ended up in another ... " the answer is no, it's impossible, i would have been notified !

Thank's in advanced

[blowdart]

See the deployment docs for the powershell script.

[zulander1]

The powershell script (https://github.com/aspnet/DataProtection/blob/release/2.2/Provision-AutoGenKeys.ps1) was executed on the server, that's why it's saying "Using 'HKEY_LOCAL_MACHINE..."

[blowdart]

OK this is weird. The error you're getting is that a key is missing, which tends to indicate it went to a different machine, or the request has come from a really old form, one before the machine was reconfigured and rebooted (or someone is messing with the payload)

[zulander1]

Hi, I think we can close this issue, i haven't seen any error... i guess the server need a reboot !

[ManuelHaas]

@blowdart: off topic: you wrote in your post on 21 Sep. "if you don't want the overhead of loading the profile ...". I configured the app pools to load the user profiles and don't recognized any performance reduction on app startup in our applications. Can you describe what or how this overhead can be? When is it relevant or can it be ignored? I don't find useful infos when googling this topic.
Thanks

[blowdart]

@ManuelHaas The time taken to load the profile, then memory, and a little disk space. It's not enough to be noticeable in most cases, but it's there.

[blowdart]

Closing as requested by the original poster.

[colinwphillips]

This just occurred (and wasted lots of effort troubleshooting) on ASP.NET Core 2.2 on Ubuntu 16.04 LTS. Was not occurring during development on localhost (Win10 Pro).
Application went from lightning fast to 5+ seconds per page load which led me to the logs.
Microsoft.AspNetCore.Antiforgery.AntiforgeryValidationException: The antiforgery token could not be decrypted. ---> System.Security.Cryptography.CryptographicException: The key {b876deca-1593-40bf-97a6-eb5b6466f40d} was not found in the key ring.
Seemingly disabled [ValidateAntiForgeryToken] and still received errors in logs on every single request as well as "The application completed without reading the entire request body...".
Cleared cookies, no dice.
Finally, rebooted server - immediately resolved issue.

[ZOXEXIVO]

Catch this error at 2.2, without any AntiForgery validation enabled

[StefanoChiodino]

Don't do the same mistake I was doing: if you log in your machine and the change the machine (e.g. use docker) and keep using the same domain/port then you must clear the browser cache or it will try to use a key generated on the previous machine.