Aspnetcore: Google+ shutdown will break OAuth provider

[Tratcher]

The Authentication.Google package implements OAuth2 with Google services. However, it uses Google+ to fetch additional user information.
https://github.com/aspnet/AspNetCore/blob/5ab3c89be3e6342f2a39c666fd0aca708fc7ec8b/src/Security/Authentication/Google/src/GoogleDefaults.cs#L21
https://github.com/aspnet/AspNetCore/blob/5ab3c89be3e6342f2a39c666fd0aca708fc7ec8b/src/Security/Authentication/Google/src/GoogleOptions.cs#L29-L34

"The Google+ Sign-in feature is fully deprecated and is being shut down on March 7, 2019. This will be a progressive shutdown, with intermittent failures starting as early as January 28, 2019. Developers should migrate to the more comprehensive Google Sign-in authentication system." ~https://developers.google.com/+/web/signin/

This is a patch candidate all the way down to 1.0 and Katana. @muratg @blowdart

Proposals:

• Find a new API that will give us basic information like name, e-mail, etc.. It's unlikely the transition would be seamless.
• Deprecate the provider and show people how to use OpenIdConnect. This has the benefit of being a docs only change. It may not work for Katana though, we'll have to see if it supported enough ODIC features.

[jamesgurung]

The scopes requested by default are:

• openid
• email
• profile
• https://www.googleapis.com/auth/userinfo.email
• https://www.googleapis.com/auth/userinfo.profile
• https://www.googleapis.com/auth/plus.me

It seems like it's just the last one which is connected with Google+. Is the fix as simple as removing this scope? I don't think the default auth flow even uses it?

This would be a breaking change, but most common use cases are covered by the other scopes - and it's going to break anyway when Google+ is shut down.

[Tratcher]

It's the call out to https://www.googleapis.com/plus/v1/people/me that's really going to break. That's where we get names, e-mails, etc.. We need to find a replacement.

[hempels]

I expect you can simply replace the call to /plus/v1/people/me with a call to /userinfo/v2/me.

It should return a response comparable to the one returned by the deprecated + API and not require any further changes (aside from removing the plus.me scope.)

https://developers.google.com/apis-explorer/#search/userinfo/m/oauth2/v2/oauth2.userinfo.v2.me.get

Alternatively, you can skip that call if using their TokenInfo endpoint to handle token validation. If the email and profile scopes are included, it automatically returns those fields.

[rhubrightj]

Pardon my ignorance on the subject.... but where do I find this call in an asp.net web api project? I'm only using token validation.

I know I'm using it somewhere, according to google I'm sending calls to plus.people.get

I do have reference to Microsoft.Owin.Security.Google

[theaswanson]

Pardon my ignorance on the subject.... but where do I find this call in an asp.net web api project? I'm only using token validation.

I know I'm using it somewhere, according to google I'm sending calls to plus.people.get

I do have reference to Microsoft.Owin.Security.Google

If you're using ASP.Net Core 2.2, you might have the following code in your Startup.cs file which makes use of the OAuth provider (taken from https://docs.microsoft.com/en-us/aspnet/core/security/authentication/social/google-logins?view=aspnetcore-2.2):

services.AddAuthentication().AddGoogle(googleOptions => { googleOptions.ClientId = Configuration["Authentication:Google:ClientId"]; googleOptions.ClientSecret = Configuration["Authentication:Google:ClientSecret"]; });

I'm currently using this and am affected by this issue.

[rhubrightj]

Definitely a breaking change on my site... I hope it's as simple as just updating Microsoft.Owin.Security.Google NuGet pkg when they are able to update it.

[ThoughtHopper]

Yup this will break our stuff.
Google has really screwed us up by pushing the date to end of march.

I tried changing the endpoints to the ones given back by the well-known configuration:
https://accounts.google.com/.well-known/openid-configuration

The current code works until it tries to retrieve the userinfo.
The response is not the same format as the response given by google+ apis.

For users like us, we do not really need to poke the userinfo endpoint, the id_token already contains information about the user. I guess in some cases it may be easier to poke the user endpoint than to actually verify the id_token.

I hope there is an answer soon from the owners of the project, Google has threaten to start failing requests as early as January 28,2019.

[muratg]

cc @DamianEdwards @Eilon @davidfowl FYI.

[4deeptech]

I confirmed that manually disabling the Google+ API in their developer console does indeed break the process. I tried enabling their People API hoping they may do some magic on their side to redirect the request but no luck there. According to their documentation, the People API allows for the profile, email, and some other related profile scopes: https://developers.google.com/people/v1/how-tos/authorizing#OAuth2Authorizing

[Tratcher]

Workaround, all the user info has changed format so you need to remap everything:

    .AddGoogle(o =>
            {
                o.ClientId = Configuration["google:clientid"];
                o.ClientSecret = Configuration["google:clientsecret"];
                o.UserInformationEndpoint = "https://www.googleapis.com/oauth2/v2/userinfo";
                o.ClaimActions.Clear();
                o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "id");
                o.ClaimActions.MapJsonKey(ClaimTypes.Name, "name");
                o.ClaimActions.MapJsonKey(ClaimTypes.GivenName, "given_name");
                o.ClaimActions.MapJsonKey(ClaimTypes.Surname, "family_name");
                o.ClaimActions.MapJsonKey("urn:google:profile", "link");
                o.ClaimActions.MapJsonKey(ClaimTypes.Email, "email");
                o.ClaimActions.MapJsonKey("urn:google:image", "picture");
            })

This should work with ASP.NET Core 2.0 and later.

Does anybody here have interest in 1.0 or 1.1?

I'll follow up on the the Microsoft.Owin components.

[trurl123]

Workaround, all the user info has changed format so you need to remap everything:

This does not work.
"Error loading external login information. "

[theaswanson]

Workaround, all the user info has changed format so you need to remap everything:

This does not work.
"Error loading external login information. "

Can confirm: I get the same error on my end. Using ASP.NET Core 2.1.

[Tratcher]

@HaoK is Identity looking for the NameIdentifier claim? I wonder if sub is the same as the old Id claim.

What if you add:
o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");

Ah, yes, there it is. https://github.com/aspnet/Identity/blob/fcc02103aa10dcdd8759e0463cac2717114f3c1e/src/Identity/SignInManager.cs#L611

Edit: I updated my sample above.

[theaswanson]

@HaoK is Identity looking for the NameIdentifier claim? I wonder if sub is the same as the old Id claim.

What if you add:
o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");

Ah, yes, there it is. https://github.com/aspnet/Identity/blob/fcc02103aa10dcdd8759e0463cac2717114f3c1e/src/Identity/SignInManager.cs#L611

Adding o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub"); did the trick! Login is working now with Google+ API disabled.

[Tratcher]

Did it recognize you as the same user, or as a new user?

[theaswanson]

Same user. I used two different accounts and both logged in correctly.

[HaoK]

Yeah we use the Name identifier to use as the key to associate logins in identity, and the email which we use as the user name

[Tratcher]

Updates for Microsoft.Owin.Security.Google are being tracked at https://github.com/aspnet/AspNetKatana/issues/251. I've posted a temporary workaround there.

[dwdickens]

some of us are still using asp.net core 1.1 - interested if there is a simple work around for us as well. Thanks for the short notice google!

[Tratcher]

@dwdickens the Microsoft.Owin example I posted here should easily adapt for Asp.Net Core 1.0 and 1.1. Let me know if it gives you any trouble.

[ADefWebserver]

the Microsoft.Owin example I posted here should easily adapt for Asp.Net Core 1.0 and 1.1. Let me know if it gives you any trouble.

I can confirm it works in .Net 4.5.2 if you update Microsoft.Owin.Security.Google to version 4.0.0

[earllocker]

Workaround, all the user info has changed format so you need to remap everything:

    .AddGoogle(o =>
            {
                o.ClientId = Configuration["google:clientid"];
                o.ClientSecret = Configuration["google:clientsecret"];
                o.UserInformationEndpoint = "https://openidconnect.googleapis.com/v1/userinfo";
                o.ClaimActions.Clear();
                o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");
                o.ClaimActions.MapJsonKey(ClaimTypes.Name, "name");
                o.ClaimActions.MapJsonKey(ClaimTypes.GivenName, "given_Name");
                o.ClaimActions.MapJsonKey(ClaimTypes.Surname, "family_Name");
                o.ClaimActions.MapJsonKey("urn:google:profile", "profile");
                o.ClaimActions.MapJsonKey(ClaimTypes.Email, "email");
                o.ClaimActions.MapJsonKey("urn:google:image", "picture");
            })

This should work with ASP.NET Core 2.0 and later.

Does anybody here have interest in 1.0 or 1.1?

I'll follow up on the the Microsoft.Owin components.

This worked great. Just an FYI that given_name and family_name should be all lowercase otherwise it won't pull that information correctly. Thanks.

[Tratcher]

@earllocker I updated the sample, thanks.

[kanadaj]

@Tratcher the hotfix works for some users, but for some others it returns with a

System.Exception: The oauth state was missing or invalid.

Not sure why exactly. Maybe missing profile picture? I don't have enough info to figure out right now.

[Tratcher]

That error doesn't seem directly related. It may be a secondary error. Turn on you application logs to see if there's anything else. Also capture a fiddler trace file.

[alanleouk]

I modified my code to use the endpoints provided by https://accounts.google.com/.well-known/openid-configuration but hit a problem where the response from the user information endpoint wasn't deserialising. I modified my user information endpoint to https://www.googleapis.com/oauth2/v2/userinfo which seems to be compatible with the old mappings. I have also taken out the profile scope for my implementation.

Shouldn't this library get it's configuration from https://accounts.google.com/.well-known/openid-configuration in the future?

authenticationBuilder.AddGoogle(options =>
{
    options.ClientId = Configuration["auth:google:clientid"];
    options.ClientSecret = Configuration["auth:google:clientsecret"];

    options.AuthorizationEndpoint = "https://accounts.google.com/o/oauth2/v2/auth";
    options.TokenEndpoint = "https://oauth2.googleapis.com/token";
    options.UserInformationEndpoint = "https://www.googleapis.com/oauth2/v2/userinfo";

    options.Scope.Clear();
    options.Scope.Add("openid");
    options.Scope.Add("email");

    options.ClaimActions.Clear();
    options.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "id");
    options.ClaimActions.MapCustomJson(ClaimTypes.Email, GoogleHelper.GetEmail);
});

[kanadaj]

That error doesn't seem directly related. It may be a secondary error. Turn on you application logs to see if there's anything else. Also capture a fiddler trace file.

There is no secondary error. Also, don't have Fiddler in the pipeline so can't really make logs with that.

Generally speaking though, I find that External Login errors seem very hard to debug since the errors are very generic. "Correlation failed" and "The oauth state was missing or invalid." aren't really actionable. A more detailed logging setup built into the pipeline would make things much easier...

[Tratcher]

@kanadaj agreed, though the trouble with those situations is that the error usually occurs on the client or IDP, the server doesn't know why it happened.

[Tratcher]

@alanleouk the .well-known endpoint is for OpenIdConnect, not OAuth2 (though the two are related). There is a proposal to replace this library with the existing OpenIdConnect implementation in a future release.

[thomaslevesque]

Using OpenID Connect is a good alternative anyway. It's just as easy to setup, you don't need a client secret, and you can get all the information from the id_token without requiring an extra API call (just add email to the requested scopes to get the email address)

[kanadaj]

@Tratcher The problem is that even if Sentry (what I use) picks up the request details, it's usually still just encrypted tokens and such. I imagine it'd be the same with Fiddler. So it'd be helpful to just log everything that the server actually knows about the login attempt.

[Tratcher]

@thomaslevesque you don't need a client secret? Last I checked Google only supported the code response type so the client secrete was required to exchange the code for an idtoken. I double checked and they do support token flows, but only for the client.

https://developers.google.com/identity/protocols/OpenIDConnect#authenticationuriparameters

response_type | (Required) | If the value is code, launches a Basic flow, requiring a POST to the token endpoint to obtain the tokens. If the value is token id_token or id_token token, launches an Implicit flow, requiring the use of Javascript at the redirect URI to retrieve tokens from the URI #fragment.

[Tratcher]

@kanadaj let's take that discussion to another thread.

[thomaslevesque]

@thomaslevesque you don't need a client secret? Last I checked Google only supported the code response type so the client secrete was required to exchange the code for an idtoken. I double checked and they do support token flows, but only for the client.

You can get an id_token directly, without needing an authorization code. Just use id_token as the response_type (this is the default when you use AddOpenIdConnect). In this case you don't need the client secret.

[thomaslevesque]

@Tratcher I see the Google documentation is misleading. It says:

If the value is token id_token or id_token token, launches an Implicit flow, requiring the use of Javascript at the redirect URI to retrieve tokens from the URI #fragment.

But I just tried it in my app, and it works just fine. Probably because it's using form_post as the response_mode, not fragment.

[Tratcher]

@thomaslevesque Ah, it does work. Well there's an undocumented feature for us.

[Tratcher]

See https://github.com/aspnet/AspNetCore/pull/6338 for the proposed patch. Note this uses a different endpoint than proposed earlier. I've updated the mitigation above to match.

[AnthonyMascia]

See #6338 for the proposed patch. Note this uses a different endpoint than proposed earlier. I've updated the mitigation above to match.

Is there any specific reason you changed the new endpoint? Is it because the current OpenID Connect endpoint might be replaced in the future?

[Tratcher]

@AnthonyMascia this one seemed more appropriate for an OAuth2 component, and it's also a later version of the API. The prior one was an OpenIdConnect specific endpoint and an older API version.

[gilm0079]

@Tratcher Thanks for submitting the fix for this. I'll keep an eye on the pull request. https://github.com/aspnet/AspNetCore/pull/6338 and re-post a question in https://github.com/IdentityServer/IdentityServer4/issues/2931 to see if the IdentityServer4 build will need to reference the new core update.

[lostllama]

@Tratcher Do you know what the ETA for the .NET Core release of this fix is (since the message from Google that you quoted says it will start being phased out from Monday next week)? Am I correct to assume that it's in the Microsoft.AspNetCore.All package?

[Tratcher]

@lostllama see https://github.com/aspnet/AspNetCore/issues/6486.

It's unclear what they meant by "intermittent failures", but in the worst case there are workarounds included in https://github.com/aspnet/AspNetCore/issues/6486 as well.

[funkysoulbro]

Hi all,
Today Google sent another message regarding the shutdown:

...It includes Google+ OAuth scope requests, which are also affected by the Google+ shutdown. A prior email sent to active API callers did not include information about OAuth requests....

From examining Google Console Platform, there are "OAuth consent screen" tab on project's credentials page. Three scopes are listed: email, profile, openid. If you hover over an "openid", the following URL can be seen: https://www.googleapis.com/auth/plus.me.

When quering https://www.googleapis.com/oauth2/v4/token, it was found out that this scope block in sent in body:
_"scope": "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/plus.me"_

The following code can be added to Startup.cs file:

.AddGoogle(options =>
                {
                    ...
                    options.Scope.Remove("openid");
                    ...
                });

These "plus.me" scope disappears in request body afterwards. This scope doesn't seem to be needed as email & profile scopes cover data that "plus.me" does.

Hope it helps.

[DenitsaGencheva]

@funkysoulbro I was just wondering if we should remove the openid scope. In the Google documentation it is said:

For a basic request, specify the following parameters:

client_id, which you obtain from the API Console.
response_type, which in a basic request should be code.
scope, which in a basic request should be openid email.

So to me, it looks like it is a mandatory scope for being able to authenticate a user.

[Tratcher]

Their scope docs for Google Sign-in also list profile, email, and openid. I recommend leaving openid enabled until we can confirm if it causes any functional problems.

[ADefWebserver]

I recommend leaving openid enabled until we can confirm if it causes any functional problems.

@Tratcher I hope you can report back and tell us what to do because I feel like my sites are about to break 😒 I am running some older .NET MVC sites, so I implemented the hotfix (that seems to be working because in te Google administration panel it is not registering any hits on the Google+ API)

[Tratcher]

I found this official migration guide that says profile email openid are the correct scopes. The scopes plus.login plus.me plus.profile.emails.read are the obsolete ones.

[lostllama]

Dear Developer,

Earlier this week we sent you an email related to your projects that will be impacted by the 
Google+ API shutdown, which also affects requests for Google+ OAuth scopes.

The email listed that one or more of your projects are requesting the “plus.me” scope,
and would thus be affected. We would like to clarify that only projects directly requesting
the “plus.me” scope are affected. This scope may have been listed in some emails, even
if not directly requested by your project. We apologize for any confusion caused.

If you are directly requesting the “plus.me” scope, any other Google+ OAuth scopes, or making any
Google+ API calls, please ensure that you remove these requests from your project before
March 7, 2019.

To see if your project is directly requesting the “plus.me” or any other Google+ OAuth scopes:

If your project is written in Google Apps Script, you can view which scopes your project is
requesting by reviewing your project properties in App Script Editor.
If your project is not written in Google Apps Script, please check your code for references to 
“plus.me” in OAuth scope requests. We recommend that you review projects using any 3rd-party 
libraries that support sign-in or social functionality, as these may also be affected by the
shutdown.

Thanks for being a valued Google+ Developer.

Sincerely,
The Google+ API team

[EricRazon12]

are we able to fix this by updating the Microsoft.Owin.Security.Google from 4.0.0 to 4.0.1?
https://www.nuget.org/packages/Microsoft.Owin.Security.Google/

please advice. Thank you

[Tratcher]

Yes.

[Tr4pSt3R]

Hi All,

I'm sure most of you have received an email from Google announcing the deprecation of Google+ APIs. From March 7th, 2019 OAuth requests to this authentication API will be shutdown completely. 

As per the following documentation snippet from Google changing scopes should be enough to continue beyond the deprecation date without any further issue.


Nonetheless, I have stumbled upon an issue where the new API scopes of my AppMaker application has changed to the new scopes. However, the changes still makes reference to the deprecated open.id 

When I hover over the new scopes the openid scope still contains URL link to the soon-to-be deprecated Google+ API.

New Scopes:
• email (https://www.googleapis.com/auth/userinfo.email)
• profile (https://www.googleapis.com/auth/userinfo.profile)
• openid (https://www.googleapis.com/auth/plus.me)
My question is, will my AppMaker application continue to work beyond the deprecation date?

Any help will be appreciated as there's currently no useful information on the web. 

[Tr4pSt3R]

@Tratcher are these New scopes with their respective URLs enough, please?

New Scopes:
•   email (https://www.googleapis.com/auth/userinfo.email)
•   profile (https://www.googleapis.com/auth/userinfo.profile)
•   openid (https://www.googleapis.com/auth/plus.me)

[Tratcher]

Yes, see https://github.com/aspnet/AspNetCore/issues/6069#issuecomment-459079871

[Tr4pSt3R]

Thanks @Tratcher 👍 👍