Google-api-dotnet-client: OAuth id_token missing information on refresh

[EatonZ]

I've run into an interesting issue (bug?) with Google OAuth/OpenID Connect authentication. While it's not _directly_ an issue with this project, I think it's the best place to report it since it has all the related authentication code, which in turn may cause confusion if other people use the authentication code in this project the same way I am. If there's a better place to report this, please let me know.

My app is a desktop app using the localhost loopback method to receive the authentication code returned from the browser to obtain tokens.

Starting from the beginning, the app launches https://accounts.google.com/o/oauth2/v2/auth with these query strings:

redirect_uri=http://localhost:13604/
response_type=code
scope=email openid profile
state
code_challenge_method=S256
code_challenge
login_hint

That works fine, so I get a token using https://www.googleapis.com/oauth2/v4/token:

redirect_uri=http://localhost:13604/
code_verifier
client_id
client_secret
grant_type=authorization_code

Since I provided OpenID scopes, I get an id_token in the response.
Looking at the token data, here's what's inside:

That all looks good.

Using the refresh_token we got back with id_token, I refresh using https://www.googleapis.com/oauth2/v4/token:

client_id
client_secret
grant_type=refresh_token

I get this id_token back:

Compare both id_token screenshots. The second one from the refresh is missing most of the identifying information. It appears there is a glitch with Google servers that makes it lose track of the scopes that were specified during the first authorization request. I wasn't able to figure out a way to reclarify these scopes during the refresh request, which makes sense from a security standpoint, but also reinforces the fact that this may indeed be a server glitch.

This is a significant issue with the seamless login I had in mind. The only workaround right now is to ditch refreshing the token and to go through the full authentication each time. Hopefully the glitch can be addressed by relaying this information to the right person, or another workaround suggested.

Happy new year!🎉

[chrisdunelm]

Thanks for reporting this. I will see if I can reproduce the problem, and make sure I understand what the expected behaviour is here.

[EatonZ]

Happy to provide a sample project if you cannot - just let me know.

[chrisdunelm]

@EatonZ Yes, if you can provide a project that'd be really helpful. Thanks.

[EatonZ]

@chrisdunelm See below for a sample project (using this library) you can run. Fill in your client info (create your OAuth credentials as "Other") and run it.

Google OAuth Sample.zip

[chrisdunelm]

@EatonZ Thanks, I'll look at this tomorrow.

[LindaLawton]

I tested this on the Oauth2 playground and sent these scopes as you showed

email openid profile

Initial request :

{
  "access_token": "Gls4BVey6jQ0o80RtiH9XgfeMMh71UJIP3ECI6Z4IPAvSFZOP8o06BUK6DHwn7tFwuF5bw40GV5KtvGUAfA4s9e58HdITK5THGtFxjaXSjUJqXPjyP6-78C1EKMp", 
  "token_type": "Bearer", 
  "expires_in": 3600, 
  "refresh_token": "1/-DuJCEuOxW6hn6veUZCll-HKvC8sJ8SFPvSnX9ecclU", 
  "id_token": "yJhbGciOiJSUzI1NiIsImtpZCI6IjFkYjAxNzVkMWM1ZjQ1YTlkZGEyNzlmYzkxYjY3YTEzOGJjMjA1NTUifQ.eyJhenAiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJhdWQiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJzdWIiOiIxMTcyMDA0NzU1MzI2NzI3NzUzNDYiLCJlbWFpbCI6ImxhdXJseTcxQGdtYWlsLmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJhdF9oYXNoIjoiS1hkTm8wcWNRbVZ1NnVVcFNEa0VOdyIsImlzcyI6Imh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbSIsImlhdCI6MTUxNTA1NDg2NiwiZXhwIjoxNTE1MDU4NDY2LCJuYW1lIjoiTGluZGEgTGF3dG9uIiwicGljdHVyZSI6Imh0dHBzOi8vbGg1Lmdvb2dsZXVzZXJjb250ZW50LmNvbS8tYTFDV2xGbkE1eEUvQUFBQUFBQUFBQUkvQUFBQUFBQUFsMUkvVWN3UGFqWk91TjQvczk2LWMvcGhvdG8uanBnIiwiZ2l2ZW5fbmFtZSI6IkxpbmRhIiwiZmFtaWx5X25hbWUiOiJMYXd0b24iLCJsb2NhbGUiOiJlbiJ9.dlS2MgVFQzyorqJrlfHtvRBWq1u0kMM8rJjsPniNIJagk_7Ykdk1cQuWkfGMy-3I1zs_SAn3blY4E-fkNoi-69nGkyPYVoDcKGhA5vz43oj1DlfjBnzIDDUMFrpP9KmQuUOICBK-P7_6Llju-oCd_w88Vn_s7Hs_vHVHyaTjYGMMv2p3gpaQggFNOzNZNZGrO7ZYVuw-Zk51WPKNYCBeFTElu0_YOQ3sc8jJHK_uHkd19Db5mzAjAj9s0SqIj1i2DSfKKQ5fcw2cZjmKwnjKv_Cf-Ds-SSphl7ufZ_iuBcHtSxdlIL6PMeOz_fz2SKESHxX7YCpmbL9UkrExO8kxLQ"
}

Decoded Id token

{
  "azp": "407408718192.apps.googleusercontent.com",
  "aud": "407408718192.apps.googleusercontent.com",
  "sub": "117200475532672775346",
  "email": "[email protected]",
  "email_verified": true,
  "at_hash": "KXdNo0qcQmVu6uUpSDkENw",
  "iss": "https://accounts.google.com",
  "iat": 1515054866,
  "exp": 1515058466,
  "name": "Linda Lawton",
  "picture": "https://lh5.googleusercontent.com/-a1CWlFnA5xE/AAAAAAAAAAI/AAAAAAAAl1I/UcwPajZOuN4/s96-c/photo.jpg",
  "given_name": "Linda",
  "family_name": "Lawton",
  "locale": "en"
}

Response from Refresh of access token

{
  "access_token": "a29.Gls4Bcwy2WScXAPdh1AvzdIrL7cxtY64ayX3_fdLxVb9VUfRbfSKMoL7-bY5I8vy3VTMwCANfebdDdUJAF_wHm7Q6XMig3-UksDehQcxArFL5AImi_lOHtGGTWC1", 
  "token_type": "Bearer", 
  "expires_in": 3600, 
  "id_token": "yJhbGciOiJSUzI1NiIsImtpZCI6IjFkYjAxNzVkMWM1ZjQ1YTlkZGEyNzlmYzkxYjY3YTEzOGJjMjA1NTUifQ.eyJhenAiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJhdWQiOiI0MDc0MDg3MTgxOTIuYXBwcy5nb29nbGV1c2VyY29udGVudC5jb20iLCJzdWIiOiIxMTcyMDA0NzU1MzI2NzI3NzUzNDYiLCJlbWFpbCI6ImxhdXJseTcxQGdtYWlsLmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJhdF9oYXNoIjoiWXowY3VVS3dueWNPMFNHdVZDNmxPdyIsImlzcyI6Imh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbSIsImlhdCI6MTUxNTA1NDk2NCwiZXhwIjoxNTE1MDU4NTY0fQ.neo9LeiHdqVrHmiMSE1UrT4uvr9q-Qy92-tmiQXE6AkkW4wPVtRl9146DniLv4emqW-qB7wskh6SJa7s6DauZuWfzV9bkkyw2FOKYB6x3j1bOHf1083puKRBkOcAtCEgkGCNdxl_GiX9dNxtdvscKqDfL5dltXadfq39_qRbs3oHxMca4cQKfnXoheX32scWFtR5y8HpKaE7JyzLPll-t2LK_PsOTs73DKJ9gmqIqyE1APXuYccB3VyrOOYB5bTC5cSNuPW2_NpUZSxr3bXxZz2A1Ni3Yg2-PjoCRD36GyfBsd51JarpslPaEfwNzUGmxIUMOuILjkmMUhOHKcjmlA"
}

Decoded Id token

{
  "azp": "407408718192.apps.googleusercontent.com",
  "aud": "407408718192.apps.googleusercontent.com",
  "sub": "117200475532672775346",
  "email": "[email protected]",
  "email_verified": true,
  "at_hash": "Yz0cuUKwnycO0SGuVC6lOw",
  "iss": "https://accounts.google.com",
  "iat": 1515054964,
  "exp": 1515058564
}

Hes right the profile claims are missing from the id token in the response from a refresh token.

The email scope should add the following scope to the claim

"email": "[email protected]",

The profile scope should add the following scopes to the claim

  "name": "Linda Lawton",
  "picture": "https://lh5.googleusercontent.com/-a1CWlFnA5xE/AAAAAAAAAAI/AAAAAAAAl1I/UcwPajZOuN4/s96-c/photo.jpg",
  "given_name": "Linda",
  "family_name": "Lawton",
  "locale": "en"

I am leaning towards this being a bug in the identity servers response as email is still there. Which to me implies that the scopes should be returned in the claim from the refresh token as well. I am not aware of anything in the rfc for open id saying that the id token response from a refresh token wouldnt include all of the original claims.

I am going to ping someone on the team. I dont think this is an issue with this client library.

[LindaLawton]

I got a response they are looking into it.

[EatonZ]

I've observed that (2 times now) if you wait a while before refreshing the token, it will contain the claims. I waited overnight, refreshed, and it was a complete ID token.

Oddly, refreshing a second time right after brings back an incomplete ID token.

UPDATE: Wait period seems to be a lot less than "overnight". Maybe around an hour or so, it seems.
UPDATE 2: Seems to be random, actually.. A refresh after a few hours yields missing claims.

[EatonZ]

@LindaLawton Any news from the team? I continue to get the full claims at random times.

[LindaLawton]

Only that they would look into it TBH i dont expect to hear from them about bugs like this. My job is to report them they do what they want after that. @chrisdunelm or @jskeet may be able to get some additional information internally

[EatonZ]

@jskeet @chrisdunelm Any chance of getting more information on the status of a fix? It doesn't seem like anything has been done yet. Usually when I start working for the day, I get full claims once and that's it.
I'm just concerned it won't get fixed and it seems like a bug in Google's auth servers would be worth looking into.

[jskeet]

I'll start asking internally tomorrow - will see where that leads.

[LindaLawton]

I just got a response back from the team

Hi Linda,

This is working as intended.
The user profile information is guaranteed to be included in the authorization code exchange response which is used by developer to sign in the user. Developers are expected to cache/persist the user profile information until the next time they get a response containing profile information during refresh token exchange.
The reason it is implemented this ways are

• Profile information changes infrequently. Returning this data in every refresh token exchange which happens every one hour is a great waste of resources for both Google and developers.
• Profile information is usually persisted by developers in their user database. In most cases they're not necessary in the refresh token exchange.

[EatonZ]

@LindaLawton Thanks for the follow up.

I guess that makes sense. I'm curious why this is a "great waste of resources" though. It doesn't really make the request significantly larger to send over the network. But maybe on Google's scale it cuts down on network costs?

In my case it's a desktop application, but I guess I will add a settings field for cached profile information, and update that information as new information becomes present in the token.

Out of curiosity, is this behavior documented anywhere?

[LindaLawton]

I think we need to consider Google's scale, some developers myself included have been known to request a new access token every request. Even small reduction in claim size would probably make a big difference on their end.

I don't think its documented but I can suggest to them that they do. its worth noteing

[EatonZ]

@LindaLawton Makes sense, and yes they should document it.

I'm ready to close this issue now, just one more question - how often does profile information get returned during the refresh token exchange? Are there set times it is returned, or is it random?

[EatonZ]

@LindaLawton Can we get some final info on that so we can close this? ^

[jskeet]

I would suggest that it wouldn't be random, but that the team is unlikely to want to make any guarantees about it.

[LindaLawton]

I agree with @jskeet things may change in the future so while they agree to document it i dont think they are going to give us any hard and fast info.

_I just wish i could use this excuse with my developers who insist on getting the in the ID token every time_

[EatonZ]

Very well, let me know if they ever document this behavior.