Google-api-dotnet-client: "Client is unauthorized to retrieve access tokens using this method" w/ Service Account

[vchtelmakh]

ENVIRONMENT DETAILS
OS: Windows 10
.NET version: .net core 3.1
Package name and version: using Google.Apis.Auth.OAuth2; (1.45.0)

---

END GOAL: Get OAuth2 token for Gmail User via service account created in free gmail account.

PROBLEM / CAUSE: I have an existing gmail account that is used by a .NET Core Application to send and receive emails - [email protected]. The application now logs in to gmail with [email protected] and the password to send emails. This requires "less secure access" to be enabled, which doesn't work when the project runs on Azure.

Consequently, the solution is to start using OAuth 2.0 for access.

I paid for a G-Suite account [email protected].

---

Questions which may make this easy to answer & close:

1. Will the service account authentication work without domain verification on the G-Suite account? If not, that's my problem and this can be closed, but I would like a definitive answer if possible.

2. Will I be able to authenticate a service account from [email protected] (free) with [email protected] (paid G-Suite account)? The documentation doesn't seem to mention it anywhere, but posts on stackoverflow said this will work. (i.e., is doing what's shown in the screenshot below enough?)

---

If the above should theoretically work, here is my code which throws an exception when I try to fetch a JWT to use:

    public async Task<IActionResult> GetToken()
    {

      using (var stream = new FileStream("my-magical-key-file.json", FileMode.Open, FileAccess.Read))
      {
        ServiceAccountCredential srvsAcctCred = 
          GoogleCredential.FromStream(stream)
                          .CreateScoped(new[] {
                            "https://mail.google.com/"
                          })
                          .CreateWithUser(SMTPConfigs.GMAIL_WITH_FORWARDED_LEADS.UserName)
                          .UnderlyingCredential as ServiceAccountCredential;

        bool result = await srvsAcctCred.RequestAccessTokenAsync(CancellationToken.None);
      }

The exception is thrown on RequestAccessTokenAsync and is:

Google.Apis.Auth.OAuth2.Responses.TokenResponseException: 'Error:"unauthorized_client", Description:"Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested.", Uri:""'

Commenting out this line .CreateWithUser(SMTPConfigs.GMAIL_WITH_FORWARDED_LEADS.UserName) resolves the error, but obviously doesn't create a token for the user.

Here is more information on my workflow / what I have done at a glance:

[jskeet]

Assigned to Amanda as she knows more about the auth flows than I do these days.

But I do wonder whether the problem is the GCP project that the service account is for. Have a look a the project-id in the service account file - is that the project that the G-Suite account is associated with?

[amanda-tarafa]

I'll take a look tomorrow morning London time.

[vchtelmakh]

Jon, by the service account file, I assume you mean the .json key file that I created for the service account. The project_id there is just trustaff-email-scraper:

  "type": "service_account",
  "project_id": "trustaff-email-scraper",
  "private_key_id":...

The free account has the following project structure, which contains the service account:

The G-Suite account has NO projects in it. The only way G-Suite knows about the service account is that it has the service account's UniqueId.

Thanks @amanda-tarafa - no rush, I know it's late for you guys. This can definitely wait.

[amanda-tarafa]

I think I know what the problem is. The scope you are using is restricted and means you'd have to go through a security assessment. Can you try with the send scope only? Here's more info about Gmail's API scopes: https://developers.google.com/gmail/api/auth/scopes

[vchtelmakh]

1. Deleted the service account authorization on the Authorize API clients page in the G-Suite account.
2. Added back the service account authorization with https://www.googleapis.com/auth/gmail.send API scope only.
3. Changed the C# code to only request that scope:

                          .CreateScoped(new[] {
                            "https://www.googleapis.com/auth/gmail.send"
                          })

1. Got the same error.

There is a rumor going around stackoverflow that these changes to Auth take some time to propagate, so I will try again later today and post an update if it works.

[amanda-tarafa]

Ok update if you see anything different, else I'll take a look tomorrow.

[amanda-tarafa]

The more I look into this the more I think it has to do with configuration and not with the Auth library. A few things to check/try if you can, in that order:

• What you need to grant access to in the Manage API client access is not to the service account but to the OAuth client associated with it, so the ID you need to grant the scopes to is the OAuth client ID. Maybe you've done that but you have referred to it as the service account ID in your comments and I just wanted to double check. Can you confirm?
  
  
  
  • Can you/have you log in manually with the G-Suite account that you are trying to impersonate and access Gmail?
  
  

[vchtelmakh]

I think there is a good chance there is a config issue. I tried to avoid coming here with one, but after reading all the docs, articles, SO posts on the subject, etc, I am having trouble finding what I am doing wrong. I gave more info in my first post about my set up - that may help you see what's wrong at a glance, especially the two things I listed in "Questions which may make this easy to answer & close"

To reply directly to the points above:

• I am including two screenshots of the id I am using, in red - I think we are talking about the same thing (both screenshots are from the free gmail account):

• Can you/have you log in manually with the G-Suite account that you are trying to impersonate and access Gmail?

This might be the point of confusion on my part - the account I am trying to impersonate to access Gmail is NOT a G-Suite account, it's a normal gmail account, in which I created a project, service account, and granted that service account domain-wide delegation.

The ONLY thing I am doing on the paid G-Suite account is setting things in "Manage API Client Access" as in the screenshot in the post above.

I can login to the free account, and our API has been able to send/receive emails with it by authentication via un/pw with less secure authentication enabled. We are trying to disable less secure authentication and move to OAuth.

I may be a little late in replying tomorrow due to our different time zones, I am in US East.

[LindaLawton]

Client is unauthorized to retrieve access tokens using this method" w/ Service Account

Means that the domain wide delegation on the gSuite domain email has not been set up properly.

Will the service account authentication work without domain verification on the G-Suite account? If not, that's my problem and this can be closed, but I would like a definitive answer if possible.

(Note i am going to assume you mean without domain wide delegation on the gsuite and not verification)

Service accounts dont work with gmail unless you set up domain wide delegation to a Gsuite account. The reason being is that a service account is its own user you need to delegate its permission to access your gmail account. This will only work with a gsuite domain email address.

Will I be able to authenticate a service account from [email protected] (free) with [email protected] (paid G-Suite account)?

There is no way to share / grant another user permission to access your standard gmail account. So there is no way for you go delegate the permissions for [email protected] to access [email protected].

I recommend using standard Oauth2 if this is a standard gmail account and saving the refresh token and storing it with your application it will then be able to request a new access token when ever it needs. As long as the refresh token is used regularly you should not have any issues with this solution.

https://developers.google.com/gmail/api/guides/delegate_settings

[amanda-tarafa]

This might be the point of confusion on my part - the account I am trying to impersonate to access Gmail is NOT a G-Suite account, it's a normal gmail account, in which I created a project, service account, and granted that service account domain-wide delegation.

Yes, I can confirm this won't work, you can impersonate G-Suite accounts but not Gmail accounts, sorry for not catching this earlier. Take into account that:

Once a service account has been granted an API scope, it can make API requests using OAuth 2.0 by impersonating any user (including a super admin) in the G Suite account that:

• Has permissions for the API being called (via a pre built or custom role)
• For some G Suite APIs (no hard list), the user being impersonated has to have logged in at least once and accepted the G Suite Terms of Service before using the API (that's why I was asking if you had logged in with the G-Suite account).
• The UI and Reports API does not explicitly show that the action was done by an impersonated user. The log entry appears as though the actions were performed by the user as normal.

I'll wait for you tou acknowledge before closing but I believe this to have been answered, and unrelated to the Auth library.

[vchtelmakh]

Thank you all for the help.

I recommend using standard Oauth2 if this is a standard gmail account and saving the refresh token and storing it with your application it will then be able to request a new access token when ever it needs. As long as the refresh token is used regularly you should not have any issues with this solution.

@LindaLawton Just to clarify, you are recommending that I obtain an access / refresh token through some means (either the 3-legged client flow or otherwise) and run the app with those credentials, correct? Going to try that. Really, really appreciate the advice.

I'll wait for you to acknowledge before closing but I believe this to have been answered, and unrelated to the Auth library.

Thank you for the help! I am really hoping this will save people some time. I will link https://stackoverflow.com/questions/33233694/gmail-api-can-i-send-email-using-the-service-account to this thread - I think it has been misleading a lot of people into thinking it's possible to access a free gmail from a service account, as long as it was "authorized" by a G-Suite acct.