Google-api-nodejs-client: Failed sending mail with service account and JWT auth

[danthareja]

I'm trying to send an email using a service account and JWT authentication and keep getting and error with a very unhelpful message: { code: 500, message: null }

This code snippet is from the following StackOverflow link: http://stackoverflow.com/questions/25207217/failed-sending-mail-through-google-api-in-nodejs

It seems like the solution there was to change the key in the parameters to resource instead of message but it's not working for me. This is strange because in the JS example in the docs (https://developers.google.com/gmail/api/v1/reference/users/messages/send) it claims the key is still message

I'm authenticating with

var jwtClient = new google.auth.JWT(config.SERVICE_EMAIL, config.SERVICE_KEY_PATH, null, config.ALLOWED_SCOPES);

then sending an email with

jwtClient.authorize(function(err, res) {
  if (err) return console.log('err', err);

  var email_lines = [];

  email_lines.push("From: \"Some Name Here\" <[email protected]>");
  email_lines.push("To: [email protected]");
  email_lines.push('Content-type: text/html;charset=iso-8859-1');
  email_lines.push('MIME-Version: 1.0');
  email_lines.push("Subject: New future subject here");
  email_lines.push("");
  email_lines.push("And the body text goes here");
  email_lines.push("<b>And the bold text goes here</b>");

  var email = email_lines.join("\r\n").trim();

  var base64EncodedEmailSafe = new Buffer(email).toString('base64').replace(/\+/g, '-').replace(/\//g, '_');

  var params = {
    auth: jwtClient,
    userId: "[email protected]",
    resource: {
      raw: base64EncodedEmailSafe
    }
  };

  gmail.users.messages.send(params, function(err, res) {
    if (err) console.log('error sending mail', err);
    else console.log('great success', res);
  });
}

What am I missing?

[ryanseys]

Hmm, weird. @Alaneor is there any way this null error could be a side
effect of the most recent change?

[danthareja]

Also as one more added piece of context, I can copy the same string that's stored in base64EncodedEmailSafe into the Google APIs Explorer for Users.messages.send and it works fine.

And the comments in https://github.com/google/google-api-nodejs-client/blob/master/apis/gmail/v1.js seem to say that resource is the proper key too:

[ryanseys]

Ah, right. You cannot authorize Gmail API requests with JWT, you must use OAuth 2.0 because it needs to be auth'd to a specific user. Or else you'd be able to do some really shady things like send messages impersonating someone else. The Google APIs Explorer is authenticated with OAuth 2.0 that's why it works. See https://developers.google.com/gmail/api/auth/about-auth for more information.

[danthareja]

Oh, I see. Thanks for the explanation @ryanseys. Is there any way I can send mail with a service account then, maybe without impersonation? I'm also still a little confused as to why the code snippet on http://stackoverflow.com/questions/25207217/failed-sending-mail-through-google-api-in-nodejs worked at one point. Was this a recent change?

[ryanseys]

As you can see in the Stack Overflow link, auth: OAuth2Client, they are using the OAuth2 client to authenticate. There is currently no way for you to send messages using the GMail API without authenticating as a specific GMail user. Service accounts do not have access to GMail the same way that regular users do.

[danthareja]

Of course, thank you very much for your help!

[ryanseys]

No problem! Glad I could help.

[robertrossmann]

@danthareja is the object you are getting ({error: 500, message: null}) an instance of Error? If so, it would be great if you could post the stack trace so we can figure out (hopefully) why the message is empty.

If you are not using the latest version (1.1.0), it would be great if you could upgrade and try again - the objects we now return are actual instances of Error, which means there will be a stack trace that we can then take a look at.

[stken2050]

This leads misunderstanding.

Gmail or Google API should work with requests with JWT/service account,

http://stackoverflow.com/questions/29307965/gmail-api-gives-failedprecondition-error
http://stackoverflow.com/questions/30991430/how-to-access-gmail-api-from-my-own-gmail

I guess it's a specific problem with this library and I try to solve by myself right now.

[kunokdev]

@kenokabe Did you find any solution yet?
@ryanseys I am trying to fetch all emails under our company. I have full admin account. Is it true that I can't do this by using JWT auth in node.js client lib? And if so, which is alternative method to avoid any direct user interaction ?

[stken2050]

Sort of surprised to have found my own post about this issue having the same trouble and googling for my other project after years.

@kunokdev Yes.
We need to set "subject (or sub) field" of google.auth.JWT which should be identical to a gmail address to be managed.
This is rarely explained for g-suite/gmail API, and an only document I could find is:

https://developers.google.com/identity/protocols/OAuth2ServiceAccount

Additional claims

In some enterprise cases, an application can request permission to act on behalf of a particular user in an organization. Permission to perform this type of impersonation must be granted before an application can impersonate a user, and is usually handled by a domain administrator. For more information on domain administration, see Managing API client access.

To obtain an access token that grants an application delegated access to a resource, include the email address of the user in the JWT claim set as the value of the sub field.

Unfortunately, this field does not accept array, so to manage multiple gmail address under the same domain, you need to obtain each jwtClients.

After all, a single service account with a single credential works for multiple gmail addresses under the same domain.

https://developers.google.com/admin-sdk/directory/v1/guides/delegation

Here is a very clean node.js sample to get started.

(() => {
  "use strict";
  //============================
  const google = require('googleapis');
  const google_key = require("/your/google-key.json"); //download from google API console

  const jwtClient = new google.auth.JWT(
    google_key.client_email,
    null,
    google_key.private_key,
    ["https://mail.google.com/"], //full access for now, you can restrict more
    '[email protected]' // subject (or sub) <-----------------------
  );

  jwtClient.authorize((err, tokens) => (err
    ? console.log(err)
    : (() => { //--------------------------------
      console.log("Google-API Authed!");
      const gmail = google.gmail({
        version: "v1",
        auth: jwtClient
      });
      gmail.users.messages.list({
        userId: '[email protected]'
      }, (err, messages) => {
        //will print out an array of messages plus the next page token
        console.log(err);
        console.dir(messages);
      });
    })() //--------------------------------
  //======================
  ));
//============================
})();

[ryanface]

you save me

[kunokdev]

Yes this is a bit old.
So my problem was that I was trying to authenticate users that I wanted to fetch from, but instead I just had to authenticate admin user that was previously delegated (in google console) and has permissions for given scopes (see constants at the top of code)

This is module that I use:

const Google = require('googleapis'),
      scopes = [
        'https://www.googleapis.com/auth/gmail.readonly',
        'https://www.googleapis.com/auth/admin.directory.user.readonly',
        'https://www.googleapis.com/auth/admin.directory.group'
      ],
      key = require('../service_key.json');

var exports = module.exports;

exports.jwt = function(email){
  return new Promise((resolve, reject)=>{

    let authClient = new Google.auth.JWT(
      key.client_email,
      key,
      key.private_key,
      scopes,
      email
    )

    authClient.authorize((err, tokens) => {
      if (err) {
        console.error(err);
        process.exit(1)
      }
      resolve(authClient)
    })

  })
}

which is then called within code like this:

args.init()
    .then(database.connect)
    .then(users.provide)
    .then(auth.jwt)
...

it will resolve auth object that you use in your requests.

As for selection of users this is how my method looks like (Users.provide)
Basically it reads from params given to script in CLI or uses default user.

exports.provide = () => {
  return new Promise((resolve, reject)=>{
    const userToProvide = status.args[3]
                      ? status.args[3]
                      : '[email protected]'
    const validator = require('./validator.js')
    validator.email(userToProvide)
    resolve(userToProvide)
  })
}

Hope it gives a hint if somebody gets stuck within similar problem.

[vigen-b]

@stken2050 :+1:

The staff with subject did the job.