Google-api-python-client: I am having trouble retrieving a decoded version of the message when I call GetMimeMessage()

[ctedunet]

I asked this question on stack overflow a few times (here) but have never been able to get a response. So I apologize if this is not the best place to be posting this but I really don't know who tot turn to anymore. Anyways here is my problemL

In my script I need to extract a set of emails that match some query. I decided to use GMail's API python client for this. Now, my understanding was that the GetMimeMessage() was supposed to return a set of decoded base 64 messages. Here is my code:

def GmailInput():

    credentials = get_credentials()
    http = credentials.authorize(httplib2.Http())
    service = discovery.build('gmail', 'v1', http=http)
    defaultList= ListMessagesMatchingQuery(service, 'me', 'subject:infringement label:unread ')
    print(defaultList)
    for msg in defaultList:
        currentMSG=GetMimeMessage(service, 'me', msg['id'])
        ....then I parse the text of the emails and extract some things

The problem is, I am unable to actually parse the message body because GetMimeMessage is not returning a base64 decoded message. So what I am actually parsing ends up being completely unreadable by humans.

I find this peculiar because GetMimeMessage (copied below for convenience) literally does a url-safe base 64 decode of the message data. Anyone have any suggestion? Im really stumped on this.

def GetMimeMessage(service, user_id, msg_id):
  """Get a Message and use it to create a MIME Message.

  Args:
    service: Authorized Gmail API service instance.
    user_id: User's email address. The special value "me"
    can be used to indicate the authenticated user.
    msg_id: The ID of the Message required.

  Returns:
    A MIME Message, consisting of data from Message.
  """
  try:
    message = service.users().messages().get(userId=user_id, id=msg_id, format='raw').execute()

    print ('Message snippet: %s' % message['snippet'])

    msg_str = base64.urlsafe_b64decode(message['raw'].encode('ASCII'))

    mime_msg = email.message_from_string(msg_str)

    return mime_msg
  except errors.HttpError, error:
    print ('An error occurred: %s' % error)

[theacodes]

Where does GetMimeMessage come from? It's not from this library is it?

[ctedunet]

GetMimeMessage I copied over from the Google API reference site. It is copied in as its own function just before my GmailInput() function, and from this function I call GetMimeMessage

[theacodes]

Do you have a link to this? I don't have a lot of context/experience with the gmail API.

[ctedunet]

The GetMimeMessage function is copied in my original post but can also be found here: https://developers.google.com/gmail/api/v1/reference/users/messages/get

[theacodes]

@ctedunet why close? Did you find a solution?

[ctedunet]

Yes, my solution was to get the content types of the mime message then decode them separately. Before, I was trying to base 64 decode the entire message object all at once. This way, I am able to decode the payload piece by piece. Here is my code, it really is just an addition to the Google-supplied GetMimeMessage function :

def GetMimeMessage(service, user_id, msg_id):
  """Get a Message and use it to create a MIME Message.

  Args:
    service: Authorized Gmail API service instance.
    user_id: User's email address. The special value "me"
    can be used to indicate the authenticated user.
    msg_id: The ID of the Message required.

  Returns:
    A MIME Message, consisting of data from Message.
  """
  try:
    message = service.users().messages().get(userId=user_id, id=msg_id,
                                             format='raw').execute()
    print( 'Message snippet: %s' % message['snippet'].encode('ASCII'))
    MessageBody=[]
    msg_str = base64.urlsafe_b64decode(message['raw'].encode('ASCII'))
    mime_msg = email.message_from_string(msg_str)
    for parts in mime_msg.walk():
      mime_msg.get_payload()
      print(parts.get_content_type())
      if parts.get_content_type() == 'application/xml':
        mytext= base64.urlsafe_b64decode(parts.get_payload().encode('UTF-8'))
      if parts.get_content_type() == 'text/plain':
        myMSG=base64.urlsafe_b64decode(parts.get_payload().encode('UTF-8'))
        MessageBody.append(myMSG)
        with open('messages.json','w') as jsonfile:
          json.dump(MessageBody,jsonfile)

  except errors.HttpError, error:
    print ('An error occurred: %s' % error)
  return my text

[theacodes]

Great, thanks for posting your solution. This can hopefully help any other users who run into the same issue.

[keepfocused]

Here is the solution:
Gmail API - "Users.messages: get" method has in response message.payload.body.data parted base64 data, it's separated by "-" symbol. It's not entire base64 encoded text, it's parts of base64 text. You have to try to decode every single part of this or make one mono string by unite and replace "-" symbol.