Error 10053 When Sending Large Attachments using G

Question

I'm trying to send emails of various sizes using the Gmail API and the functions below.

Generally this works perfectly, however for attachments over around 10MB (which are rare but will happen) I recieve Errno 10053 which I think is because I timeout when sending the message including the large attachment.

Is there a way around this by say, specifying size or increasing the timeout limit? There's reference to size in the Gmail API docs, but I'm struggling to understand how to use in Python or whether it would even help.

def CreateMessageWithAttachment(sender, to, cc, subject,
                                message_text, file_dir, filename):
  """Create a message for an email.

  Args:
    sender: Email address of the sender.
    to: Email address of the receiver.
    subject: The subject of the email message.
    message_text: The text of the email message.
    file_dir: The directory containing the file to be attached.
    filename: The name of the file to be attached.

  Returns:
    An object containing a base64url encoded email object.
  """
  message = MIMEMultipart()
  message['to'] = to
  if cc != None:
    message['cc'] = cc
  message['from'] = sender
  message['subject'] = subject

  msg = MIMEText(message_text)
  message.attach(msg)

  path = os.path.join(file_dir, filename)
  content_type, encoding = mimetypes.guess_type(path)

  QCoreApplication.processEvents()

  if content_type is None or encoding is not None:
    content_type = 'application/octet-stream'
  main_type, sub_type = content_type.split('/', 1)
  if main_type == 'text':
    fp = open(path, 'rb')
    msg = MIMEText(fp.read(), _subtype=sub_type)
    fp.close()
  elif main_type == 'image':
    fp = open(path, 'rb')
    msg = MIMEImage(fp.read(), _subtype=sub_type)
    fp.close()
  elif main_type == 'audio':
    fp = open(path, 'rb')
    msg = MIMEAudio(fp.read(), _subtype=sub_type)
    fp.close()
  else:
    fp = open(path, 'rb')
    msg = MIMEBase(main_type, sub_type)
    msg.set_payload(fp.read())
    fp.close()
  QCoreApplication.processEvents()

  msg.add_header('Content-Disposition', 'attachment', filename=filename)
  message.attach(msg)
  return {'raw': base64.urlsafe_b64encode(message.as_string())}



def SendMessage(service, user_id, message, size):
  """Send an email message.

  Args:
    service: Authorized Gmail API service instance.
    user_id: User's email address. The special value "me"
    can be used to indicate the authenticated user.
    message: Message to be sent.

  Returns:
    Sent Message.
  """

  try:
    message = (service.users().messages().send(userId=user_id, body=message)
               .execute())
    QCoreApplication.processEvents()
    return message
  except errors.HttpError, error:
    pass

Answer 1

You need to use the MEDIA /upload option for things that large. Then you can send emails up to the max Gmail allows. Docs for how to use /upload: https://developers.google.com/gmail/api/v1/reference/users/messages/send

The 10MB limitation is not well documented.

Answer 2

I succeed to insert/send message with large file, pythons code. The google api documentations is not friendly for developers, and the "/upload" issue is totally unclear and not well documented, and it confusing a lot of developers.

The final line do the magic :)

def insert_message(service, message):
        try:
                if message['sizeEstimate'] > 6000000:
                        insert_large_message(service, message)
                else:
                        insert_small_message(service, message)
        except:
                print ('Error: ----type: %s, ----value: %s, ----traceback: %s ************' % (sys.exc_info()[0],sys.exc_info()[1],sys.exc_info()[2]))

def insert_small_message(service, message):
        body = {'raw': message['raw'],'labelIds':message['labelIds'],'internalDateSource':'dateHeader'}
        message = service.users().messages().insert(userId='me',body=body).execute()

def insert_large_message(service, message):
        b = io.BytesIO()
        message_bytes = base64.urlsafe_b64decode(str(message['raw']))
        b.write(message_bytes)
        body = {'labelIds':message['labelIds'],'internalDateSource':'dateHeader'}
        media_body = googleapiclient.http.MediaIoBaseUpload(b, mimetype='message/rfc822' )
        print('load big data!')
        message = service.users().messages().insert(userId='me',body=body,media_body=media_body).execute()

Answer 3

'g' is my authorized api context. The call method will invoke execute on the object. The important thing is the Media calls and using both the media_body and the body params. This causes the message to be inserted with the label INBOX, and it will allow at least a 24MB file.

I ended up with two copies because the read timeout was too short:

f fetch 8:9 (flags INTERNALDATE RFC822.SIZE)
* 8 FETCH (RFC822.SIZE 24000720 INTERNALDATE "19-Jul-2007 17:12:26 +0000" FLAGS (\Seen))
* 9 FETCH (RFC822.SIZE 24000720 INTERNALDATE "19-Jul-2007 17:12:26 +0000" FLAGS (\Seen))

Sample code:

import mailbox
import StringIO
import googleapiclient.http

f = 'my-mbox-file.mbox'
params = {}
params[ 'internalDateSource' ] = 'dateHeader'
for m in mailbox.mbox( f, create=False ):
    message_string = m.as_string()
    params[ 'body' ] = { 'labelIds': [ 'INBOX' ] }

    if len(message_string) > 6000000:
        s = StringIO.StringIO()
        s.write( message_string )
        params[ 'media_body' ] = googleapiclient.http.MediaIoBaseUpload(
                            s, mimetype='message/rfc822' )
    else:
        params['body']['raw'] = (
            base64.urlsafe_b64encode( message_string ) )

    g.call( g.auth.users().messages().insert, params )

    try:
        del params[ 'media_body' ]
    except KeyError:
        pass