We Keep Coding

How to replace/overwrite default header of EmailMultiAlternatives

Environment: Ubuntu 18.10, Python 2.7.15, Django 1.11.16
I'm trying to send an email containing an inline image. I have the following code:
msg = EmailMultiAlternatives(some_subject, some_body, 'from#some-domain.com', ['to#some#domain'])
img_data = open('path/to/image.png', 'rb').read()
img = MIMEImage(img_data)
msg.attach(img)
msg.send()
(I've only included the code that I think is relevant but I can add more on demand.)
The above properly works and the image is properly displayed on most of the email clients (about 7 of them, both mobile, desktop or webmail ones) that I tested on, with two exceptions: Mozilla Thunderbird 60 and some macOS native email client.
On Thunderbird the image is not displayed inline but at the very end of the message. On the macOS client, the image is displayed inline but additionally it is also displayed at the very end of the message.
I composed and sent a test message from another email client, containing an inline image which was properly displayed on both Thunderbird and macOS. I compared the headers of this message with the headers of the message generated by my code.
I noticed that the faulty message has the 'Content-Type' set to 'multipart/mixed' while the properly displayed message had the same header set to 'multipart/related'.
I saved the faulty message in an eml file and manually changed the value of that header and then loaded the message in Thunderbird. The message was properly displayed and the image was in the right place.
If I could set that header to the proper value, the problem would be solved.
So, my question is: is there any possibility to tell EmailMultiAlternatives to set 'Content-Type' : 'multipart/related' instead of the default value of 'multipart/mixed'?
I tried to add the header like this but it is not working:
msg = EmailMultiAlternatives(some_subject, some_body, 'from#some-domain.com', ['to#some#domain'], headers={'Content-Type' : 'multipart/related'})
I got the following error ( I use Amazon SES):
400 Bad Request
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>InvalidParameterValue</Code>
<Message>Duplicate header 'Content-Type'.</Message>
</Error>
<RequestId>xxxxxxxxxx</RequestId>
</ErrorResponse>
If I can't modify that header, do you suggest any alternatives?

If you look at the source code, you'll see that EmailMultiAlternatives is a subclass of EmailMessage, which itself has a class attribute:
mixed_subtype = 'mixed'
So if you create your own subclass to override this, you should get what you need:
class EmailMultiAlternativesRelated(EmailMultiAlternatives):
mixed_subtype = 'related'
That's it, now you just use this new class, and it will use "multipart/related".
(the _create_attachments() method passes this subtype to python's SafeMIMEMultipart which creates the actual headers for each attachment.)

NoneType in a xml parsing but there is a message

I get the next xml response in a webservice:
<error>failed to communicate with device</error>
And I want to save the error message, so I do the next (I do it in others situations and work but not in this case):
[...]
if tree.tag == 'error':
mensaje = tree.find('error')
print('The error is: '+mensaje)
And I get None. If I put mensaje = tree.find('error').text I get a error because NoneType has no attribute text.
Why can't I get the error message?
Thanks

Importing e-mails from one gmail mailbox to another

I have 2 Gmail accounts, and I use one ("gmail1") to forward all e-mail to the other one ("gmail2"). This works fine, but I discovered recently (after years!) that in fact, Gmail does not forward all my e-mail, but only the e-mail it considers not to be spam. Therefore, checking my spam folder in gmail2 for missing e-mails, I have often blamed senders, when really the e-mail had gotten lost on gmail1. I want to solve this programatically, by regularly checking for spam e-mails in gmail1 and importing them into gmail2 (then I'm fine with gmail2 categorizing as spam or not, as long as the mail makes its way there).
I'm only a very amateur programmer, and so far thanks to some samples from the gmail API docs, I've managed to log in to my accounts, and retrieve some messages corresponding to a query, including for example recent spam e-mails. However, I'm struggling with the "import" concept and how to use it. There are no Python examples for this, and I couldn't solve the problems I'm facing.
Where I am right now:
- if I "get" a message from gmail1 and attempt an import_ call on gmail2, using the message I just retrieved, I get an error because threadId is not allowed
- if I do the same and then "del message['threadId']" then the error becomes error 400 : 'raw' RFC822 payload message string or uploading message via /upload/* URL required. I've seen that there are some situations where upload is required, but I am completely lost as to what I should do to make this work.
Here's what I have so far (sorry for the very hacky style):
# skipping imports
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly','https://www.googleapis.com/auth/gmail.modify']
def getMessage(service, user_id, msg_id):
"""from gmail API examples"""
try:
message = service.users().messages().get(userId=user_id, id=msg_id).execute()
return message
except errors.HttpError as error:
print ('An error occurred:' , error)
def listMessagesMatchingQuery(service, user_id, query=''):
"""from gmail API examples"""
# skipping some code
return messages
def login(accountId):
"""from gmail API examples, adapted to handle 2 accounts"""
# skipping some code
return build('gmail', 'v1', credentials=creds)
def importMessage(service, user_id, msg):
"""my daring attempt at using import, without any Python sample to use as a basis"""
try:
message = service.users().messages().import_(userId=user_id, body=msg).execute()
return message
except errors.HttpError as error:
print ('An error occurred:' , error)
if __name__ == '__main__':
service_gmail = login('gmail2')
service_dnt = login('gmail1')
messages = listMessagesMatchingQuery(service_dnt,"me","in:spam is:unread after:" + str(int((datetime.now() - timedelta(hours=12)).timestamp())))
# this gets me some recent unread spam messages
m=getMessage(service_dnt,"me",messages[0]['id'])
# now I have a full message - I'm just investigating for now so the first message is enough
del m['threadId']
# if I don't do that, the error I get is that threadId is not allowed here, so I remove it
imported = importMessage(service_gmail,"me",m)
# this now gives me error 400 : 'raw' RFC822 payload message string or uploading message via /upload/* URL required
I'd like to find the way to make this work, so that the e-mail appears in gmail2 as if it had been received by gmail2 directly (though I would like to keep the To: address, as I use a catch-all on gmail1 and want to know which e-mail address the e-mail was directed to). But right now, I get only errors about having to use upload; I'm not sure if that's what I really should be doing, and if it is , I have no idea how.
Thanks a lot in advance for any help!

In the end I managed. I had missed the fact that there are 2 ways to "get" messages, a simple one and a "raw" one. With the "raw" way to access the message, I can use the import_ function easily:
message = service_dnt.users().messages().get(userId="me", id=messageId,format='raw').execute()
imported = importMessage(service_gmail,"me",{'raw': message['raw']})
whereby the importMessage function is unchanged vs. the original code I posted.

why doesn't this imaplib request work all the time?

I'm using imaplib and attempting to parse messages from a gmail acct. My code has worked for months, and now all of the sudden it's failing miserably. I have no idea what to attribute this to.
The following works around 1/3 of the time. By 'works', I mean succeeds in printing something other than 'no new messages' when I receive an email. Would anyone have any suggestions of more robust ways to attempt this? Or maybe suggestions of now to configure gmail accounts to get this to work more reliably?
I'm also generally interested if the way I've coded this seems like good practice.
Thanks for any and all help...
def check_email(interval):
while True:
server.select('INBOX')
status, ids = server.search(None, 'UnSeen')
if not ids or ids[0] is '':
print 'no new messages'
else:
print 'found a message; attempting to parse...'
latest_id = ids[0]
status, msg_data = server.fetch(latest_id, '(UID BODY[TEXT])')
raw_data = msg_data[0][1]
char_array = list(raw_data)
print 'message result: ', char_array
time.sleep(interval)
EDIT1: I am now getting the following error:
"imaplib.error: FETCH command error: BAD ['Could not parse command']"
Does anyone know what I can attribute this to? It's apparently a result of the line
status, msg_data = server.fetch(latest_id, '(UID BODY[TEXT])')
EDIT2: I found out that I can log into the gmail account, click on the 'more' tab, then click on 'mark all as read' and all of the sudden the code works as expected. Is there a way to mark all messages as read remotely with imaplib??

How to receive mail using python

I would like to receive email using python. So far I have been able to get the subject but not the body. Here is the code I have been using:
import poplib
from email import parser
pop_conn = poplib.POP3_SSL('pop.gmail.com')
pop_conn.user('myusername')
pop_conn.pass_('mypassword')
#Get messages from server:
messages = [pop_conn.retr(i) for i in range(1, len(pop_conn.list()[1]) + 1)]
# Concat message pieces:
messages = ["\n".join(mssg[1]) for mssg in messages]
#Parse message intom an email object:
messages = [parser.Parser().parsestr(mssg) for mssg in messages]
for message in messages:
print message['subject']
print message['body']
pop_conn.quit()
My issue is that when I run this code it properly returns the Subject but not the body. So if I send an email with the subject "Tester" and the body "This is a test message" it looks like this in IDLE.
>>>>Tester >>>>None
So it appears to be accurately assessing the subject but not the body, I think it is in the parsing method right? The issue is that I don't know enough about these libraries to figure out how to change it so that it returns both a subject and a body.

The object message does not have a body, you will need to parse the multiple parts, like this:
for part in message.walk():
if part.get_content_type():
body = part.get_payload(decode=True)
The walk() function iterates depth-first through the parts of the email, and you are looking for the parts that have a content-type. The content types can be either text/plain or text/html, and sometimes one e-mail can contain both (if the message content_type is set to multipart/alternative).

The email parser returns an email.message.Message object, which does not contain a body key, as you'll see if you run
print message.keys()
What you want is the get_payload() method:
for message in messages:
print message['subject']
print message.get_payload()
pop_conn.quit()
But this gets complicated when it comes to multi-part messages; get_payload() returns a list of parts, each of which is a Message object. You can get a particular part of the multipart message by using get_payload(i), which returns the ith part, raises an IndexError if i is out of range, or raises a TypeError if the message is not multipart.
As Gustavo Costa De Oliveir points out, you can use the walk() method to get the parts in order -- it does a depth-first traversal of the parts and subparts of the message.
There's more about the email.parser module at http://docs.python.org/library/email.message.html#email.message.Message.

it also good return data in correct encoding in message contains some multilingual content
charset = part.get_content_charset()
content = part.get_payload(decode=True)
content = content.decode(charset).encode('utf-8')

Here is how I solved the problem using python 3 new capabilities:
import imaplib
import email
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login(username, password)
mail.select(readonly=True) # refresh inbox
status, message_ids = mail.search(None, 'ALL') # get all emails
for message_id in message_ids[0].split(): # returns all message ids
# for every id get the actual email
status, message_data = mail.fetch(message_id, '(RFC822)')
actual_message = email.message_from_bytes(message_data[0][1])
# extract the needed fields
email_date = actual_message["Date"]
subject = actual_message["Subject"]
message_body = get_message_body(actual_message)
Now get_message_body is actually pretty tricky due to MIME format. I used the function suggested in this answer.
This particular example works with Gmail, but IMAP is a standard protocol, so it should work for other email providers as well, possibly with minor changes.

if u want to use IMAP4. Use outlook python library, download here : https://github.com/awangga/outlook
to retrieve unread email from your inbox :
import outlook
mail = outlook.Outlook()
mail.login('emailaccount#live.com','yourpassword')
mail.inbox()
print mail.unread()
to retrive email element :
print mail.mailbody()
print mail.mailsubject()
print mail.mailfrom()
print mail.mailto()