I'm having difficulty setting the References: field in the header of an outgoing SMTP email for Gmail. I'm using Python 3.8 with smtplib and email.message libraries. The code is:
reference_ids = [
'<BN8PR17MB27372595A957D7912CEE184FBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>',
'<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>',
'<BN8PR17MB27371C71A65834531DF028BBBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>',
'<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>']
in_reply_to = reference_ids[0]
smtp = smtplib.SMTP_SSL(es.smtp_server)
smtp.login(es.username, es.password)
msg = email.message.EmailMessage()
if (reference_ids is not None):
msg.add_header('In-Reply-To', in_reply_to)
msg.add_header('References', (' ').join(reference_ids))
msg['Subject'] = request.vars.subject
msg['From'] = es.email
msg['To'] = request.vars.to
msg['CC'] = request.vars.cc
msg['BCC'] = request.vars.bcc
msg.set_content(request.vars.message)
smtp.send_message(msg)
smtp.quit()
where reference_ids is a list of Message-IDs previous "in_reply_to"s of "message_id"s back to the originating email.
I can send the email without errors and when I view the "Show original" the References look ok. it is the proper list of Message-IDs of the form "BNX_blah_blah#mail.gmail.com", without quotes, separated by a space.
However, when I try to later read the sent email with the imaplib library and email.message_from_bytes(raw_email_response_body[1]) I get a real mess of characters. Most of the addresses in the References loose their BNX_blah_blah#mail.gmail.com" form. Message-ID and In-Reply-To looks ok though.
References: =?utf-8?q?=22=3CBN8PR17MB27372595A957D7912CEE184FBF6F9=40BN8PR17?=
=?utf-8?q?MB2737=2Enamprd17=2Eprod=2Eoutlook=2Ecom=3E?=
<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>
=?utf-8?q?=3CBN8PR17MB27371C71A65834531DF028BBBF6F9=40BN8PR17MB2737=2Enampr?=
=?utf-8?q?d17=2Eprod=2Eoutlook=2Ecom=3E_=3CBN8PR17MB27377F609B669D0E72638D6?=
=?utf-8?q?9BF6F9=40BN8PR17MB2737=2Enamprd17=2Eprod=2Eoutlook=2Ecom=3E?=
<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>
Am I encoding the References properly? Am I decoding the References I read from IMAP properly?
thanx to everyone for giving me direction, especially tripleee with "That's bog-standard RFC2047 encoding." which led me the email.header library.
so,,, when i send the email via gmail smtp, i could set the 6th line to simply:
msg.add_header('References', ' '.join(reference_ids))
where reference_ids is a python list of straight address strings, like:
['<BN8PR17MB27372595A957D7912CEE184FBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>', '<BN8PR17MB27371C71A65834531DF028BBBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<BN8PR17MB27377F609B669D0E72638D69BF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>']
where gmail would show the list under "Show original" as a long string line with space delimited separation.
the problem comes in when you attempt to read it via imaplib where it is encoded via RFC2047. so upon reading the email header, i processed it like:
import imaplib, smtplib, email
from email.header import decode_header, make_header
...
emsg = email.message_from_bytes(raw_email_response_body[1])
...
References = emsg['References']
if References:
References = make_header(decode_header(References))
References = str(References).strip().replace('"', '').replace('\r', '').replace('\t', '').replace('\n', ' ').replace(' '*3, ' '*2).replace(' '*2, ' ').split(' ')
which will faithfully return the list of Message-IDs as originally sent. hopefully this helps a few people out there. and, i thank everyone for the help and direction. lucas
You seem to have uncovered a bug in Python's email library. The References: and In-Reply-To: headers should not be subject to RFC2047 encoding at all.
As a quick and dirty demonstration, I can avoid the problem by shortening the long ones.
As a similarly quick and dirty workaround, you can override the email.policy object with a different one which doesn't force these lines to be shortened. This is a slightly obscure corner of the Python email library; the documentation really presupposes a fair amount of prior knowledge of both email in general and Python's email library in particular.
from email.policy import default
...
custom_policy = default.clone(max_line_length=100)
msg = email.message.EmailMessage(policy=custom_policy)
...
# in your IMAP reader
copy = message_from_bytes(imap_response[1], policy=custom_policy)
Notice, however, that this sets the maximum line length everywhere, not just in the headers. You'd really want to be able to override this setting just for the duration of the addition of these specific headers.
Notice also that message_from_bytes needs a policy= keyword argument to construct a modern EmailMessage object. Without the keyword, you end up creating a legacy email.message.Message object which lacks several of the modern methods of the 3.3+ email API.
Here's a demo: https://ideone.com/eEzIxe
Related
This question already has answers here:
How to send SMTP email for office365 with python using tls/ssl
(5 answers)
Closed 1 year ago.
So I am sending an automated text to my number at mynumber#tmomail.net and I am trying to send just a body as apposed to an email and it doesn't work but also doesn't throw an error. My code is below and it works when I add everything but doesn't work when only adding just the body (got from here).
import smtplib
email = "myemail#outlook.com"
pwd = "my_password"
phone_num = "my_number#tmomail.net"
server = smtplib.SMTP('smtp.office365.com',587)
server.starttls()
server.login(email, pwd)
body = "This working?"
server.sendmail(email, phone_num, body)
server.quit
This doesn't seem to work but if I add more to it and give it multipart containing from, to and subject it works just fine.
All you need to do is add a newline:
body = "\nThis working?"
The server.sendmail is a pretty low-level function that requires you to do the work to format the message correctly (and as expected by your SMTP server). At first, I tried modifying your code as follows and received the message successfully:
body = (f"From: {email}\r\nTo: {phone_num}\r\n\r\n")
body += "This working?"
server.sendmail(email, phone_num, msg=body)
server.quit()
The code above provides the headers as described here: https://docs.python.org/3/library/smtplib.html#smtp-example (I just converted their example to f-strings)
I think you are looking to use a bit higher level function where you don't have to worry about doing the headers yourself.
You may want to look into using smpt.send_message()
https://docs.python.org/3/library/smtplib.html#smtplib.SMTP.send_message which helps you construct the correct headers.
I see that you were looking to avoid adding the "from" and "to" as per the example you were looking at. It will work with just that new line as detailed in this related question: How to send SMTP email for office365 with python using tls/ssl
I'm using a standard smtplib.sendmail() call in my Python 3 program to send emails, as follows:
smtp_session.sendmail('The Sender <sender#domain.com>', ['The ÅÄÖ Recipient <recipient#domain.com>'], 'Simple test body here')
The SMTP session has already been successfully established prior to this code line being executed, and it also always works just fine as long as there are no "international characters" in the recipient name.
BUT, as soon as I include e.g. "ÅÄÖ" in the recipient name (which is even just 8-bit ASCII characters, not even "real unicode" or whatever), as can be seen above, the email just disappears and never reaches the sender, although no errors or exceptions are returned or raised by the sendmail() method, nor anything inside it (I have single-stepped it in a debugger while doing this).
I know for a fact that I can send emails with such characters in the recipient names through this exact same SMTP server of mine, using a normal email client program like Thunderbird, so I can only assume that this problem has something to do with some encoding or similar?
Also, the solution shouldn't be related to that mail_options=['SMTPUTF8'] thingy either, because the server just replies that it doesn't support this if I try to use it (and again, emails using these exact recipient names can still be sent through the exact same SMTP server with a normal email client like Thunderbird).
So, is there some simple solution based on using some kind of "MIME related" encoding or similar on the recipient strings that will solve this, or how can I otherwise send an email from Python with such a recipient name?
Characters in SMTP headers are required to be printable ASCII, in the numeric range 33-126 inclusive. If you need to represent characters outside that range in an SMTP header then you must use the encoding method defined by RFC 2231 (which is an evolution of an earlier method defined by RFC 2047).
Historically in Python you would have used the Header class from the email.header module to build suitably-encoded headers. That's still available in Python 3, but in Python 3 the newer recommendation is to use the EmailMessage class from the email.message module to construct the entire message, and let it take care of encoding any headers that need special treatment.
The argument to smtplib.sendmail() should not have human-readable labels, just the address terminus.
smtp_session.sendmail('sender#domain.com', ['recipient#domain.com'],
'Simple test body here')
The email.headerregistry module in Python 3.6+ has a facility for extracting just the email terminus, by way of parsing structured headers into objects with attributes.
from email.headerregistry import AddressHeader
hdr = dict()
AddressHeader.parse('To: The ÅÄÖ Recipient <recipient#domain.com>', hdr)
for grp in hdr['groups']:
for addr in grp.addresses:
print('{0}#{1}'.format(addr.username, addr.domain))
(I really hope there is a less convoluted way to access this functionality but at the very least this produces the expected result.)
In the actual message, Python takes care of properly RFC2047-encoding any headers with Unicode content (if you use the correct methods from the email library to construct a prop0er MIME message); but this is pure presentation (RFC5322) not transport (RFC5321). So in the message itself you might see
From: The Sender <sender#domain.com>
To: The =?utf-8?Q?=C3=85=C3=84=C3=96_Recipient?= <recipient#domain.com>
though keep in mind that there is no requirement for the message content to actually reveal the transport sender or recipient headers. (Maybe tangentially see Header "To:" for a Bulk Email Sender)
It seems like there should be a better way to do this. I have a plain text email, and I'd like to convert it to a multipart email. I can do it like this:
import commonmark
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.parser import BytesParser
raw_msg = b'''
To: you#example.com
From: me#example.com
Subject: This is a sample
# This should be h1
Something
## This should be h2
Email from markdown? Sweet
'''.strip()
msg = BytesParser().parsebytes(raw_msg)
html = commonmark.commonmark(msg.get_payload())
new_msg = MIMEMultipart()
for key in msg.keys():
new_msg[key] = msg.get(key)
new_msg.attach(MIMEText(msg.get_payload()))
new_msg.attach(MIMEText(html, 'html'))
print(str(new_msg))
Which appears to work, but it also seems clunky to me. Is there a way in Python to just create a multipart email from my original message? Or am I doing it the right way?
If your input is a single-part email message then yeah, parsing it and extracting the payload is probably the way to go.
Having the parser "convert" the message to multipart on the fly doesn't seem feasible, or particularly well-defined. Be happy that the input is simple.
The outer container for your new message should probably be explicitly multipart/alternative; the default type multipart/mixed does not convey the semantics of variant encodings of the same message which you seem to be striving for.
Notice that the email library was officially overhauled in Python 3.5. You might want to explore transitioning your code to use the newer EmailMessage class hierarchy instead, though I don't believe it will bring any immediate benefits for this particular use case. Overall, the new code base is more versatile and disciplined than the old somewhat ad-hoc legacy email library (now known as compat32, as in Python 3.2; the overhaul was unofficially introduced already in Python 3.3).
I'm able to send emails using smtplib with ease. What I'm struggling with is reading the actual headers that were sent one. Specifically, I'm looking to read the Message-ID and References.
I thought at first that sendmail() would return them, but it does not.
Found that I'm able to redirect smtpilb.stderr to my own function and parse out the data that I need. Is there a better way that would allow me to do say:
headers['References']
If you use sendmail() I am not sure how to access the headers, because you don't have a Message object in that case. However, if you use send_message instead - which is very similar to sendmail() - and pass it an email.message.Message object, then all of the email message headers and their values are stored in a dict in your Message object. So e.g., Message-ID can be accessed from an email message object msg with msg['Message-ID'], subject can be accessed using msg['Subject'], etc. I don't think anything will be stored in message-id unless you put it there yourself though. You can 'roll your own' Message-ID using make_msgid() from email.utils:
from email.utils import make_msgid
msg['Message-ID'] = make_msgid()
I'm trying to teaching myself how to program by building programs/scrips that will be useful to me. I'm trying to retool a script I found online to send an email through gmail using a python script (Source).
This example has a portion of code to attach files, which I don't want/need. I have tweaked the code so that I don't have to attach any files, but when I do this I lose the body of the email. Any help/tips on how to modify the code to keep the body of the email intact?
Appreciate the help.
The sample code you're using creates a multi-part MIME message. Everything is an attachment, including the message body. If you just want to send a plain old single-part plain text or HTML message, you don't need any of the MIME stuff. It just adds complexity. See that bit in your sample's sendmail() call where it says msg.as_string()? Well, that just converts the MIME objects you've created to text. It's easy enough to specify the text yourself, if you are dealing with text to start with.
The function below is similar to code I used for mailing a log file in a script I wrote. It takes a plain text body and converts it to preformatted HTML (to work better in Outlook). If you want to keep it plain text, just take out the line that adds the HTML tags, and change the Content-Type header to "text/plain."
import smtplib
def sendmail(sender, recipient, subject, body, server="localhost"):
"Sends an e-mail to the specified recipient."
body = ("<html><head></head><body><pre>%s</pre></body></html>" %
body.replace("&", "&").replace("<", "<"))
headers = ["From: " + sender,
"Subject: " + subject,
"To: " + recipient,
"MIME-Version: 1.0",
"Content-Type: text/html"]
headers = "\r\n".join(headers)
session = smtplib.SMTP(server)
session.sendmail(sender, recipient, headers + "\r\n\r\n" + body)
session.quit()
I see you have a clue about what you did wrong.
In this case, the method attach() refers to adding something to the email. This is confusing because when we thing about attaching things and email, we think about adding extra files, not the body.
Based on some other examples, it seems that the attach method is used to add either text to the body or a file to the email.
So, to answer your question, the attach method does what you think it does and more--it also adds text to the body.
Welcome to SO, by the way. Smart choice picking Python for learning how to script.