How can I properly send batch/bulk/mass emails using MailGun in Django using SMTP protocol?
What I've tried so far?
I am using django.core.mail.backends.smtp.EmailBackend as my EMAIL_BACKEND
and this is the code snippet that I have tried to send the emails.
from django.core.mail import EmailMultiAlternatives
import json
to_emails = [
"mail_1#example.com",
"mail_2#example.com",
"mail_3#example.com",
"mail_4#example.com",
"jerinpetergeorge#gmail.com",
]
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
to=to_emails,
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
response = mail.send()
print(response)
and I've got the mail as below,
As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
So, how can I tell the Mailgun/Django to parse my variables properly in order to make the emails looks more personal?
Notes
I prefer to use SMTP protocol
I've tried the REST APIs of Mailgun and it was a success (but, I prefer SMTP)
I found django-anymail and seems it has the feature. But, It also uses the APIs (correct me if I am wrong)
Update-1
Updated the to argument to to="%recipient%" But, got
TypeError: "to" argument must be a list or tuple
Updated the to argument to to=["%recipient%"] But, got
smtplib.SMTPRecipientsRefused: {'=?utf-8?q?=25recipient=25?=': (501, b'Invalid command or cannot parse to address')}
As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
It is not properly supported with SMTP by Mailgun.
However, relying on the (unintuitive) implementation of BCC in Mailgun, there is a workaround:
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
# to=to_emails, # Replace this
bcc=to_emails, # with this
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["To"] = "%recipient%" # Add this
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
Reference: https://stackoverflow.com/questions/37948729/mailgun-smtp-batch-sending-with-recipient-variables-shows-all-recipients-in-to-field
Why does to=["%recipient%"] not work with SMTP?
It's the standard in the protocol.
From https://documentation.mailgun.com/_/downloads/en/latest/pdf/:
SMTP send will error with “cannot parse to address” or “cannot parse from address” if the provided email address fails syntax checks in accordance with RFC5321, RFC5322, RFC6854.
What to do for proper support of Batch Sending with Mailgun?
Use the API.
From https://stackoverflow.com/questions/30787399/laravel-5-sending-group-emails (multiposted to https://laracasts.com/discuss/channels/laravel/sending-email-to-1000s-of-reciepents):
So far, I have created an array of recipient email addresses, sent the email to a webmaster type address, and included the end recipients in BCC
While this works, it's not ideal.
Rather than using Laravel's built in Mail, I elected to use Mailgun's API (specifically batch sending) directly
This also allows me to access unique recipient variables within my email template
(It's not specific to Laravel/PHP, but to SMTP via Mailgun.)
What do you mean by "unintuitive" implementation of BCC in Mailgun?
Mailgun effectively personalises the email for each BCC recipient using recipient-variables.
From https://github.com/mailgun/mailgun-js-boland/issues/89:
the bcc person is receiving the email as it was addressed to them instead of being part of the bcc
This causes a separate issue when you actually want BCC recipients to get the same content.
From https://stackoverflow.com/questions/48887866/bcc-in-mailgun-batch-send-does-not-include-substitutions:
In the copy sent to the bcc address, the recip_vars substitution has not been made.
According to the good people at Mailgun, this is not possible, at least in the current release of the service.
Related
I've tried with no conclusions to resend emails with Python.
Once I've logged in SMTP and IMAP with TLS, this is what I have written:
status, data = self._imapserver.fetch(id, "(RFC822)")
email_data = data[0][1]
# create a Message instance from the email data
message = email.message_from_string(email_data)
# replace headers (could do other processing here)
message.replace_header("From", 'blablabla#bliblibli.com')
message.replace_header("To", 'blobloblo#blublublu.com')
self._smtpserver.sendmail('blablabla#bliblibli.com', 'blobloblo#blublublu.com', message.as_string())
But the problem is that the variable data doesn't catch the information from the email, even if the ID is the one I need.
It tells me:
b'The specified message set is invalid.'
How can I transfer an email with Python?
Like the error message says, whatever you have in id is invalid. We don't know what you put there, so all we can tell you is what's already in the error message.
(Also, probably don't use id as a variable name, as you will shadow the built-in function with the same name.)
There are additional bugs further on in your code; you need to use message_from_bytes if you want to parse it, though there is really no need to replace the headers just to resend it.
status, data = self._imapserver.fetch(correct_id, "(RFC822)")
self._smtpserver.sendmail('blablabla#bliblibli.com', 'blobloblo#blublublu.com', data[0][1])
If you want to parse the message, you should perhaps add a policy argument; this selects the modern EmailMessage API which was introduced in Python 3.6.
from email.policy import default
...
message = email.message_from_bytes(data[0][1], policy=default)
message["From"] = "blablabla#bliblibli.com"
message["To"] = "blobloblo#blublublu.com"
self._smtpserver.send_message(message)
The send_message method is an addition to the new API. If the message could contain other recipient headers like Cc:, Bcc: etc, perhaps using the good old sendmail method would be better, as it ignores the message's headers entirely.
I am working on a project that requires a JSON file response through Gmail API which contains a list of all the emails (Sender's Info, Subject, and Body) between given dates. Then that JSON file will be processed as needed.
I'm not sure how to generate a request that could provide me with the required JSON file using Gmail API through Python.
I'm a beginner, Please help me.
Assuming that you have gone through OAuth and built the service (if not, check the quickstart), you have to do the following:
Call users.messages.list, using the parameter q to filter your messages by date. This q uses the same syntax as in Gmail UI: see Search operators you can use with Gmail. For example, if you wanted to retrieve your messages from the first 9 months in 2020, you would do this:
user_id = "me"
searchFilter = "after:2020/01/01 before:2020/10/01"
messages = service.users().messages().list(userId=user_id, q=searchFilter).execute()
When listing messages, only the id and the threadId are populated. In order to get the full message resource for all these messages, you should loop through them and call users.messages.get for each message, using its id:
for message in messages["messages"]:
messageId = message["id"]
message = service.users().messages().get(userId=user_id, id=messageId).execute()
For each of these retrieved messages, you want to get the subject, body and sender's email. Subject and sender's email can be found on the message headers. You should loop through the headers and look for one named Subject and another one named From. See these answers for more info: Python: how to get the subject of an email from gmail API, Get sender email from gmail-api. Regarding the body, it is to be found at ["payload"]["body"]. See this answer: How to retrieve the whole message body using Gmail API (python).
Reference:
Searching for Messages
REST Resource: users.messages
list(userId=*, labelIds=None, q=None, pageToken=None, maxResults=None, includeSpamTrash=None)
get(userId=, id=, format=None, metadataHeaders=None)
Looking to create and send messages with multiple files attached. Per the online gmail api documentation, there is a function for building messages with an attachment but no documentation for howto use it to create a message with multiple attachments.
Can I use the gmail API to send messages with multiple attachments programmatically? How might one do this?
With this function, you can send to one or multiple recipient emails, and also you can attach zero, one or more files. Coding improvement recommendations are welcome, however the way it is now, it works.
Python v3.7
smtplib from https://github.com/python/cpython/blob/3.7/Lib/smtplib.py (download the code and create the smtplib.py on your project folder)
def send_email(se_from, se_pwd, se_to, se_subject, se_plain_text='', se_html_text='', se_attachments=[]):
""" Send an email with the specifications in parameters
The following youtube channel helped me a lot to build this function:
https://www.youtube.com/watch?v=JRCJ6RtE3xU
How to Send Emails Using Python - Plain Text, Adding Attachments, HTML Emails, and More
Corey Schafer youtube channel
Input:
se_from : email address that will send the email
se_pwd : password for authentication (uses SMTP.SSL for authentication)
se_to : destination email. For various emails, use ['email1#example.com', 'email2#example.com']
se_subject : email subject line
se_plain_text : body text in plain format, in case html is not supported
se_html_text : body text in html format
se_attachments : list of attachments. For various attachments, use ['path1\file1.ext1', 'path2\file2.ext2', 'path3\file3.ext3']. Follow your OS guidelines for directory paths. Empty list ([]) if no attachments
Returns
-------
se_error_code : returns True if email was successful (still need to incorporate exception handling routines)
"""
import smtplib
from email.message import EmailMessage
# Join email parts following smtp structure
msg = EmailMessage()
msg['From'] = se_from
msg['To'] = se_to
msg['Subject'] = se_subject
msg.set_content(se_plain_text)
# Adds the html text only if there is one
if se_html_text != '':
msg.add_alternative("""{}""".format(se_html_text), subtype='html')
# Checks if there are files to be sent in the email
if len(se_attachments) > 0:
# Goes through every file in files list
for file in se_attachments:
with open(file, 'rb') as f:
file_data = f.read()
file_name = f.name
# Attaches the file to the message. Leaves google to detect the application to open it
msg.add_attachment(file_data, maintype='application', subtype='octet-stream', filename=file_name)
# Sends the email that has been built
with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp:
smtp.login(se_from, se_pwd)
smtp.send_message(msg)
return True
Don't forget to activate less secure apps on your google account (https://myaccount.google.com/lesssecureapps) for this code to work.
Hope this helps
In Django EmailMultiAlternatives documentation there is nothing about how to add headers like "format" or "Reply-To" in EmailMultiAlternatives. It took a while for me to figure it out and I am sending this post to help others with saving their time.
As you can see in django's source code, EmailMultiAlternatives inherits from EmailMessage, so they take the same parameters in the init constructor. This way, we can add headers like:
msg = EmailMultiAlternatives(
subject, message, from_email, to_list,
headers={'Reply-To': "email#example.com", 'format': 'flowed'}
)
Back in 2015 OP complained, that there were no instructions in documentation, how to add headers such as "Format" and "Reply-To" in Django mail (django.core.mail) module. However today, while using same documentation link. We can find description and examples in 2018 easily:
class EmailMessage[source]
The EmailMessage class is initialized with the following parameters
(in the given order, if positional arguments are used). All parameters
are optional and can be set at any time prior to calling the send()
method.
subject: The subject line of the email.
body: The body text. This should be a plain text message.
from_email: The sender’s address. Both fred#example.com and Fred <fred#example.com> forms are legal. If omitted, the DEFAULT_FROM_EMAIL
setting is used.
to: A list or tuple of recipient addresses.
bcc: A list or tuple of addresses used in the “Bcc” header when sending the email.
connection: An email backend instance. Use this parameter if you want to use the same connection for multiple messages. If omitted, a
new connection is created when send() is called.
attachments: A list of attachments to put on the message. These can be either email.MIMEBase.MIMEBase instances, or (filename, content, mimetype) triples.
headers: A dictionary of extra headers to put on the message. The keys are the header name, values are the header values. It’s up to the
caller to ensure header names and values are in the correct format for
an email message. The corresponding attribute is extra_headers.
cc: A list or tuple of recipient addresses used in the “Cc” header when sending the email.
For example:
email = EmailMessage('Hello', 'Body goes here', 'from#example.com',
['to1#example.com', 'to2#example.com'], ['bcc#example.com'],
headers = {'Reply-To': 'another#example.com', 'format': 'flowed'})
As we see from examples, EmailMessage has headers argument (dictionary) too, EmailMultiAlternatives according to docstring in source code is:
A version of EmailMessage that makes it easy to send multipart/alternative
messages. For example, including text and HTML versions of the text is
made easier.
So if you don't need something specific, EmailMessage is fine, because currently EmailMultiAlternatives is for easy inclusion of text and HTML versions of text.
I have this python IMAP script, but my problem is that, every time I want to get the sender's email address, (From), I always get the sender's first name followed by their email address:
Example:
Souleiman Benhida <souleb#gmail.com>
How can i just extract the email address (souleb#gmail.com)
I did this before, in PHP:
$headerinfo = imap_headerinfo($connection, $count)
or die("Couldn't get header for message " . $count . " : " . imap_last_error());
$from = $headerinfo->fromaddress;
But, in python I can only get the full name w/address, how can I get the address alone? I currently use this:
typ, data = M.fetch(num, '(RFC822)')
mail = email.message_from_string(data[0][1])
headers = HeaderParser().parsestr(data[0][1])
message = parse_message(mail) #body
org = headers['From']
Thanks!
Just one more step, using email.utils:
email.utils.parseaddr(address)
Parse address – which should be the value of some address-containing field such as To or Cc – into its constituent realname and email address parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of ('', '') is returned.
Note: originally referenced rfc822, which is now deprecated.
to = email.utils.parseaddr(msg['cc'])
This works for me.
My external lib https://github.com/ikvk/imap_tools
let you work with mail instead read IMAP specifications.
from imap_tools import MailBox, A
# get all emails from INBOX folder
with MailBox('imap.mail.com').login('test#mail.com', 'pwd', 'INBOX') as mailbox:
for msg in mailbox.fetch(A(all=True)):
print(msg.date, msg.from_, msg.to, len(msg.text or msg.html))
msg.from_, msg.to - parsed addresses, like: 'Sender#ya.ru'
I didn't like the existing solutions so I decided to make a sister library for my email sender called Red Box.
Here is how to search and process emails including getting the from address:
from redbox import EmailBox
# Create email box instance
box = EmailBox(
host="imap.example.com",
port=993,
username="me#example.com",
password="<PASSWORD>"
)
# Select an email folder
inbox = box["INBOX"]
# Search and process messages
for msg in inbox.search(unseen=True):
# Process the message
print(msg.from_)
print(msg.to)
print(msg.subject)
print(msg.text_body)
print(msg.html_body)
# Flag the email as read/seen
msg.read()
I also wrote extensive documentation for it. It also has query language that fully supports nested logical operations.