I have tried manipulating headers, tried adding/replacing encoding to 'gzip' but adding new headers for to accept 'gzip' format or replacing current headers only increases the overall size of the email rather than reducing it.
My code:
def create_message(sender, to, subject, message_text):
"""Create a message for an email.
Args:
sender: Email address of the sender.
to: Email address of the receiver.
subject: The subject of the email message.
message_text: The text of the email message.
Returns:
An object containing a base64url encoded email object.
"""
message = MIMEText(message_text)
message['to'] = to
message['from'] = sender
message['subject'] = subject
# message.add_header('Accept-Encoding', 'gzip')
# message.add_header('User-Agent', 'gzip')
message.add_header('Content-Encoding', 'gzip')
# message.replace_header('Content-Transfer-Encoding', 'gzip')
to_be_sent = {'raw': base64.urlsafe_b64encode(message.as_string().encode()).decode()}
return to_be_sent
def send_message(service, user_id, message):
"""Send an email message.
Args:
service: Authorized Gmail API service instance.
user_id: User's email address. The special value "me"
can be used to indicate the authenticated user.
message: Message to be sent.
Returns:
Sent Message.
"""
try:
message = (service.users().messages().send(userId=user_id, body=message).execute())
# print('Message Id: %s' % message['id'])
return message
except Exception as error:
print('An error occurred: %s' % error)
To read the message:
# Call the Gmail API - read
message = service.users().messages().get(userId="me", id=MESSAGE_ID, format="full").execute()
The Gmail API performance tip states that:
An easy and convenient way to reduce the bandwidth needed for each request is to enable gzip compression.In order to receive a gzip-encoded response you must do two things: Set an Accept-Encoding header, and modify your user agent to contain the string gzip.
Accept-Encoding: gzip
User-Agent: my program (gzip)
Still not working for me. Only keeps the size either constant or increases it.
It won't be possible with the method you are asking for. As #VPfB mentioned, the documentation says response.
Although I thought of alternative and it is to compress the content/body if your sole purpose is to diminish the mail size. You can also decompress the content afterwards if needed be.
I did the test earlier. See the following results below:
Test conditions:
Tested in Google Apps Script using Gmail API (python is unavailable in my device)
Long string content test length: 8802
Short string content test length: 452
Test cases are generated through here
Compression used is LZ string compression
Output (sizeEstimate):
Long uncompressed: 18928
Long compression: 18421
Short uncompressed: 1627
Short compression: 2081
Conclusion:
If you are planning to use this on fairly short content emails, don't bother, it will not save you any storage and will only enlarge the size.
If you are using it on long content emails, it saves a very little storage but it's up to you to decide if that is worth saving that much but having the need to decompress it every time you want to read the email.
Reference includes both compress and decompress functions.
See Python implemenation by eduardtomasek if you want to test it on your end.
Related
I am trying to build a email chatbot but it has this bug where after it sends the first message, and then gets a response it keeps spamming the answered to the response it got until it gets another response which then it repeats again I was thinking to solve this I should use a variable which detects emails and later down the code a condition that responds only if a email is received, does anyone have any idea on how I could fix this? Thanks
def receive_email():
try:
mail = imaplib.IMAP4_SSL("smtp.gmail.com")
mail.login(email_address, email_password)
mail.select('inbox')
#searches inbox
status, data = mail.search(None, 'Recent')
mail_ids = data[0].split()
latest_email_id = mail_ids[-1]
status, data = mail.fetch(latest_email_id, '(RFC822)')
#gets message
for response_part in data:
if isinstance(response_part, tuple):
msg = email.message_from_bytes(response_part[1])
sender = msg['from']
subject = msg['subject']
if msg.is_multipart():
for part in msg.get_payload():
if part.get_content_type() == 'text/plain':
return part.get_payload()
message = msg.get_payload()
return message,
except Exception as e:
print("Error: ", e)
print("Could not receive email")
return None, None
This is the usual problem for an email autoresponder, if I understand you correctly, and RFC 3834 offers good advice.
Since answers should be self-contained I offer a summary:
Add the Auto-Submitted: auto-replied header field on your outgoing messages. Any value other than no will prevent well-written autoresponders from replying to your outgoing messages.
Set the \answered flag on the message you reply to, immediately before you send the reply.
Change the search key from recent to unanswered not header "auto-submitted" "". unanswered means that the search won't match the messages on which you set the \answered flag, not header "auto-submitted" "" means that you'll not match messages that contain any auto-submitted header field.
Direct your replies to the address in return-path or sender, not the one in from. This is a matter of convention. Auto-submitted mail will often have a special return-path that points to an address that never sends any autoreply.
You may also extend the search key with more details from RFC 3834. The one I suggest should work, but not header "precedence" "junk" will for example prevent your code from replying to a bit of autogenerated mail. Sendgrid and its friends also add header fields you may want to look for and exclude.
If the incoming message has headers like this (use the "view headers" function of most mail readers to see it):
From: example#example.com
Subject: Weekend
To: srtai22#gmail.com
Message-id: <56451182ae7a62978cd6f6ff06dd21e0#example.com>
Then your reply should have headers like this:
Return-Path: <>
From: srtai22#gmail.com
To: example#example.com
Auto-Submitted: auto-replied
Subject: Auto: Weekend
References: <56451182ae7a62978cd6f6ff06dd21e0#example.com>
There'll be many more fields in both, of course. Your reply's return-path says that nothing should respond automatically, From and To are as expected, auto-submitted specifies what sort of response this is, subject doesn't matter very much but this one's polite and well-behaved, and finally references links to the original message.
I've tried with no conclusions to resend emails with Python.
Once I've logged in SMTP and IMAP with TLS, this is what I have written:
status, data = self._imapserver.fetch(id, "(RFC822)")
email_data = data[0][1]
# create a Message instance from the email data
message = email.message_from_string(email_data)
# replace headers (could do other processing here)
message.replace_header("From", 'blablabla#bliblibli.com')
message.replace_header("To", 'blobloblo#blublublu.com')
self._smtpserver.sendmail('blablabla#bliblibli.com', 'blobloblo#blublublu.com', message.as_string())
But the problem is that the variable data doesn't catch the information from the email, even if the ID is the one I need.
It tells me:
b'The specified message set is invalid.'
How can I transfer an email with Python?
Like the error message says, whatever you have in id is invalid. We don't know what you put there, so all we can tell you is what's already in the error message.
(Also, probably don't use id as a variable name, as you will shadow the built-in function with the same name.)
There are additional bugs further on in your code; you need to use message_from_bytes if you want to parse it, though there is really no need to replace the headers just to resend it.
status, data = self._imapserver.fetch(correct_id, "(RFC822)")
self._smtpserver.sendmail('blablabla#bliblibli.com', 'blobloblo#blublublu.com', data[0][1])
If you want to parse the message, you should perhaps add a policy argument; this selects the modern EmailMessage API which was introduced in Python 3.6.
from email.policy import default
...
message = email.message_from_bytes(data[0][1], policy=default)
message["From"] = "blablabla#bliblibli.com"
message["To"] = "blobloblo#blublublu.com"
self._smtpserver.send_message(message)
The send_message method is an addition to the new API. If the message could contain other recipient headers like Cc:, Bcc: etc, perhaps using the good old sendmail method would be better, as it ignores the message's headers entirely.
How can I properly send batch/bulk/mass emails using MailGun in Django using SMTP protocol?
What I've tried so far?
I am using django.core.mail.backends.smtp.EmailBackend as my EMAIL_BACKEND
and this is the code snippet that I have tried to send the emails.
from django.core.mail import EmailMultiAlternatives
import json
to_emails = [
"mail_1#example.com",
"mail_2#example.com",
"mail_3#example.com",
"mail_4#example.com",
"jerinpetergeorge#gmail.com",
]
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
to=to_emails,
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
response = mail.send()
print(response)
and I've got the mail as below,
As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
So, how can I tell the Mailgun/Django to parse my variables properly in order to make the emails looks more personal?
Notes
I prefer to use SMTP protocol
I've tried the REST APIs of Mailgun and it was a success (but, I prefer SMTP)
I found django-anymail and seems it has the feature. But, It also uses the APIs (correct me if I am wrong)
Update-1
Updated the to argument to to="%recipient%" But, got
TypeError: "to" argument must be a list or tuple
Updated the to argument to to=["%recipient%"] But, got
smtplib.SMTPRecipientsRefused: {'=?utf-8?q?=25recipient=25?=': (501, b'Invalid command or cannot parse to address')}
As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
It is not properly supported with SMTP by Mailgun.
However, relying on the (unintuitive) implementation of BCC in Mailgun, there is a workaround:
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
# to=to_emails, # Replace this
bcc=to_emails, # with this
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["To"] = "%recipient%" # Add this
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
Reference: https://stackoverflow.com/questions/37948729/mailgun-smtp-batch-sending-with-recipient-variables-shows-all-recipients-in-to-field
Why does to=["%recipient%"] not work with SMTP?
It's the standard in the protocol.
From https://documentation.mailgun.com/_/downloads/en/latest/pdf/:
SMTP send will error with “cannot parse to address” or “cannot parse from address” if the provided email address fails syntax checks in accordance with RFC5321, RFC5322, RFC6854.
What to do for proper support of Batch Sending with Mailgun?
Use the API.
From https://stackoverflow.com/questions/30787399/laravel-5-sending-group-emails (multiposted to https://laracasts.com/discuss/channels/laravel/sending-email-to-1000s-of-reciepents):
So far, I have created an array of recipient email addresses, sent the email to a webmaster type address, and included the end recipients in BCC
While this works, it's not ideal.
Rather than using Laravel's built in Mail, I elected to use Mailgun's API (specifically batch sending) directly
This also allows me to access unique recipient variables within my email template
(It's not specific to Laravel/PHP, but to SMTP via Mailgun.)
What do you mean by "unintuitive" implementation of BCC in Mailgun?
Mailgun effectively personalises the email for each BCC recipient using recipient-variables.
From https://github.com/mailgun/mailgun-js-boland/issues/89:
the bcc person is receiving the email as it was addressed to them instead of being part of the bcc
This causes a separate issue when you actually want BCC recipients to get the same content.
From https://stackoverflow.com/questions/48887866/bcc-in-mailgun-batch-send-does-not-include-substitutions:
In the copy sent to the bcc address, the recip_vars substitution has not been made.
According to the good people at Mailgun, this is not possible, at least in the current release of the service.
I am currently trying to write a script to send off a request token, I have the header, and the claimset, but I don't understand the signature! OAuth requires my private key to be encrypted with SHA256withRSA (also known as RSASSA-PKCS1-V1_5-SIGN with the SHA-256 hash function), but the closest I could find was RSAES-PKCS1-v1_5 (has RSA, and the SHA-256 hash). I followed the example, and tweaked it, so I could get it set, but heres my dillema:
signature = ""
h = SHA.new (signature)
key = RSA.importKey(open('C:\Users\Documents\Library\KEY\My Project 905320c6324f.json').read())
cipher = PKCS1_v1_5.new(key)
ciphertext = cipher.encrypt(message+h.digest())
print(ciphertext)
I'm a bit lost, the JSON file I was given has both public key, and private, do I copy and paste the private key into the signature variable (it gave me a invalid syntax)? Or do I past the directory again? I am so lost, and way over my head haha. I am currently running Python 3.4, with pyCrypto for the signature.
Based on what you've said below about wanting to write a command system using gmail, I wrote a simple script to do this using IMAP. I think this is probably simpler than trying to use Google APIs for a single user, unless you were wanting to do that simply for the exercise.
import imaplib, logging
from time import sleep
USERNAME = 'YOUR_USERNAME_HERE' # For gmail, this is your full email address.
PASSWORD = 'YOUR_PASSWORD_HERE'
CHECK_DELAY = 60 # In seconds
LOGGING_FORMAT = '%(asctime)s %(message)s'
logging.basicConfig(filename='imapTest.log', format=LOGGING_FORMAT, level=logging.INFO)
logging.info("Connecting to IMAP server...")
imap = imaplib.IMAP4_SSL('imap.gmail.com')
imap.login(USERNAME, PASSWORD)
logging.info("Connected to IMAP server.")
def get_command_messages():
logging.info("Checking for new commands.")
imap.check()
# Search the inbox (server-side) for messages containing the subject 'COMMAND' and which are from you.
# Substitute USERNAME below for the sending email address if it differs.
typ, data = imap.search(None, '(FROM "%s" SUBJECT "COMMAND")' %(USERNAME))
return data[0]
def delete_messages(message_nums):
logging.info("Deleting old commands.")
for message in message_nums.split():
imap.store(message, '+FLAGS', '\\DELETED')
imap.expunge()
# Select the inbox
imap.select()
# Delete any messages left over that match commands, so we are starting 'clean'.
# This probably isn't the nicest way to do this, but saves checking the DATE header.
message_nums = get_command_messages()
delete_messages(message_nums)
try:
while True:
sleep(CHECK_DELAY)
# Get the message body and sent time. Use BODY.PEEK instead of BODY if you don't want to mark the message as read, but we're deleting it anyway below.
message_nums = get_command_messages()
if message_nums:
# search returns space-separated message IDs, but we need them comma-separated for fetch.
typ, messages = imap.fetch(message_nums.replace(' ', ','), '(BODY[TEXT])')
logging.info("Found %d commands" %(len(messages[0])))
for message in messages[0]:
# You now have the message body in the message variable.
# From here, you can check against it to perform commands, e.g:
if 'shutdown' in message:
print("I got a shutdown command!")
# Do stuff
delete_messages(message_nums)
finally:
try:
imap.close()
except:
pass
imap.logout()
If you're set on using the Gmail API, though, Google strongly encourage you to use their existing Python library rather than attempt to do full authentication etc. yourself as you appear to be. With that, it should - more or less - be a case of replacing the imap calls above with the relevant Gmail API ones.
I would like to receive email using python. So far I have been able to get the subject but not the body. Here is the code I have been using:
import poplib
from email import parser
pop_conn = poplib.POP3_SSL('pop.gmail.com')
pop_conn.user('myusername')
pop_conn.pass_('mypassword')
#Get messages from server:
messages = [pop_conn.retr(i) for i in range(1, len(pop_conn.list()[1]) + 1)]
# Concat message pieces:
messages = ["\n".join(mssg[1]) for mssg in messages]
#Parse message intom an email object:
messages = [parser.Parser().parsestr(mssg) for mssg in messages]
for message in messages:
print message['subject']
print message['body']
pop_conn.quit()
My issue is that when I run this code it properly returns the Subject but not the body. So if I send an email with the subject "Tester" and the body "This is a test message" it looks like this in IDLE.
>>>>Tester >>>>None
So it appears to be accurately assessing the subject but not the body, I think it is in the parsing method right? The issue is that I don't know enough about these libraries to figure out how to change it so that it returns both a subject and a body.
The object message does not have a body, you will need to parse the multiple parts, like this:
for part in message.walk():
if part.get_content_type():
body = part.get_payload(decode=True)
The walk() function iterates depth-first through the parts of the email, and you are looking for the parts that have a content-type. The content types can be either text/plain or text/html, and sometimes one e-mail can contain both (if the message content_type is set to multipart/alternative).
The email parser returns an email.message.Message object, which does not contain a body key, as you'll see if you run
print message.keys()
What you want is the get_payload() method:
for message in messages:
print message['subject']
print message.get_payload()
pop_conn.quit()
But this gets complicated when it comes to multi-part messages; get_payload() returns a list of parts, each of which is a Message object. You can get a particular part of the multipart message by using get_payload(i), which returns the ith part, raises an IndexError if i is out of range, or raises a TypeError if the message is not multipart.
As Gustavo Costa De Oliveir points out, you can use the walk() method to get the parts in order -- it does a depth-first traversal of the parts and subparts of the message.
There's more about the email.parser module at http://docs.python.org/library/email.message.html#email.message.Message.
it also good return data in correct encoding in message contains some multilingual content
charset = part.get_content_charset()
content = part.get_payload(decode=True)
content = content.decode(charset).encode('utf-8')
Here is how I solved the problem using python 3 new capabilities:
import imaplib
import email
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login(username, password)
mail.select(readonly=True) # refresh inbox
status, message_ids = mail.search(None, 'ALL') # get all emails
for message_id in message_ids[0].split(): # returns all message ids
# for every id get the actual email
status, message_data = mail.fetch(message_id, '(RFC822)')
actual_message = email.message_from_bytes(message_data[0][1])
# extract the needed fields
email_date = actual_message["Date"]
subject = actual_message["Subject"]
message_body = get_message_body(actual_message)
Now get_message_body is actually pretty tricky due to MIME format. I used the function suggested in this answer.
This particular example works with Gmail, but IMAP is a standard protocol, so it should work for other email providers as well, possibly with minor changes.
if u want to use IMAP4. Use outlook python library, download here : https://github.com/awangga/outlook
to retrieve unread email from your inbox :
import outlook
mail = outlook.Outlook()
mail.login('emailaccount#live.com','yourpassword')
mail.inbox()
print mail.unread()
to retrive email element :
print mail.mailbody()
print mail.mailsubject()
print mail.mailfrom()
print mail.mailto()