This question already has answers here:
Fetch an email with imaplib but do not mark it as SEEN
(4 answers)
Closed 7 years ago.
Right now its a gmail box but sooner or later I want it to scale.
I want to sync a copy of a live personal mailbox (inbox and outbox) somewhere else, but I don't want to affect the unread state of any unread messages.
what type of access will make this easiest? I can't find any information if IMAP will affect the read state, but it appears I can manually reset a message to unread. Pop by definition doesn't affect unread state but nobody seems to use pop to access their gmail, why?
In the IMAP world, each message has flags. You can set the individual flags on each message. When you Fetch a message, it's actually possible to read the message, without applying the \Seen flag.
Most mail clients will apply the \Seen flag when the message is read. So, if the message has already been read, outside of your app, then you will need to remove the \Seen flag.
Just as fyi...here is the relevant part about flags from the RFCs:
A system flag is a flag name that is pre-defined in this
specification. All system flags begin with "\". Certain system
flags (\Deleted and \Seen) have special semantics described
elsewhere. The currently-defined system flags are:
\Seen
Message has been read
\Answered
Message has been answered
\Flagged
Message is "flagged" for urgent/special attention
\Deleted
Message is "deleted" for removal by later EXPUNGE
\Draft
Message has not completed composition (marked as a draft).
\Recent
Message is "recently" arrived in this mailbox. This session
is the first session to have been notified about this
message; if the session is read-write, subsequent sessions
will not see \Recent set for this message. This flag can not
be altered by the client.
If it is not possible to determine whether or not this
session is the first session to be notified about a message,
then that message SHOULD be considered recent.
If multiple connections have the same mailbox selected
simultaneously, it is undefined which of these connections
will see newly-arrived messages with \Recent set and which
will see it without \Recent set.
There is a .PEEK option on the FETCH command in IMAP that will explicitly not set the /Seen flag.
Look at the FETCH command in RFC 3501 and scroll down a bit to page 57 or search for "BODY.PEEK".
You need to specify section when you use BODY.PEEK. Sections are explained in IMAP Fetch Command documentations under BODY[<section>]<<partial>>
import getpass, imaplib
M = imaplib.IMAP4()
M.login(getpass.getuser(), getpass.getpass())
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(BODY.PEEK[])')
print 'Message %s\n%s\n' % (num, data[0][5])
M.close()
M.logout()
PS: I wanted to fix answer given Gene Wood but was not allowed because edit was smaller than 6 characters (BODY.PEEK -> BODY.PEEK[])
Nobody uses POP because typically they want the extra functionality of IMAP, such as tracking message state. When that functionality is only getting in your way and needs workarounds, I think using POP's your best bet!-)
if it helps anyone, GAE allows you to receive email as an HTTP request, so for now i'm just forwarding emails there.
To follow up on Dan Goldstein's answer above, in python the syntax to use the ".PEEK" option would be to call IMAP4.fetch and pass it "BODY.PEEK"
To apply this to the example in the python docs :
import getpass, imaplib
M = imaplib.IMAP4()
M.login(getpass.getuser(), getpass.getpass())
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(BODY.PEEK)')
print 'Message %s\n%s\n' % (num, data[0][5])
M.close()
M.logout()
Related
I'm using a standard smtplib.sendmail() call in my Python 3 program to send emails, as follows:
smtp_session.sendmail('The Sender <sender#domain.com>', ['The ÅÄÖ Recipient <recipient#domain.com>'], 'Simple test body here')
The SMTP session has already been successfully established prior to this code line being executed, and it also always works just fine as long as there are no "international characters" in the recipient name.
BUT, as soon as I include e.g. "ÅÄÖ" in the recipient name (which is even just 8-bit ASCII characters, not even "real unicode" or whatever), as can be seen above, the email just disappears and never reaches the sender, although no errors or exceptions are returned or raised by the sendmail() method, nor anything inside it (I have single-stepped it in a debugger while doing this).
I know for a fact that I can send emails with such characters in the recipient names through this exact same SMTP server of mine, using a normal email client program like Thunderbird, so I can only assume that this problem has something to do with some encoding or similar?
Also, the solution shouldn't be related to that mail_options=['SMTPUTF8'] thingy either, because the server just replies that it doesn't support this if I try to use it (and again, emails using these exact recipient names can still be sent through the exact same SMTP server with a normal email client like Thunderbird).
So, is there some simple solution based on using some kind of "MIME related" encoding or similar on the recipient strings that will solve this, or how can I otherwise send an email from Python with such a recipient name?
Characters in SMTP headers are required to be printable ASCII, in the numeric range 33-126 inclusive. If you need to represent characters outside that range in an SMTP header then you must use the encoding method defined by RFC 2231 (which is an evolution of an earlier method defined by RFC 2047).
Historically in Python you would have used the Header class from the email.header module to build suitably-encoded headers. That's still available in Python 3, but in Python 3 the newer recommendation is to use the EmailMessage class from the email.message module to construct the entire message, and let it take care of encoding any headers that need special treatment.
The argument to smtplib.sendmail() should not have human-readable labels, just the address terminus.
smtp_session.sendmail('sender#domain.com', ['recipient#domain.com'],
'Simple test body here')
The email.headerregistry module in Python 3.6+ has a facility for extracting just the email terminus, by way of parsing structured headers into objects with attributes.
from email.headerregistry import AddressHeader
hdr = dict()
AddressHeader.parse('To: The ÅÄÖ Recipient <recipient#domain.com>', hdr)
for grp in hdr['groups']:
for addr in grp.addresses:
print('{0}#{1}'.format(addr.username, addr.domain))
(I really hope there is a less convoluted way to access this functionality but at the very least this produces the expected result.)
In the actual message, Python takes care of properly RFC2047-encoding any headers with Unicode content (if you use the correct methods from the email library to construct a prop0er MIME message); but this is pure presentation (RFC5322) not transport (RFC5321). So in the message itself you might see
From: The Sender <sender#domain.com>
To: The =?utf-8?Q?=C3=85=C3=84=C3=96_Recipient?= <recipient#domain.com>
though keep in mind that there is no requirement for the message content to actually reveal the transport sender or recipient headers. (Maybe tangentially see Header "To:" for a Bulk Email Sender)
I've configured postfix on the email server with .forward file which saves a copy of email and invokes a python script. These emails are stored in Maildir format.
I want to use this Python script to send a reply to the sender acknowledging that the email has been received. I was wondering if there is any way I can open/access that e-mail, get the header info and sender address and send email back.
I looked at several examples of Maildir functions of Python, but they mostly add/delete e-mails. How can I open the latest e-mail received in Maildir/new and get the required information?
The program I have so far:
md = mailbox.Maildir('/home/abcd/Maildir')
message = md.iterkeys().next()
#print message
#for msg in md:
# subject = msg.get('Subject',"")
# print subject
print message
sender = message.get('From',"")
print sender
When I execute this, I do get the sender name. But It is rather the oldest email arrived in Maildir/new folder not the latest one.
Also, if I use get_date function, what if two (or more) e-mails arrive on the same day?
The MaildirMessage's method .get_date() gets you the timestamp of the
message file on disc. Depending on your filesystem, this may have anywhere between two second and nanosecond accuracy. The changes of two messages giving the same value with .get_date() are vastly smaller than when this actually returned a date only.
However if the message files were touched for some reason the return from .get_date() would not be relevant at all. Dovecot e.g. explicitly states that a files mtime should not be changed.
There are several dates associated with a MaildirMessage:
The arrival time timestamp, as encoded in the name of message (the part before the first dot, these are "whole" seconds). If the part
between the first and second dot has a segment of the form Mn than n is the microsecond arrival time, and be used to improve the resolution of the timestamp.
The timestamp of the file on disc
The 'Date:' header field as set by the sending program (or added by some
MTA)
The dates added by intermediate MTA in the 'Received:' header field
The last of these might not be available e.g. if you and the sender are on the same mail server. The third can be easily faked/incorrect (ever got spam in your inbox dated many years ago?). And the second is incorrect if the file ever got touched.
That leaves selecting on the first option:
d = {}
for name in md.keys():
d.setdefault(int(name.split('.', 1)[0]), []).append(name)
result = sorted(d.items())[-1][1]
assert len(result) == 1 # might fail
msg = md.get_message(result[0])
If you are lucky result is a list with a single item. But this value has only second resolution, so you might have multiple emails and then you have to decide on how to decide which message to select based on one of the other values (e.g. by sorting using the files timestamp .get_date()) or just select the first, randomly select one. (If you have the log file, you can search for the result messages' keys in there to determine which one arrived latest).
If you wouldn't convert to int, and have old emails (i.e. before 2001-09-09 03:46:40) a string comparison would probably not give you the message with the latest arrival time.
Some hints for this:
You can open a Maildir with the mailbox.Maildir class (see the Documentation for mailbox)
You can iterate over all the mails in a Maildir via the method itervalues
Now you get all the mails in the Maildir. One of them is the most recent one.
The mails are objects of the class MaildirMessage, which is a subclass of Message. For these classes, also a documentation exists (on the same page as mailbox, currently)
With the method "get_date" on those objects, you can find out, which one is the most recent one. You still have to select it yourself.
So much as beginners help: A little bit you should also do by yourself.
You should make yourself familiar with the Python documentation - I agree, that it is not easy to find the right packages and how to use them, but you can try them directly in the Python shell.
Ok, here another code snippet:
newest = None
for message in md.itervalues():
if newest == None or message.get_date() > newest.get_date():
newest = message
# now newest should contain the newest message
Did not see your last question: get_date does not only contain the date, but also the time, because it gives the number of seconds since (normally) 1970.
I am trying to send a message in Python with smtplib. I have a sendEmail(message) function:
def sendEmail(message)
server = smtplib.SMTP_SSL('mailserver.example.com', 465)
server.ehlo()
server.login("username", "password")
print message
msg = message
fromaddr = "myaddr#someplace.com"
toaddr = "theiraddr#someotherplace.com"
server.sendmail(fromaddr, toaddr, msg)
server.quit()
I am trying to do something like this:
myMessage = "A String Literal" + someString
sendEmail(myMessage)
But when I do that I sometimes receive an empty email message. someString is a unicode string pulled from sqlite. I have tried various string concatenations combining literals with literals, sqlite strings with sqlite strings, sqlite strings with literals, each by themselves. I have used + and .join and anything else I can think of to join the strings. It seems like everything works sometimes but nothing works every time.
The "print message" inside sendEmail function always prints the string I expect. smtplib always sends the email message to the correct email address without complaining, it is just sometimes an empty message.
Since I have had everything work and also not work, I don't really trust any solution I come up with if I don't fully understand what is going on. Can anybody help me understand what is happening with these strings so I can construct my message in the most appropriate way and be confident that my application will work reliably?
Here is a more detailed example of what I am trying to do:
#connect to the database
dbconn = sqlite3.connect(config["database"])
dbconn.row_factory = sqlite3.Row
dbc = dbconn.cursor()
while true:
#step 1: Check the things and set alarm status in the alarms table
#this step reads from and writes to the database
#step 2: Check each alarm and send a message if necessary
dbc.execute('SELECT * FROM alarms')
theAlarms = dbc.fetchall()
for theAlarm in theAlarms:
if bool(theAlarm['alertShouldBeSent'):
sendEmail('ALARM!!: ' + theAlarm['message'])
dbc.execute('UPDATE alarms SET alertShouldBeSent=0 WHERE id=?',(theAlarm['id']),)
elif bool(theAlarm['allClearShouldBeSent']):
sendEmail('NORMAL: ' + theAlarm['message'])
dbc.execute('UPDATE alarms SET allClearShouldBeSent=0 WHERE id=?',(theAlarm['id'],))
dbconn.commit()
Each row in the alarm table defines a condition that should trigger an alarm and has a field that indicates whether the alarm condition is met making the alarm active
theAlarm['message'] from the database is something like "The batteries are on fire" or "Somebody left the door open." There are currently only 4 or 5 rows in the alarms table. The logic that determines theAlarm['alertShouldBeSent'] makes sure an alert is only sent once per alarm condition and won't be continuously sent if the alarm doesn't first reset. The same is true for theAlarm['allClearShouldBeSent']. The loop runs continuously but emails are sent infrequently. In testing, I set one row into an alarm state. I can verify that the code to send the email is triggered when it should be.
Had the same problem as you, so I went googling and I found your post. As I was reading I realised it must be the damn parser smtplib is using. It doesn't know how to parse ":".
There may be other characters it can't parse, but I haven't found any.
Searching here and on the internet, there are a lot of examples to how to mark a message as SEEN, even though this is automatic with imap.
But how can I mark an email as UNSEEN or UNREAD.
I have a script in python which receives UNSEEN messages, and it works great. But after reading them, imap automatically marks them as SEEN which works fine but only if the script has no errors, because if it raises an exception, I want the email to be marked again as UNSEEN, so next time the script will read that message again.
How can I achieved this?
I have also used mail.select(mail_label,readonly=True), but it doesn't help because with that I cannot mark a message as SEEN which I also need. I also want this to work with Gmail.
You can easily clear the \Seen flags with this command:
tag UID STORE -FLAGS (\Seen)
but your software will probably be more robost if you only set the \Seen flag in the first place after you have successfully processed a message. That way, if anything goes wrong while you are processing a message (even if the connection to the IMAP server is broken) the flag remains unset and you can retry that message the next time the script runs. You do this by avoiding the IMAP server's automatic setting of the \Seen flag by using BODY.PEEK instead of BODY.
In Python, I think that STORE command should be issued like this but I haven't tried it.
connection.uid('STORE', '-FLAGS', '(\Seen)')
In Python, the imaplib module describes STORE as:
(typ, [data]) = <instance>.store(message_set, command, flags)
so, the following line will let you set the message to READ ('+FLAGS') or UNREAD ('-FLAGS') as required.
connection.uid('STORE', MESSAGE_ID, '+FLAGS', '\SEEN')
As you see, the secrets is on the FLAGS command ;)
`imap = imaplib.IMAP4_SSL(server)
imap.login(username, password)
imap.select("inbox", readonly=False)`
if readonly="True" you can't change any flags.
But,if it is false, you can do as follow,
imap.store(id, '-FLAGS', '\Seen')
THEN EMAIL WILL MARK AS UNREAD
(-) means REMOVE flag and (+) means ADD flag.
ex:you can set imap.store(id, '+FLAGS', '\Deleted') to delete email as well.
Like this you can set,any flag in below
\Seen Message has been read
\Answered Message has been answered
\Flagged Message is "flagged" for urgent/special attention
\Deleted Message is "deleted" for removal by later EXPUNGE
\Draft Message has not completed composition (marked as a
draft).
More details :https://www.rfc-editor.org/rfc/rfc2060.html#page-9
You may use imap_tools package:
https://pypi.org/project/imap-tools/
from imap_tools import MailBox, MailMessageFlags, A
with MailBox('imap.mail.com').login('test#mail.com', 'pwd', 'INBOX') as mailbox:
# FLAG unseen messages in current folder as Answered and Flagged, *in bulk.
flags = (MailMessageFlags.ANSWERED, MailMessageFlags.FLAGGED)
mailbox.flag(mailbox.uids(A(seen=False)), flags, True)
# SEEN: mark all messages sent at 05.03.2007 in current folder as unseen, *in bulk
mailbox.flag(mailbox.uids("SENTON 05-Mar-2007"), MailMessageFlags.SEEN, False)
I am lib author.
I created an IMAP library, Red Box, which is quite intuitive to use for such cases.
Configure
First we configure the email box (read more about configuring):
from redbox import EmailBox
# Create email box instance
box = EmailBox(
host="imap.example.com",
port=993,
username="me#example.com",
password="<PASSWORD>"
)
# Select an email folder
inbox = box["INBOX"]
Search Messages
Then we search for an email (read more about querying):
from redbox.query import SUBJECT
msgs = inbox.search(SUBJECT("Example email"))
msg = msgs[0]
Set Flags
Then we set flags:
# Set \Seen flag
msg.read()
# Remove \Seen flag
msg.unread()
Alternatively you can use set method:
# Set \Seen flag
msg.set(seen=True)
# Remove \Seen flag
msg.set(seen=False)
Read more about messages here.
Relevant links:
Source code
Documentation
I'm trying to set an email as read using the Python IMAP library. Just like in this example:
Python, IMAP and GMail. Mark messages as SEEN
I select a GMail account's "All email" folder then obtain the UID of a specific email through the search command. Then I use that UID to try and mark it as read, but fail.
result, data = mail.uid('fetch', email_uid, '(FLAGS)')
print "RESULT: ",result,data
mail.select(folder_name, readonly=False)
result, data = mail.store(email_uid, '+FLAGS', '\\Seen')
print "RESULT OF OPERATION FOR UID",email_uid,result,data
No matter how many times I run this piece of code I always obtain the flags not containing the \Seen flag.
Sample execution (ignore Django prints):
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
RESULT OF OPERATION FOR UID 54264 OK [None]
[09/Mar/2013 17:30:37] "GET /api/mark_as_read/320/?token=8e3c057b841b75c864685786b2a9657aadf17e3bfb991b103b7c4b3ffdd2a753&refresh_mode=all&folder_mode=inbox HTTP/1.1" 200 144
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
RESULT OF OPERATION FOR UID 54264 OK [None]
[09/Mar/2013 17:30:44] "GET /api/mark_as_read/320/?token=8e3c057b841b75c864685786b2a9657aadf17e3bfb991b103b7c4b3ffdd2a753&refresh_mode=all&folder_mode=inbox HTTP/1.1" 200 144
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk \\Seen))']
RESULT OF OPERATION FOR UID 54264 OK [None]
I tried multiple variations. I tried setting the flag \Seen, Seen, \Seen, SEEN, with parenthesis, without parenthesis, and have no idea what I might try apart from this.
I suspect there might be some very specific detail here - maybe the GMail "All email" folder does not allow to mark an email as read?
I also don't know if, in the (result,data) tuple returned in the mail.store command, if the data variable is supposed to have the "[None]" variable for a normal execution, or if it means that something is wrong.
If I try to set a wrong flag like \\Seen the mail.store command raises an error. So when setting the \Seen flag it is a valid IMAP flag.
I have no idea what to try next.
Any help would be very appreiated.
The IMAP specification for the STORE command does not have an option to set flags by UID. You can only set them (like in the example you linked to) by message set.
The only reason it works sometimes is that the sequences for your UID's and message set's are relatively close (31424 vs 54264).
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
You're not actually ever setting the flags by UID, you're setting them by message set...which occasionally corresponds to a UID.
You can set flags by UID however (just not with a STORE command). You can issue an UID Command with the first argument being store to set flags:
result, data = mail.uid('store', '542648', '+FLAGS', '(\\Seen)')
If successful, the server will respond with the message set corresponding to the UID you used -eg (using your example):
OK ['31424 (FLAGS (\\Seen))']
You may use my package:
https://pypi.org/project/imap-tools/
with MailBox('imap.mail.com').login('test#mail.com', 'pwd', 'INBOX') as mailbox:
# SEEN: mark all messages sent at 05.03.2007 in current folder as unseen, *in bulk
mailbox.seen(mailbox.uids("SENTON 05-Mar-2007"), False)