Searching here and on the internet, there are a lot of examples to how to mark a message as SEEN, even though this is automatic with imap.
But how can I mark an email as UNSEEN or UNREAD.
I have a script in python which receives UNSEEN messages, and it works great. But after reading them, imap automatically marks them as SEEN which works fine but only if the script has no errors, because if it raises an exception, I want the email to be marked again as UNSEEN, so next time the script will read that message again.
How can I achieved this?
I have also used mail.select(mail_label,readonly=True), but it doesn't help because with that I cannot mark a message as SEEN which I also need. I also want this to work with Gmail.
You can easily clear the \Seen flags with this command:
tag UID STORE -FLAGS (\Seen)
but your software will probably be more robost if you only set the \Seen flag in the first place after you have successfully processed a message. That way, if anything goes wrong while you are processing a message (even if the connection to the IMAP server is broken) the flag remains unset and you can retry that message the next time the script runs. You do this by avoiding the IMAP server's automatic setting of the \Seen flag by using BODY.PEEK instead of BODY.
In Python, I think that STORE command should be issued like this but I haven't tried it.
connection.uid('STORE', '-FLAGS', '(\Seen)')
In Python, the imaplib module describes STORE as:
(typ, [data]) = <instance>.store(message_set, command, flags)
so, the following line will let you set the message to READ ('+FLAGS') or UNREAD ('-FLAGS') as required.
connection.uid('STORE', MESSAGE_ID, '+FLAGS', '\SEEN')
As you see, the secrets is on the FLAGS command ;)
`imap = imaplib.IMAP4_SSL(server)
imap.login(username, password)
imap.select("inbox", readonly=False)`
if readonly="True" you can't change any flags.
But,if it is false, you can do as follow,
imap.store(id, '-FLAGS', '\Seen')
THEN EMAIL WILL MARK AS UNREAD
(-) means REMOVE flag and (+) means ADD flag.
ex:you can set imap.store(id, '+FLAGS', '\Deleted') to delete email as well.
Like this you can set,any flag in below
\Seen Message has been read
\Answered Message has been answered
\Flagged Message is "flagged" for urgent/special attention
\Deleted Message is "deleted" for removal by later EXPUNGE
\Draft Message has not completed composition (marked as a
draft).
More details :https://www.rfc-editor.org/rfc/rfc2060.html#page-9
You may use imap_tools package:
https://pypi.org/project/imap-tools/
from imap_tools import MailBox, MailMessageFlags, A
with MailBox('imap.mail.com').login('test#mail.com', 'pwd', 'INBOX') as mailbox:
# FLAG unseen messages in current folder as Answered and Flagged, *in bulk.
flags = (MailMessageFlags.ANSWERED, MailMessageFlags.FLAGGED)
mailbox.flag(mailbox.uids(A(seen=False)), flags, True)
# SEEN: mark all messages sent at 05.03.2007 in current folder as unseen, *in bulk
mailbox.flag(mailbox.uids("SENTON 05-Mar-2007"), MailMessageFlags.SEEN, False)
I am lib author.
I created an IMAP library, Red Box, which is quite intuitive to use for such cases.
Configure
First we configure the email box (read more about configuring):
from redbox import EmailBox
# Create email box instance
box = EmailBox(
host="imap.example.com",
port=993,
username="me#example.com",
password="<PASSWORD>"
)
# Select an email folder
inbox = box["INBOX"]
Search Messages
Then we search for an email (read more about querying):
from redbox.query import SUBJECT
msgs = inbox.search(SUBJECT("Example email"))
msg = msgs[0]
Set Flags
Then we set flags:
# Set \Seen flag
msg.read()
# Remove \Seen flag
msg.unread()
Alternatively you can use set method:
# Set \Seen flag
msg.set(seen=True)
# Remove \Seen flag
msg.set(seen=False)
Read more about messages here.
Relevant links:
Source code
Documentation
Related
I'm having difficulty setting the References: field in the header of an outgoing SMTP email for Gmail. I'm using Python 3.8 with smtplib and email.message libraries. The code is:
reference_ids = [
'<BN8PR17MB27372595A957D7912CEE184FBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>',
'<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>',
'<BN8PR17MB27371C71A65834531DF028BBBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>',
'<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>']
in_reply_to = reference_ids[0]
smtp = smtplib.SMTP_SSL(es.smtp_server)
smtp.login(es.username, es.password)
msg = email.message.EmailMessage()
if (reference_ids is not None):
msg.add_header('In-Reply-To', in_reply_to)
msg.add_header('References', (' ').join(reference_ids))
msg['Subject'] = request.vars.subject
msg['From'] = es.email
msg['To'] = request.vars.to
msg['CC'] = request.vars.cc
msg['BCC'] = request.vars.bcc
msg.set_content(request.vars.message)
smtp.send_message(msg)
smtp.quit()
where reference_ids is a list of Message-IDs previous "in_reply_to"s of "message_id"s back to the originating email.
I can send the email without errors and when I view the "Show original" the References look ok. it is the proper list of Message-IDs of the form "BNX_blah_blah#mail.gmail.com", without quotes, separated by a space.
However, when I try to later read the sent email with the imaplib library and email.message_from_bytes(raw_email_response_body[1]) I get a real mess of characters. Most of the addresses in the References loose their BNX_blah_blah#mail.gmail.com" form. Message-ID and In-Reply-To looks ok though.
References: =?utf-8?q?=22=3CBN8PR17MB27372595A957D7912CEE184FBF6F9=40BN8PR17?=
=?utf-8?q?MB2737=2Enamprd17=2Eprod=2Eoutlook=2Ecom=3E?=
<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>
=?utf-8?q?=3CBN8PR17MB27371C71A65834531DF028BBBF6F9=40BN8PR17MB2737=2Enampr?=
=?utf-8?q?d17=2Eprod=2Eoutlook=2Ecom=3E_=3CBN8PR17MB27377F609B669D0E72638D6?=
=?utf-8?q?9BF6F9=40BN8PR17MB2737=2Enamprd17=2Eprod=2Eoutlook=2Ecom=3E?=
<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>
Am I encoding the References properly? Am I decoding the References I read from IMAP properly?
thanx to everyone for giving me direction, especially tripleee with "That's bog-standard RFC2047 encoding." which led me the email.header library.
so,,, when i send the email via gmail smtp, i could set the 6th line to simply:
msg.add_header('References', ' '.join(reference_ids))
where reference_ids is a python list of straight address strings, like:
['<BN8PR17MB27372595A957D7912CEE184FBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<CAM9Ku=FZ5RGMvw3VzNrZz+DA78zyq2Am8fz9JNLmjUQ9ZEXpDQ#mail.gmail.com>', '<BN8PR17MB27371C71A65834531DF028BBBF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<BN8PR17MB27377F609B669D0E72638D69BF6F9#BN8PR17MB2737.namprd17.prod.outlook.com>', '<CAM9Ku=E1wmpj=AMRhsh-Sk1RHqmK_x-J5ey8szVehefYQvn13w#mail.gmail.com>']
where gmail would show the list under "Show original" as a long string line with space delimited separation.
the problem comes in when you attempt to read it via imaplib where it is encoded via RFC2047. so upon reading the email header, i processed it like:
import imaplib, smtplib, email
from email.header import decode_header, make_header
...
emsg = email.message_from_bytes(raw_email_response_body[1])
...
References = emsg['References']
if References:
References = make_header(decode_header(References))
References = str(References).strip().replace('"', '').replace('\r', '').replace('\t', '').replace('\n', ' ').replace(' '*3, ' '*2).replace(' '*2, ' ').split(' ')
which will faithfully return the list of Message-IDs as originally sent. hopefully this helps a few people out there. and, i thank everyone for the help and direction. lucas
You seem to have uncovered a bug in Python's email library. The References: and In-Reply-To: headers should not be subject to RFC2047 encoding at all.
As a quick and dirty demonstration, I can avoid the problem by shortening the long ones.
As a similarly quick and dirty workaround, you can override the email.policy object with a different one which doesn't force these lines to be shortened. This is a slightly obscure corner of the Python email library; the documentation really presupposes a fair amount of prior knowledge of both email in general and Python's email library in particular.
from email.policy import default
...
custom_policy = default.clone(max_line_length=100)
msg = email.message.EmailMessage(policy=custom_policy)
...
# in your IMAP reader
copy = message_from_bytes(imap_response[1], policy=custom_policy)
Notice, however, that this sets the maximum line length everywhere, not just in the headers. You'd really want to be able to override this setting just for the duration of the addition of these specific headers.
Notice also that message_from_bytes needs a policy= keyword argument to construct a modern EmailMessage object. Without the keyword, you end up creating a legacy email.message.Message object which lacks several of the modern methods of the 3.3+ email API.
Here's a demo: https://ideone.com/eEzIxe
I'm using a standard smtplib.sendmail() call in my Python 3 program to send emails, as follows:
smtp_session.sendmail('The Sender <sender#domain.com>', ['The ÅÄÖ Recipient <recipient#domain.com>'], 'Simple test body here')
The SMTP session has already been successfully established prior to this code line being executed, and it also always works just fine as long as there are no "international characters" in the recipient name.
BUT, as soon as I include e.g. "ÅÄÖ" in the recipient name (which is even just 8-bit ASCII characters, not even "real unicode" or whatever), as can be seen above, the email just disappears and never reaches the sender, although no errors or exceptions are returned or raised by the sendmail() method, nor anything inside it (I have single-stepped it in a debugger while doing this).
I know for a fact that I can send emails with such characters in the recipient names through this exact same SMTP server of mine, using a normal email client program like Thunderbird, so I can only assume that this problem has something to do with some encoding or similar?
Also, the solution shouldn't be related to that mail_options=['SMTPUTF8'] thingy either, because the server just replies that it doesn't support this if I try to use it (and again, emails using these exact recipient names can still be sent through the exact same SMTP server with a normal email client like Thunderbird).
So, is there some simple solution based on using some kind of "MIME related" encoding or similar on the recipient strings that will solve this, or how can I otherwise send an email from Python with such a recipient name?
Characters in SMTP headers are required to be printable ASCII, in the numeric range 33-126 inclusive. If you need to represent characters outside that range in an SMTP header then you must use the encoding method defined by RFC 2231 (which is an evolution of an earlier method defined by RFC 2047).
Historically in Python you would have used the Header class from the email.header module to build suitably-encoded headers. That's still available in Python 3, but in Python 3 the newer recommendation is to use the EmailMessage class from the email.message module to construct the entire message, and let it take care of encoding any headers that need special treatment.
The argument to smtplib.sendmail() should not have human-readable labels, just the address terminus.
smtp_session.sendmail('sender#domain.com', ['recipient#domain.com'],
'Simple test body here')
The email.headerregistry module in Python 3.6+ has a facility for extracting just the email terminus, by way of parsing structured headers into objects with attributes.
from email.headerregistry import AddressHeader
hdr = dict()
AddressHeader.parse('To: The ÅÄÖ Recipient <recipient#domain.com>', hdr)
for grp in hdr['groups']:
for addr in grp.addresses:
print('{0}#{1}'.format(addr.username, addr.domain))
(I really hope there is a less convoluted way to access this functionality but at the very least this produces the expected result.)
In the actual message, Python takes care of properly RFC2047-encoding any headers with Unicode content (if you use the correct methods from the email library to construct a prop0er MIME message); but this is pure presentation (RFC5322) not transport (RFC5321). So in the message itself you might see
From: The Sender <sender#domain.com>
To: The =?utf-8?Q?=C3=85=C3=84=C3=96_Recipient?= <recipient#domain.com>
though keep in mind that there is no requirement for the message content to actually reveal the transport sender or recipient headers. (Maybe tangentially see Header "To:" for a Bulk Email Sender)
Apologies for the amateur question. I'm just learning Python and I'm fumbling around this XMPP bot script using XMPP.
I have a bot built using the MUC bot example from SleekXMPP: http://sleekxmpp.com/getting_started/muc.html
Where my bot differs from the example is my script creates a SQLite database and on each group_message event, parses the XML to retrieve the nick and message body text and write it to the database with a timestamp.
Here is the part of my bot that's recording the msg output from the XMPP channel:
def groupchat_message(self, msg):
if msg['type'] in ('groupchat'):
raw = str(msg) # Save raw XML as a string in the database for debugging purposes
timestamp = datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')
fromuser = str(msg['from']) # Convert "from" attribute to string so it can be split
author = fromuser.split('/')[1] # Split "from" attribute to remove channel address leaving only nick behind
body = msg['body']
msginsert = [timestamp, author, body, raw] # Database input list to be handed to placeholders.
db.execute("INSERT INTO messages VALUES (?,?,?,?)", msginsert) # "?" placeholder is REQUIRED for automatic database sanitization.
dbconn.commit()
print("[",timestamp,"]",author,": ",body, sep='')
else:
print(msg)
The print statements are just for debugging purposes so I can watch the messages tick by in the terminal so I know the script is running.
In the recorded information I would like to also include the user's role or affiliation in the XMPP channel so admins and moderators can be singled out. This is what I'm having trouble wrapping my head around. It seems like I should be able to do it based on the SleekXMPP stanza docs but I'm having trouble figuring out how I get from the message XML to the role/affiliation information.
How can I take the information provided in the message XML and get back the role and/or affiliation of the user who posted that message?
For reference, this is the raw XML output from the XMPP channel for messages:
<message to="username#example.com" from="channelname#channels.example.com/User Nick" id="1453" type="groupchat">
<body>This is the message body text</body>
<x xmlns="jabber:x:event">
<composing />
</x>
</message>
The role/affiliation isn't included in the message, it is part of the state of the chat.
As a reminder, XMPP has three different stanzas that get send: <message>, which you had as an example, <iq>, which are used to retrieve or set things and <presence>, which indicates the presence of things. The role/affiliation information is included in a presence stanza. For example in the first presence stanzas your client receives to inform it of who is present in the room already, see Example 21 from XEP-0045 (Multi-User Chat). Your client also receives new presence stanzas whenever someone's nickname, role or affiliation changes or when they leave the room.
You should make sure you store this information yourself, because SleekXMPP doesn't do it for you. This can be done by creating a dictionary that stores, for every nickname, their role and one for their affiliation. On presence changes you should make sure to update this information. Then you can use those dictionaries in your message handler to log their role/affiliation.
So something like:
def __init__(...):
self.roles = dict()
self.affiliations = dict()
self.add_event_handler(""groupchat_presence"", self.muc_presence)
...
def muc_presence(self, presence):
nick = presence['muc']['nick']
self.roles[nick] = presence['muc']['role']
self.affiliations[nick] = presence['muc']['affiliation']
This is the general idea, you'll need to do some more work to make it handle nickname changes and people leaving the room properly.
I'm trying to set an email as read using the Python IMAP library. Just like in this example:
Python, IMAP and GMail. Mark messages as SEEN
I select a GMail account's "All email" folder then obtain the UID of a specific email through the search command. Then I use that UID to try and mark it as read, but fail.
result, data = mail.uid('fetch', email_uid, '(FLAGS)')
print "RESULT: ",result,data
mail.select(folder_name, readonly=False)
result, data = mail.store(email_uid, '+FLAGS', '\\Seen')
print "RESULT OF OPERATION FOR UID",email_uid,result,data
No matter how many times I run this piece of code I always obtain the flags not containing the \Seen flag.
Sample execution (ignore Django prints):
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
RESULT OF OPERATION FOR UID 54264 OK [None]
[09/Mar/2013 17:30:37] "GET /api/mark_as_read/320/?token=8e3c057b841b75c864685786b2a9657aadf17e3bfb991b103b7c4b3ffdd2a753&refresh_mode=all&folder_mode=inbox HTTP/1.1" 200 144
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
RESULT OF OPERATION FOR UID 54264 OK [None]
[09/Mar/2013 17:30:44] "GET /api/mark_as_read/320/?token=8e3c057b841b75c864685786b2a9657aadf17e3bfb991b103b7c4b3ffdd2a753&refresh_mode=all&folder_mode=inbox HTTP/1.1" 200 144
FINAL QUERY: (SENTSINCE 07-Mar-2013 SENTBEFORE 11-Mar-2013 LARGER 7508 SMALLER 7510)
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk \\Seen))']
RESULT OF OPERATION FOR UID 54264 OK [None]
I tried multiple variations. I tried setting the flag \Seen, Seen, \Seen, SEEN, with parenthesis, without parenthesis, and have no idea what I might try apart from this.
I suspect there might be some very specific detail here - maybe the GMail "All email" folder does not allow to mark an email as read?
I also don't know if, in the (result,data) tuple returned in the mail.store command, if the data variable is supposed to have the "[None]" variable for a normal execution, or if it means that something is wrong.
If I try to set a wrong flag like \\Seen the mail.store command raises an error. So when setting the \Seen flag it is a valid IMAP flag.
I have no idea what to try next.
Any help would be very appreiated.
The IMAP specification for the STORE command does not have an option to set flags by UID. You can only set them (like in the example you linked to) by message set.
The only reason it works sometimes is that the sequences for your UID's and message set's are relatively close (31424 vs 54264).
RESULT: OK ['31424 (UID 54264 FLAGS (NotJunk $NotJunk))']
You're not actually ever setting the flags by UID, you're setting them by message set...which occasionally corresponds to a UID.
You can set flags by UID however (just not with a STORE command). You can issue an UID Command with the first argument being store to set flags:
result, data = mail.uid('store', '542648', '+FLAGS', '(\\Seen)')
If successful, the server will respond with the message set corresponding to the UID you used -eg (using your example):
OK ['31424 (FLAGS (\\Seen))']
You may use my package:
https://pypi.org/project/imap-tools/
with MailBox('imap.mail.com').login('test#mail.com', 'pwd', 'INBOX') as mailbox:
# SEEN: mark all messages sent at 05.03.2007 in current folder as unseen, *in bulk
mailbox.seen(mailbox.uids("SENTON 05-Mar-2007"), False)
This question already has answers here:
Fetch an email with imaplib but do not mark it as SEEN
(4 answers)
Closed 7 years ago.
Right now its a gmail box but sooner or later I want it to scale.
I want to sync a copy of a live personal mailbox (inbox and outbox) somewhere else, but I don't want to affect the unread state of any unread messages.
what type of access will make this easiest? I can't find any information if IMAP will affect the read state, but it appears I can manually reset a message to unread. Pop by definition doesn't affect unread state but nobody seems to use pop to access their gmail, why?
In the IMAP world, each message has flags. You can set the individual flags on each message. When you Fetch a message, it's actually possible to read the message, without applying the \Seen flag.
Most mail clients will apply the \Seen flag when the message is read. So, if the message has already been read, outside of your app, then you will need to remove the \Seen flag.
Just as fyi...here is the relevant part about flags from the RFCs:
A system flag is a flag name that is pre-defined in this
specification. All system flags begin with "\". Certain system
flags (\Deleted and \Seen) have special semantics described
elsewhere. The currently-defined system flags are:
\Seen
Message has been read
\Answered
Message has been answered
\Flagged
Message is "flagged" for urgent/special attention
\Deleted
Message is "deleted" for removal by later EXPUNGE
\Draft
Message has not completed composition (marked as a draft).
\Recent
Message is "recently" arrived in this mailbox. This session
is the first session to have been notified about this
message; if the session is read-write, subsequent sessions
will not see \Recent set for this message. This flag can not
be altered by the client.
If it is not possible to determine whether or not this
session is the first session to be notified about a message,
then that message SHOULD be considered recent.
If multiple connections have the same mailbox selected
simultaneously, it is undefined which of these connections
will see newly-arrived messages with \Recent set and which
will see it without \Recent set.
There is a .PEEK option on the FETCH command in IMAP that will explicitly not set the /Seen flag.
Look at the FETCH command in RFC 3501 and scroll down a bit to page 57 or search for "BODY.PEEK".
You need to specify section when you use BODY.PEEK. Sections are explained in IMAP Fetch Command documentations under BODY[<section>]<<partial>>
import getpass, imaplib
M = imaplib.IMAP4()
M.login(getpass.getuser(), getpass.getpass())
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(BODY.PEEK[])')
print 'Message %s\n%s\n' % (num, data[0][5])
M.close()
M.logout()
PS: I wanted to fix answer given Gene Wood but was not allowed because edit was smaller than 6 characters (BODY.PEEK -> BODY.PEEK[])
Nobody uses POP because typically they want the extra functionality of IMAP, such as tracking message state. When that functionality is only getting in your way and needs workarounds, I think using POP's your best bet!-)
if it helps anyone, GAE allows you to receive email as an HTTP request, so for now i'm just forwarding emails there.
To follow up on Dan Goldstein's answer above, in python the syntax to use the ".PEEK" option would be to call IMAP4.fetch and pass it "BODY.PEEK"
To apply this to the example in the python docs :
import getpass, imaplib
M = imaplib.IMAP4()
M.login(getpass.getuser(), getpass.getpass())
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(BODY.PEEK)')
print 'Message %s\n%s\n' % (num, data[0][5])
M.close()
M.logout()