Newline characters disappearing - python

The issue that I am having is that when I download email messages from a Microsoft outlook webmail account, sometimes newline characters are disappearing, resulting in onelongunbrokenline. But only sometimes. Here is the example I am dealing with right now:
Original body of message being downloaded from Microsoft Outlook Web App (dollar signs included because I have :set list on in vim):
Gobble$
This is a message with$
Multiple lines$
$
Hello$
Body of message that I actually end up receiving (also has :set list on in vim):
GobbleThis is a message withMultiple lines^M$
Hello ^I^I ^I ^I^I =^M$
There are clearly a few other things going on here which I also don't understand - where are the tab (^I) characters coming from? Where is that equals sign coming from?
Here is the code that does the downloading (using the python library IMAPClient):
## Connect, login and select the INBOX
server = IMAPClient(HOST, use_uid=True, ssl=ssl)
server.login(USERNAME, PASSWORD)
select_info = server.select_folder('INBOX')
#Get messages since a certain time:
message_list = server.search(['SINCE %s' % cutoff.strftime('%d-%b-%Y')])
response = server.fetch(message_list, ['RFC822'])
for msgid, data in response.iteritems():
msg_string = data['RFC822'].__str__()
msg = email.message_from_string(msg_string)
payload = msg.get_payload()
body = payload
print body

Related

I am trying to make a email chatbot but it spams how could i fix this?

I am trying to build a email chatbot but it has this bug where after it sends the first message, and then gets a response it keeps spamming the answered to the response it got until it gets another response which then it repeats again I was thinking to solve this I should use a variable which detects emails and later down the code a condition that responds only if a email is received, does anyone have any idea on how I could fix this? Thanks
def receive_email():
try:
mail = imaplib.IMAP4_SSL("smtp.gmail.com")
mail.login(email_address, email_password)
mail.select('inbox')
#searches inbox
status, data = mail.search(None, 'Recent')
mail_ids = data[0].split()
latest_email_id = mail_ids[-1]
status, data = mail.fetch(latest_email_id, '(RFC822)')
#gets message
for response_part in data:
if isinstance(response_part, tuple):
msg = email.message_from_bytes(response_part[1])
sender = msg['from']
subject = msg['subject']
if msg.is_multipart():
for part in msg.get_payload():
if part.get_content_type() == 'text/plain':
return part.get_payload()
message = msg.get_payload()
return message,
except Exception as e:
print("Error: ", e)
print("Could not receive email")
return None, None
This is the usual problem for an email autoresponder, if I understand you correctly, and RFC 3834 offers good advice.
Since answers should be self-contained I offer a summary:
Add the Auto-Submitted: auto-replied header field on your outgoing messages. Any value other than no will prevent well-written autoresponders from replying to your outgoing messages.
Set the \answered flag on the message you reply to, immediately before you send the reply.
Change the search key from recent to unanswered not header "auto-submitted" "". unanswered means that the search won't match the messages on which you set the \answered flag, not header "auto-submitted" "" means that you'll not match messages that contain any auto-submitted header field.
Direct your replies to the address in return-path or sender, not the one in from. This is a matter of convention. Auto-submitted mail will often have a special return-path that points to an address that never sends any autoreply.
You may also extend the search key with more details from RFC 3834. The one I suggest should work, but not header "precedence" "junk" will for example prevent your code from replying to a bit of autogenerated mail. Sendgrid and its friends also add header fields you may want to look for and exclude.
If the incoming message has headers like this (use the "view headers" function of most mail readers to see it):
From: example#example.com
Subject: Weekend
To: srtai22#gmail.com
Message-id: <56451182ae7a62978cd6f6ff06dd21e0#example.com>
Then your reply should have headers like this:
Return-Path: <>
From: srtai22#gmail.com
To: example#example.com
Auto-Submitted: auto-replied
Subject: Auto: Weekend
References: <56451182ae7a62978cd6f6ff06dd21e0#example.com>
There'll be many more fields in both, of course. Your reply's return-path says that nothing should respond automatically, From and To are as expected, auto-submitted specifies what sort of response this is, subject doesn't matter very much but this one's polite and well-behaved, and finally references links to the original message.

Adding Content-Disposition header in Python - email isn't sent

Following the directions in Python's email examples and in several Stack Overflow questions, I wrote the following function (sending through the Gmail SMTP server):
def send_email(emailaddr, message, attachmentfile = None, subject = None):
try:
smtpconn = smtplib.SMTP(mainconf["SMTPHOST"], mainconf["SMTPPORT"])
smtpconn.set_debuglevel(1)
smtpconn.ehlo()
smtpconn.starttls()
smtpconn.login(mainconf["SMTPUSER"], mainconf["SMTPPASS"])
if not attachmentfile:
message = '\n' + message
smtpconn.sendmail(mainconf["EMAILFROM"], emailaddr, message)
else:
multipart = MIMEMultipart()
multipart['Subject'] = subject if subject else "Attachment"
multipart['From'] = mainconf["EMAILFROM"]
multipart['To'] = emailaddr
with open(attachmentfile, 'rb') as fp:
filepart = MIMEApplication(fp.read())
multipart.attach(filepart)
multipart.attach(message)
smtpconn.sendmail(mainconf["EMAILFROM"], emailaddr, multipart.as_string())
generallog.info("Sent an email to {0}".format(emailaddr))
except:
generallog.warn("Email sending to {0} failed with error message {1}".format(emailaddr, traceback.format_exception_only(sys.exc_info()[0], sys.exc_info()[1])))
This works fine for sending an email with an attachment, but it results in the famous noname problem (that's just one of several SO questions on the subject). So I add in this:
filepart.add_header('Content-Disposition','attachment',filename=os.basename(attachmentfile))
Then Gmail simply refuses to send the email. When I run this code in the Python shell, I get the standard message accepted message, but the message is never delivered.
Why might this be happening?

how to sign request tokens?

I am currently trying to write a script to send off a request token, I have the header, and the claimset, but I don't understand the signature! OAuth requires my private key to be encrypted with SHA256withRSA (also known as RSASSA-PKCS1-V1_5-SIGN with the SHA-256 hash function), but the closest I could find was RSAES-PKCS1-v1_5 (has RSA, and the SHA-256 hash). I followed the example, and tweaked it, so I could get it set, but heres my dillema:
signature = ""
h = SHA.new (signature)
key = RSA.importKey(open('C:\Users\Documents\Library\KEY\My Project 905320c6324f.json').read())
cipher = PKCS1_v1_5.new(key)
ciphertext = cipher.encrypt(message+h.digest())
print(ciphertext)
I'm a bit lost, the JSON file I was given has both public key, and private, do I copy and paste the private key into the signature variable (it gave me a invalid syntax)? Or do I past the directory again? I am so lost, and way over my head haha. I am currently running Python 3.4, with pyCrypto for the signature.
Based on what you've said below about wanting to write a command system using gmail, I wrote a simple script to do this using IMAP. I think this is probably simpler than trying to use Google APIs for a single user, unless you were wanting to do that simply for the exercise.
import imaplib, logging
from time import sleep
USERNAME = 'YOUR_USERNAME_HERE' # For gmail, this is your full email address.
PASSWORD = 'YOUR_PASSWORD_HERE'
CHECK_DELAY = 60 # In seconds
LOGGING_FORMAT = '%(asctime)s %(message)s'
logging.basicConfig(filename='imapTest.log', format=LOGGING_FORMAT, level=logging.INFO)
logging.info("Connecting to IMAP server...")
imap = imaplib.IMAP4_SSL('imap.gmail.com')
imap.login(USERNAME, PASSWORD)
logging.info("Connected to IMAP server.")
def get_command_messages():
logging.info("Checking for new commands.")
imap.check()
# Search the inbox (server-side) for messages containing the subject 'COMMAND' and which are from you.
# Substitute USERNAME below for the sending email address if it differs.
typ, data = imap.search(None, '(FROM "%s" SUBJECT "COMMAND")' %(USERNAME))
return data[0]
def delete_messages(message_nums):
logging.info("Deleting old commands.")
for message in message_nums.split():
imap.store(message, '+FLAGS', '\\DELETED')
imap.expunge()
# Select the inbox
imap.select()
# Delete any messages left over that match commands, so we are starting 'clean'.
# This probably isn't the nicest way to do this, but saves checking the DATE header.
message_nums = get_command_messages()
delete_messages(message_nums)
try:
while True:
sleep(CHECK_DELAY)
# Get the message body and sent time. Use BODY.PEEK instead of BODY if you don't want to mark the message as read, but we're deleting it anyway below.
message_nums = get_command_messages()
if message_nums:
# search returns space-separated message IDs, but we need them comma-separated for fetch.
typ, messages = imap.fetch(message_nums.replace(' ', ','), '(BODY[TEXT])')
logging.info("Found %d commands" %(len(messages[0])))
for message in messages[0]:
# You now have the message body in the message variable.
# From here, you can check against it to perform commands, e.g:
if 'shutdown' in message:
print("I got a shutdown command!")
# Do stuff
delete_messages(message_nums)
finally:
try:
imap.close()
except:
pass
imap.logout()
If you're set on using the Gmail API, though, Google strongly encourage you to use their existing Python library rather than attempt to do full authentication etc. yourself as you appear to be. With that, it should - more or less - be a case of replacing the imap calls above with the relevant Gmail API ones.

Wrong "uid" when searching with imaplib

I am experiencing a strange problem while developing my python application.
The application is supposed to parse a mail inbox for the unread messages, get a specific ones, process the body and store it into a database.
Everything seem to be working fine for my first 7 mails from my domain, but with the last one and the 4 #gmails, the results don't match with the expected, the mails stored into the database aren't the correct ones, in fact, they are exactly the 4th mail after the correct one.
I am showing the code that I developed, don't be too hard with me, I am kinda new coding:
main.py
from src_pckg import reader
reader.read("***#m***p.com", "***", "imap.***.com", 993, "noreply#s***d.com")
reader.py
def read(username, password, host, port, sender_of_interest):
#Con details
imap_con = imaplib.IMAP4_SSL(host, port)
imap_con.login(username, password)
imap_con.select("INBOX")
#Print all unread messages from a certain sender
status, response = imap_con.search(None, 'UNSEEN', '(FROM "%s")' % (sender_of_interest))
unread_msg_nums = response[0].split()
print(len(unread_msg_nums))
for e_id in unread_msg_nums:
status, data = imap_con.uid('fetch', e_id, '(RFC822)')
msg = data[0][1].decode(encoding='UTF-8')
if re.search("has given you a gift subscription", msg):
#Process the mail
return True

How to receive mail using python

I would like to receive email using python. So far I have been able to get the subject but not the body. Here is the code I have been using:
import poplib
from email import parser
pop_conn = poplib.POP3_SSL('pop.gmail.com')
pop_conn.user('myusername')
pop_conn.pass_('mypassword')
#Get messages from server:
messages = [pop_conn.retr(i) for i in range(1, len(pop_conn.list()[1]) + 1)]
# Concat message pieces:
messages = ["\n".join(mssg[1]) for mssg in messages]
#Parse message intom an email object:
messages = [parser.Parser().parsestr(mssg) for mssg in messages]
for message in messages:
print message['subject']
print message['body']
pop_conn.quit()
My issue is that when I run this code it properly returns the Subject but not the body. So if I send an email with the subject "Tester" and the body "This is a test message" it looks like this in IDLE.
>>>>Tester >>>>None
So it appears to be accurately assessing the subject but not the body, I think it is in the parsing method right? The issue is that I don't know enough about these libraries to figure out how to change it so that it returns both a subject and a body.
The object message does not have a body, you will need to parse the multiple parts, like this:
for part in message.walk():
if part.get_content_type():
body = part.get_payload(decode=True)
The walk() function iterates depth-first through the parts of the email, and you are looking for the parts that have a content-type. The content types can be either text/plain or text/html, and sometimes one e-mail can contain both (if the message content_type is set to multipart/alternative).
The email parser returns an email.message.Message object, which does not contain a body key, as you'll see if you run
print message.keys()
What you want is the get_payload() method:
for message in messages:
print message['subject']
print message.get_payload()
pop_conn.quit()
But this gets complicated when it comes to multi-part messages; get_payload() returns a list of parts, each of which is a Message object. You can get a particular part of the multipart message by using get_payload(i), which returns the ith part, raises an IndexError if i is out of range, or raises a TypeError if the message is not multipart.
As Gustavo Costa De Oliveir points out, you can use the walk() method to get the parts in order -- it does a depth-first traversal of the parts and subparts of the message.
There's more about the email.parser module at http://docs.python.org/library/email.message.html#email.message.Message.
it also good return data in correct encoding in message contains some multilingual content
charset = part.get_content_charset()
content = part.get_payload(decode=True)
content = content.decode(charset).encode('utf-8')
Here is how I solved the problem using python 3 new capabilities:
import imaplib
import email
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login(username, password)
mail.select(readonly=True) # refresh inbox
status, message_ids = mail.search(None, 'ALL') # get all emails
for message_id in message_ids[0].split(): # returns all message ids
# for every id get the actual email
status, message_data = mail.fetch(message_id, '(RFC822)')
actual_message = email.message_from_bytes(message_data[0][1])
# extract the needed fields
email_date = actual_message["Date"]
subject = actual_message["Subject"]
message_body = get_message_body(actual_message)
Now get_message_body is actually pretty tricky due to MIME format. I used the function suggested in this answer.
This particular example works with Gmail, but IMAP is a standard protocol, so it should work for other email providers as well, possibly with minor changes.
if u want to use IMAP4. Use outlook python library, download here : https://github.com/awangga/outlook
to retrieve unread email from your inbox :
import outlook
mail = outlook.Outlook()
mail.login('emailaccount#live.com','yourpassword')
mail.inbox()
print mail.unread()
to retrive email element :
print mail.mailbody()
print mail.mailsubject()
print mail.mailfrom()
print mail.mailto()

Categories