I was having a problem accessing my Gmail Atom feeds using feedparser module. For a non-password protected fees like a blog, for example,
import feedparser
d = feedparser.parse('http://karanjthakkar.wordpress.com/feed/')
print d.feed.title
The values that the feedparser module returned were correct. However when I used it using this to access my Gmail feed,
import urllib2, feedparser
def main():
pwdmgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
pwdmgr.add_password("New mail feed", 'http://mail.google.com/', "karanjthakkar", "my-password")
auth = urllib2.HTTPBasicAuthHandler(pwdmgr)
opener = urllib2.build_opener(auth)
data = opener.open('http://mail.google.com/mail/feed/atom')
d = feedparser.parse(data)
print d
if __name__ == '__main__'
main()
I got an Error 401 in the feed that was captured. This is what was captured:
Am I missing something? I am not from a CS background so whatever I know is what I've read around. I intend to use the Gmail feeds captured to check the number of unread messages and display them using an Arduino.
I had no luck with HTTPDigestAuthHandler, but was able to get it working with HTTPBasicAuthHandler.
import urllib2, feedparser
pwdmgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
pwdmgr.add_password("New mail feed", 'http://mail.google.com/', username, password)
auth = urllib2.HTTPBasicAuthHandler(pwdmgr)
opener = urllib2.build_opener(auth)
data = opener.open('http://mail.google.com/mail/feed/atom')
d = feedparser.parse(data)
print d
Related
I am using python module of sendgrid (https://github.com/sendgrid/sendgrid-python) to send transactional emails. I need help with the syntax to send the email at a scheduled time.
The general sendgrid documentation asks to modify json as "{ “send_at”: 1409348513 }". I believe this json is not directly accessible in sendgrid-python. I need syntax for doing equivalent thing with the python library.
My current code is equivalent to the one copied below. It would be great if some one can suggest how to modify this code to schedule it at a particular time e.g. datetime.dateime.now() + datetime.timedelta(days=1)
import sendgrid
from sendgrid.helpers.mail import Email, Content, Substitution, Mail
import urllib2 as urllib
def send_email_custom():
sg = sendgrid.SendGridAPIClient(apikey=myApiKey)
from_email = Email(sendEmail)
to_email = Email(custEmail)
reply_to_email = Email(receiveEmail)
content = Content("text/html", "Introduction")
mail = Mail(from_email, subject="Hi!", to_email=to_email, content=content)
mail.personalizations[0].add_substitution(Substitution("_firstName_", firstName))
mail.set_template_id(templateId)
try:
response = sg.client.mail.send.post(request_body=mail.get())
except urllib.HTTPError as e:
print(e.read())
return False
if response.status_code >=200 and response.status_code < 300:
return True
else:
return False
send_at is a component of the personalizations Object, so you can define it at that level, allowing you to set distinct sending times for each recipient/personalization set.
If you don't need that, you can also set it at the top mail level.
Looks like you should be able to do that by following this example. You can specify all of the raw fields you need.
Exact code solution:
mail = Mail(from_email, subject="Hi!", to_email=to_email, content=content)
mail.send_at = SendAt(1461775053) # Time is specified in UNIX form.
# This takes advantage of Sendgrid's inbuilt email scheduler
# but you can't schedule more than 72 hours in advance.
This removes the need to deal with personalizations for send time.
Source: line 308 of https://github.com/sendgrid/sendgrid-python/blob/main/examples/helpers/mail_example.py
Posting this for those who Googled their way here in 2022 or later.
I have tried requests, pydap, urllib, and netcdf4 and keep either getting redirect errors or permission errors when trying to download the following NASA data:
GLDAS_NOAH025SUBP_3H: GLDAS Noah Land Surface Model L4 3 Hourly 0.25 x 0.25 degree Subsetted V001 (http://disc.sci.gsfc.nasa.gov/uui/datasets/GLDAS_NOAH025SUBP_3H_V001/summary?keywords=Hydrology)
I am attempting to download about 50k files, here is an example of one, which works when pasted into google chrome browser (if you have proper username and password):
http://hydro1.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FGLDAS_V1%2FGLDAS_NOAH025SUBP_3H%2F2016%2F244%2FGLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016256190725.grb&FORMAT=TmV0Q0RGLw&BBOX=-11.95%2C28.86%2C-0.62%2C40.81&LABEL=GLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016286201048.pss.nc&SHORTNAME=GLDAS_NOAH025SUBP_3H&SERVICE=SUBSET_GRIB&VERSION=1.02&LAYERS=AAAB&DATASET_VERSION=001
Anyone have any experience getting OPeNDAP NASA data from the web using python? I am happy to provide more information if desired.
Here is the requests attempt which gives 401 error:
import requests
def httpdownload():
'''loop through each line in the text file and open url'''
httpfile = open(pathlist[0]+"NASAdownloadSample.txt", "r")
for line in httpfile:
print line
outname = line[-134:-122]+".hdf"
print outname
username = ""
password = "*"
r = requests.get(line, auth=("username", "password"), stream=True)
print r.text
print r.status_code
with open(pathlist[0]+outname, 'wb') as out:
out.write(r.content)
print outname, "finished" # keep track of progress
And here is the pydap example which gives redirect error:
import install_cas_client
from pydap.client import open_url
def httpdownload():
'''loop through each line in the text file and open url'''
username = ""
password = ""
httpfile = open(pathlist[0]+"NASAdownloadSample.txt", "r")
fileone = httpfile.readline()
filetot = fileone[:7]+username+":"+password+"#"+fileone[7:]
print filetot
dataset = open_url(filetot)
I did not find a solution using python, but given the information I have now it should be possible. I used wget with a .netrc file and cookie file shown as follows (https://disc.gsfc.nasa.gov/information/howto?title=How%20to%20Download%20Data%20Files%20from%20HTTP%20Service%20with%20wget):
#!/bin/bash
cd # path to output files
touch .netrc
echo "machine urs.earthdata.nasa.gov login <username> password <password>" >> .netrc
chmod 0600 .netrc
touch .urs_cookies
wget --content-disposition --trust-server-names --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies
-i <path to text file of url list>
Hope it helps anyone else working with NASA data from this server.
I realize it's a bit late to answer this question for the original poster, but I stumbled across this question while trying to do the same thing so I'll leave my solution here. It seems the NASA server uses redirects and Basic Authorization in a way the standard libraries don't expect. When you download from (for example) https://hydro1.gesdisc.eosdis.nasa.gov, you'll get redirected to https://urs.earthdata.nasa.gov for authentication. That server sets an authentication token as a cookie and redirects you back to download the file. If you're not handling cookies properly, you'll be stuck in an infinite redirection loop. If you're not handling authentication and redirection properly, you'll get an access denied error.
To get around this problem, chain HTTPRedirectHandler, HTTPCookieProcessor, and HTTPPasswordMgrWithDefaultRealm together and set it as the default opener or just use that opener directly.
from urllib import request
username = "<your username>"
password = "<your password>"
url = "<remote url of file>"
filename = "<local destination of file>"
redirectHandler = request.HTTPRedirectHandler()
cookieProcessor = request.HTTPCookieProcessor()
passwordManager = request.HTTPPasswordMgrWithDefaultRealm()
passwordManager.add_password(None, "https://urs.earthdata.nasa.gov", username, password)
authHandler = request.HTTPBasicAuthHandler(passwordManager)
opener = request.build_opener(redirectHandler,cookieProcessor,authHandler)
request.install_opener(opener)
request.urlretrieve(url,filename)
I want to send transactional and marketing emails using 'SendInBlue'. I also want to use Python language to do the same. I have visited the API doc of SendInBlue and followed the same procedure, still unsuccessful in sending the emails.
from mailin import Mailin
m = Mailin("https://api.sendinblue.com/v2.0","ScrWGqd296ya0CWq")
data = { "to" : {"aman#gmail.com":"to whom!"},
"from" : ["amandeep#gmail.com", "from email!"],
"subject" : "Subject...",
"html" : "This is the <h1>HTML</h1>",
"attachment" : ["https://example.com/path-to-file/filename1.pdf", "https://example.com/path-to-file/filename2.jpg"]
}
result = m.send_email(data)
print(result)
I have also downloaded mailin-api-python from github and ran this script. I don't have any idea where to setup to my smtp details.
**I have changed the API key just for security purpose.
I would strongly recommend you to use the latest version of SendinBlue's Python wrapper where they have provided an example
from __future__ import print_function
import time
import sib_api_v3_sdk
from sib_api_v3_sdk.rest import ApiException
from pprint import pprint
# Configure API key authorization: api-key
configuration = sib_api_v3_sdk.Configuration()
configuration.api_key['api-key'] = 'API-KEY'
# create an instance of the API class
api_instance = sib_api_v3_sdk.SMTPApi(sib_api_v3_sdk.ApiClient(configuration))
senderSmtp = sib_api_v3_sdk.SendSmtpEmailSender(name="test",email="youremail#gmail.com")
sendTo = sib_api_v3_sdk.SendSmtpEmailTo(email="recipientEmail#gmail.com",name="Recipient Name")
arrTo = [sendTo] #Adding `to` in a list
send_smtp_email = sib_api_v3_sdk.SendSmtpEmail(sender=senderSmtp,to=arrTo,html_content="This is a test",subject="This is a test subject") # SendSmtpEmail | Values to send a transactional email
try:
# Send a transactional email
api_response = api_instance.send_transac_email(send_smtp_email)
pprint(api_response)
except ApiException as e:
print("Exception when calling SMTPApi->send_transac_email: %s\n" % e)
I got a sample script working:
I successfully received the email and the messageId as the response.
Please let me know if it helps!
I'm writing a script to receive emails from my gmail email in python. I'm managing to download the raw email however I am then unable to access certain types of it, E.G BODY, TO, FROM etc.
import imaplib, email
msrvr = imaplib.IMAP4_SSL('imap.gmail.com', 993)
unm = 'stackoverflow#gmail.com'
pwd = 'lovetocode'
msrvr.login(unm,pwd)
stat,cnt = msrvr.select('Inbox')
stat, dta = msrvr.fetch(cnt[0], '(RFC822)')
b = email.message_from_string(str(dta))
print(b)
print(b['[To]'])
msrvr.close()
msrvr.logout()
Where am I going wrong?
You might find it easier to use native Python Google SDK's for working with their email:
https://developers.google.com/appengine/docs/python/mail/
The imaplib module you are using is will only give you a subset of all gmail features..
Here's some code that parses an email and prints some header fields:
msg = email.message_from_string(raw_email)
for field in ('From', 'Subject', 'Received', 'Message-ID'):
print '{0}: {1}'.format(field, msg[field])
For debugging, also print the raw parts of the Message object:
print msg.__dict__
(Note: I'm using Python2.7, but I believe there's not much difference.)
I am trying to post to the wall of a facebook page that I am administrator (not profile), however no luck. How do I achieve this ? I'm stucked at the page access token retrieval part.
#!/usr/bin/python
# coding: utf-8
import facebook
import urllib
import urlparse
import subprocess
import warnings
# Hide deprecation warnings. The facebook module isn't that up-to-date (facebook.GraphAPIError).
warnings.filterwarnings('ignore', category=DeprecationWarning)
# Parameters of your app and the id of the profile you want to mess with.
FACEBOOK_APP_ID = 'XXXXXXXXXXXXXX'
FACEBOOK_APP_SECRET = 'XXXXXXXXXXXXXXXXXXXXX'
FACEBOOK_PROFILE_ID = 'XXXXXXXXXXX'
# Trying to get an access token. Very awkward.
oauth_args = dict(client_id = FACEBOOK_APP_ID,
client_secret = FACEBOOK_APP_SECRET,
scope = 'manage_pages',
response_type = 'token'
)
oauth_curl_cmd = ['curl',
'https://graph.facebook.com/oauth/access_token?' + urllib.urlencode(oauth_args)]
oauth_response = subprocess.Popen(oauth_curl_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE).communicate()[0]
print urllib.urlencode(oauth_args)
try:
oauth_access_token = urlparse.parse_qs(str(oauth_response))['access_token'][0]
except KeyError:
print('Unable to grab an access token!')
exit()
print oauth_access_token
facebook_graph = facebook.GraphAPI(oauth_access_token)
# Try to post something on the wall.
try:
fb_response = facebook_graph.put_wall_post('Hello from Python', \
profile_id = FACEBOOK_PROFILE_ID)
print fb_response
except facebook.GraphAPIError as e:
print 'Something went wrong:', e.type, e.message
I would not recommend doing this through the command line with curl as it is less secure and less reliable. You can do all of this with the urllib2 and json modules
to get the access token you just want to make a call to https://graph.facebook.com/oauth/access_token?client_id=YOUR_APP_ID&client_secret=YOUR_APP_SECRET&grant_type=client_credentials
so you would do:
url='https://graph.facebook.com/oauth/access_token?client_id=YOUR_APP_ID&client_secret=YOUR_APP_SECRET&grant_type=client_credentials'
target=urllib2.urlopen(url)
token = target.read()[13:]
EDIT:
My bad, I forgot that facebook/oauth gives you the access token in plain text so you don't need the json module. I've updated the example to show what you should be doing. Note target.read() will give you the string 'access_token=ACCESS_TOKEN' and then you are just parsing it to remove the identifier.
to see what response is go to the url and put in your information you will a json dict with acess_token.
the second half of this page should have all the information you need.