We Keep Coding

Accessing Exchange inbox using exchangelib from Python with oauth

Since Microsoft has now dropped support for accessing an Exchange mailbox using basic authentication, I've had to upgrade some code to use oauth based access to the mailbox instead.
I've setup an Azure AD app following these docs:
https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
Now I'd like to access emails from the mailbox using exchangelib and exchangelib.OAuth2Credentials
Here's the code I'm using:
from exchangelib import Account, OAuth2Credentials
AZURE_CLIENT_ID = "MY_CLIENT_ID"
AZURE_TENANT_ID = "MY_TENANT_ID"
AZURE_CLIENT_CREDENTIAL = "MY_CLIENT_CREDENTIAL"
MY_EMAIL = "me#example.com"
credentials = OAuth2Credentials(
client_id=AZURE_CLIENT_ID, client_secret=AZURE_CLIENT_CREDENTIAL, tenant_id=AZURE_TENANT_ID
)
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True)
for item in account.inbox.all().order_by('-datetime_received')[:100]:
print(item.subject, item.sender, item.datetime_received)
However I'm getting an error saying The SMTP address has no mailbox associated with it:
Traceback (most recent call last):
File "exchangelib_test.py", line 19, in <module>
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/account.py", line 119, in __init__
email=primary_smtp_address, credentials=credentials, auth_type=auth_type, retry_policy=retry_policy
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 114, in discover
ad_response = self._quick(protocol=ad_protocol)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 201, in _quick
return self._step_5(ad=ad)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 531, in _step_5
ad.raise_errors()
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/properties.py", line 313, in raise_errors
raise ErrorNonExistentMailbox('The SMTP address has no mailbox associated with it')
exchangelib.errors.ErrorNonExistentMailbox: The SMTP address has no mailbox associated with it
Does anyone have an idea what might be going wrong here? The SMTP address definitely does exist and is an Exchange mailbox which I can log in to and view via Outlook.

If your using the client credentials flow then you will also need to use Impersonation access_type=IMPERSONATION eg
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True, access_type=IMPERSONATION)
generally with Office365 you don't want to use Autodiscover as the endpoint is always outlook.office365.com there is a sample of bypassing that in the samples page https://ecederstrand.github.io/exchangelib/ eg
config = Configuration(server='outlook.office365.com',
credentials=credentials)
account = Account(primary_smtp_address='john#example.com', config=config,
autodiscover=False, access_type=IMPERSONATION)

Can't have authorization for the app_installations methode

I'm working on a github application and my actual task is to retrieve list of github user account / github organization on which the app has been installed.
So basically, I'll be using the app_installations() Method in Github3.py Library.
This method require authentication. So by running the following code :
def get_users():
user = "bilel.e***********"
password = "5QPN74dHD******"
gh = login(user, password)
#gh is an instance of <class 'github3.github.GitHub'>
scopes = ['user', 'repo']
auth = gh.authorize(user, password, scopes)
ghi = gh.app_installations()
return ()
get_users()
I receive the following output :
Traceback (most recent call last):
File "ghe-admin-app/service.py", line 33, in <module>
get_users()
File "ghe-admin-app/service.py", line 29, in get_users
auth = gh.authorize(user, password, scopes)
File "/usr/local/lib/python3.7/dist-packages/github3/github.py", line 503, in authorize
json = self._json(self._post(url, data=data), 201)
File "/usr/local/lib/python3.7/dist-packages/github3/models.py", line 156, in _json
raise exceptions.error_for(response)
github3.exceptions.UnprocessableEntity: 422 Validation Failed
I should mention that I tried to put a wrong password on purpose and the result was as expected :
github3.exceptions.AuthenticationFailed: 401 Bad credentials

Fixed issue !
I was using the wrong Authentication method.
I should use login_as_app(private_key_pem=key_file_pem, app_id=app_id) instead in order to use the app_installations() method.

Working with Gmail API from Google AppEngine

In GAE standard environment, I'm struggling with registering the watch() call against Gmail API for the Pub/Sub push notification using google-api-python-client.
Here is the relevant excerpt from my code:
import googleapiclient.discovery
from oauth2client import service_account
SCOPES = ['https://www.googleapis.com/auth/gmail.modify']
SERVICE_ACCOUNT_FILE = '<My-project>-<short-ID>.json'
credentials = service_account.ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
gmail = googleapiclient.discovery.build('gmail', 'v1', credentials=credentials)
watchRequest = {
'labelIds' : ['INBOX'],
'topicName' : 'projects/<my-project>/topics/<my-topic>'
}
gmail.users().watch(userId='<email-I-need-to-watch>', body=watchRequest).execute()
After firing-off this part of the code I get:
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/e~<my-project>-191008/20180124t154459.407164278206739455/main.py", line 68, in <module>
gmail.users().watch(userId='<email-I-need-to-watch>', body=watchRequest).execute()
File "/base/data/home/apps/e~<my-project>/20180124t154459.407164278206739455/lib/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/base/data/home/apps/e~<my-project>/20180124t154459.407164278206739455/lib/googleapiclient/http.py", line 844, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 400 when requesting https://www.googleapis.com/gmail/v1/users/<email-I-need-to-watch>/watch?alt=json returned "Bad Request">
In regards to the authentication and authorization, here is what I have done so far:
I've created a Pub/Sub topic and this is the one I'm passing into the watch() request
I use G-Suite and the email inbox I intend to watch is part of my G-Suite business domain.
For this task I use a service account with enabled G-Suite Domain-wide Delegation. I've downloaded the .json service account file which I'm supplying in order to acquire the oauth2client.service_account.Credentials object (I see the access and refresh tokens being exchanged successfully in the logs). The json service file is placed in the same folder as my main.py script (root of my project).
In my G-Suite administration panel I've enabled the api access to the service account from 2. with the scope of https://www.googleapis.com/auth/gmail.modify. I'm using the gmail.modify access level as I intend to read, write and send both emails and drafts.
Is there something I'm missing out in my code or in the authentication and authorization steps?

Problem solved. I was missing the part of the code for impersonating a user from my domain in order to read his/her mailbox (as explained here).
The corrected code looks like this:
import googleapiclient.discovery
from google.oauth2 import service_account
SCOPES = ['https://www.googleapis.com/auth/gmail.modify']
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES
)
credentials = credentials.with_subject('<email-I-need-to-watch>')
gmail = googleapiclient.discovery.build('gmail', 'v1', credentials=credentials)
watchRequest = {
'labelIds' : ['INBOX'],
'topicName' : 'projects/<my-project>/topics/<my-topic>'
}
gmail.users().watch(userId='me', body=watchRequest).execute()

Accessing the gmail API with delegated domain-wide authority

I have a gmail account in my g-suite organization for which I want to automatically read its gmail messages.
Since this needs to be run periodically with an automated process, the regular OAuth flow is not useful, since someone needs to open a browser and give permission to the automated access.
So far I've created a service account with domain-wide authority, as documented here.
The code I am using is:
import httplib2
from apiclient import discovery
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', ]
CLIENT_SECRET_FILE = '/path/to/client_secrets.json'
def get_credentials():
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_json_keyfile_name(CLIENT_SECRET_FILE, SCOPES)
delegated_credentials = credentials.create_delegated('myproject#myproject-123456.iam.gserviceaccount.com')
return delegated_credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('gmail', 'v1', http=http)
results = service.users().messages().list(userId='user_to_impersonate#mycompany.com').execute()
if __name__ == '__main__':
main()
But I get a 400 Bad Request error:
Traceback (most recent call last):
File "gmail.py", line 77, in <module>
main()
File "gmail.py", line 65, in main
results = service.users().messages().list(userId='user_to_impersonate#mycompany.com').execute()
File "/usr/local/lib/python3.6/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 842, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/gmail/v1/users/user_to_impersonate%40mycompany.com/messages?alt=json returned "Bad Request">
Is it possible at all to access a specific gmail account without having to grant permission from the browser? Do I need to perform any special step first? Or, is there any way get more information for debugging?

Based on the docs, you need to use delegated_credentials = credentials.create_delegated('user_to_impersonate#mycompany.com') instead of delegated_credentials = credentials.create_delegated('myproject#myproject-123456.iam.gserviceaccount.com').

Using Google Picasa API with Python

I have been using Google's Picasa API successfully from last 6 months or so.
Today I have started getting an error
raise GooglePhotosException(e.args[0])
GooglePhotosException: (403, 'Forbidden', 'Authorization required')
I checked my credentials.
self.gd_client = gdata.photos.service.PhotosService()
self.gd_client.email = EmailOfTheUploadingPictureAccount
self.gd_client.password = PasswordOfTheAccount
self.gd_client.source = 'destipak' #Not sure about that
self.feed_url = "/data/feed/api/user/"
self.entry_url = "/data/entry/api/user/"
self.gd_client.ProgrammaticLogin()
Everything was working well since yesterday. Anyone has any clues?
EDIT
Example given on Picasa for python is also not working.
URL
#!/usr/bin/python2.5
import gdata.photos.service
import gdata.media
import gdata.geo
gd_client = gdata.photos.service.PhotosService()
gd_client.email = '=change=' # Set your Picasaweb e-mail address...
gd_client.password = '=change=' # ... and password
gd_client.source = 'api-sample-google-com'
gd_client.ProgrammaticLogin()
albums = gd_client.GetUserFeed()
for album in albums.entry:
print 'Album: %s (%s)' % (album.title.text, album.numphotos.text)
photos = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s?kind=photo' % (album.gphoto_id.text))
for photo in photos.entry:
print ' Photo:', photo.title.text
tags = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s/photoid/%s?kind=tag' % (album.gphoto_id.text, photo.gphoto_id.text))
for tag in tags.entry:
print ' Tag:', tag.title.text
comments = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s/photoid/%s?kind=comment' % (album.gphoto_id.text, photo.gphoto_id.text))
for comment in comments.entry:
print ' Comment:', comment.content.text
EDIT 2
Full traceback
Traceback (most recent call last):
File "/Users/mac/Picasa_API.py", line 158, in <module>
check_api()
File "/Users/mac/Picasa_API.py", line 140, in check_api
albums = gd_client.GetUserFeed()
File "/Users/mac/destipak/env/lib/python2.7/site-packages/gdata/photos/service.py", line 235, in GetUserFeed
return self.GetFeed(uri, limit=limit)
File "/Users/mac/destipak/env/lib/python2.7/site-packages/gdata/photos/service.py", line 180, in GetFeed
raise GooglePhotosException(e.args[0])
gdata.photos.service.GooglePhotosException: (403, 'Forbidden', 'Authorization required')

Here is the code I use to get OAuth2 authentication working with Picasa. First you need to create a client ID through the Google Developer Console: at https://console.developers.google.com/ and then you must download the client secrets as JSON and pass the filename to OAuth2Login.
The first time you run this code, you will have to authorize the client through your web browser, and paste the code you get there into the application. The credentials are then stored in the file specified by credential_store.
def OAuth2Login(client_secrets, credential_store, email):
scope='https://picasaweb.google.com/data/'
user_agent='myapp'
storage = Storage(credential_store)
credentials = storage.get()
if credentials is None or credentials.invalid:
flow = flow_from_clientsecrets(client_secrets, scope=scope, redirect_uri='urn:ietf:wg:oauth:2.0:oob')
uri = flow.step1_get_authorize_url()
webbrowser.open(uri)
code = raw_input('Enter the authentication code: ').strip()
credentials = flow.step2_exchange(code)
storage.put(credentials)
if (credentials.token_expiry - datetime.utcnow()) < timedelta(minutes=5):
http = httplib2.Http()
http = credentials.authorize(http)
credentials.refresh(http)
gd_client = gdata.photos.service.PhotosService(source=user_agent,
email=email,
additional_headers={'Authorization' : 'Bearer %s' % credentials.access_token})
return gd_client

I ran into this as well. Seems like email/password authentication has been turned off, and we need to switch to OAuth2. See
https://groups.google.com/forum/#!topic/Google-Picasa-Data-API/4meiAJ40l3E