Since Microsoft has now dropped support for accessing an Exchange mailbox using basic authentication, I've had to upgrade some code to use oauth based access to the mailbox instead.
I've setup an Azure AD app following these docs:
https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
Now I'd like to access emails from the mailbox using exchangelib and exchangelib.OAuth2Credentials
Here's the code I'm using:
from exchangelib import Account, OAuth2Credentials
AZURE_CLIENT_ID = "MY_CLIENT_ID"
AZURE_TENANT_ID = "MY_TENANT_ID"
AZURE_CLIENT_CREDENTIAL = "MY_CLIENT_CREDENTIAL"
MY_EMAIL = "me#example.com"
credentials = OAuth2Credentials(
client_id=AZURE_CLIENT_ID, client_secret=AZURE_CLIENT_CREDENTIAL, tenant_id=AZURE_TENANT_ID
)
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True)
for item in account.inbox.all().order_by('-datetime_received')[:100]:
print(item.subject, item.sender, item.datetime_received)
However I'm getting an error saying The SMTP address has no mailbox associated with it:
Traceback (most recent call last):
File "exchangelib_test.py", line 19, in <module>
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/account.py", line 119, in __init__
email=primary_smtp_address, credentials=credentials, auth_type=auth_type, retry_policy=retry_policy
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 114, in discover
ad_response = self._quick(protocol=ad_protocol)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 201, in _quick
return self._step_5(ad=ad)
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/discovery.py", line 531, in _step_5
ad.raise_errors()
File "/home/me/.local/lib/python3.6/site-packages/exchangelib/autodiscover/properties.py", line 313, in raise_errors
raise ErrorNonExistentMailbox('The SMTP address has no mailbox associated with it')
exchangelib.errors.ErrorNonExistentMailbox: The SMTP address has no mailbox associated with it
Does anyone have an idea what might be going wrong here? The SMTP address definitely does exist and is an Exchange mailbox which I can log in to and view via Outlook.
If your using the client credentials flow then you will also need to use Impersonation access_type=IMPERSONATION eg
account = Account(primary_smtp_address=MY_EMAIL, credentials=credentials, autodiscover=True, access_type=IMPERSONATION)
generally with Office365 you don't want to use Autodiscover as the endpoint is always outlook.office365.com there is a sample of bypassing that in the samples page https://ecederstrand.github.io/exchangelib/ eg
config = Configuration(server='outlook.office365.com',
credentials=credentials)
account = Account(primary_smtp_address='john#example.com', config=config,
autodiscover=False, access_type=IMPERSONATION)
Related
I'm working on a github application and my actual task is to retrieve list of github user account / github organization on which the app has been installed.
So basically, I'll be using the app_installations() Method in Github3.py Library.
This method require authentication. So by running the following code :
def get_users():
user = "bilel.e***********"
password = "5QPN74dHD******"
gh = login(user, password)
#gh is an instance of <class 'github3.github.GitHub'>
scopes = ['user', 'repo']
auth = gh.authorize(user, password, scopes)
ghi = gh.app_installations()
return ()
get_users()
I receive the following output :
Traceback (most recent call last):
File "ghe-admin-app/service.py", line 33, in <module>
get_users()
File "ghe-admin-app/service.py", line 29, in get_users
auth = gh.authorize(user, password, scopes)
File "/usr/local/lib/python3.7/dist-packages/github3/github.py", line 503, in authorize
json = self._json(self._post(url, data=data), 201)
File "/usr/local/lib/python3.7/dist-packages/github3/models.py", line 156, in _json
raise exceptions.error_for(response)
github3.exceptions.UnprocessableEntity: 422 Validation Failed
I should mention that I tried to put a wrong password on purpose and the result was as expected :
github3.exceptions.AuthenticationFailed: 401 Bad credentials
Fixed issue !
I was using the wrong Authentication method.
I should use login_as_app(private_key_pem=key_file_pem, app_id=app_id) instead in order to use the app_installations() method.
I enabled the Gmail API and downloaded the .json file containing the token. I placed it in the same folder as the script. When I try to run it, I get this error:
Traceback (most recent call last):
File "email_clean.py", line 14, in <module>
creds = store.get()
File "C:\Python27\lib\site-packages\oauth2client\client.py", line 407, in get
return self.locked_get()
File "C:\Python27\lib\site-packages\oauth2client\file.py", line 54, in locked_get
credentials = client.Credentials.new_from_json(content)
File "C:\Python27\lib\site-packages\oauth2client\client.py", line 302, in new_from_json
module_name = data['_module']
KeyError: '_module'
I am aware that there are several other questions on SO for this same issue, but those solutions did not help me. The token.json file is in the same folder as the script, and the URL appears to be spelled correctly as best I can tell. The JSON file looks correctly formatted. Any help would be greatly appreciated.
Here is the script:
"""
Shows basic usage of the Gmail API.
Lists the user's Gmail labels.
"""
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Gmail API
SCOPES = 'https://www.googleapis.com/auth/gmail.readonly'
store = file.Storage('token.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('credentials.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('gmail', 'v1', http=creds.authorize(Http()))
# Call the Gmail API
results = service.users().labels().list(userId='me').execute()
labels = results.get('labels', [])
if not labels:
print('No labels found.')
else:
print('Labels:')
for label in labels:
print(label['name'])
It's not finding / reading your token.json file. (You can look at that file and see it should have the first line read something like:
{"_module": "oauth2client.client",
...
Simple fix (assuming your file has the correct info) is to pass the full absolute path name to file.Storage(), rather than just the file name.
In your case, you say your json file starts with
{"installed": {"client_id": ...
Thats's the your client secret json file, not the credentials json -- you'll need to use the former to get the latter. That file (with "installed") is the one to be used with your call to client.flow_from_clientsecrets() . On success, that will write the file as indicated by file.Storage(), and on subsequent attempts, the flow will look in the token file and use information in there (rather than clientsecrets).
A fix may be as simple as making sure your token.json file does not exist on first attempt as then it will attempt execute the flow & re-create the file.
A really good explanation of the flow is :Ashok Yogi's A beginners guide to Google OAuth and Google APIs
Figured it out. The problem was that I had saved the .json file as token.json, not realizing that this is a file that gets created once it authenticates. The trick was to delete token.json and then save the .json file from the Developer's Console as credentials.json.
In GAE standard environment, I'm struggling with registering the watch() call against Gmail API for the Pub/Sub push notification using google-api-python-client.
Here is the relevant excerpt from my code:
import googleapiclient.discovery
from oauth2client import service_account
SCOPES = ['https://www.googleapis.com/auth/gmail.modify']
SERVICE_ACCOUNT_FILE = '<My-project>-<short-ID>.json'
credentials = service_account.ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
gmail = googleapiclient.discovery.build('gmail', 'v1', credentials=credentials)
watchRequest = {
'labelIds' : ['INBOX'],
'topicName' : 'projects/<my-project>/topics/<my-topic>'
}
gmail.users().watch(userId='<email-I-need-to-watch>', body=watchRequest).execute()
After firing-off this part of the code I get:
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27/54c5883f70296ec8_unzipped/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/e~<my-project>-191008/20180124t154459.407164278206739455/main.py", line 68, in <module>
gmail.users().watch(userId='<email-I-need-to-watch>', body=watchRequest).execute()
File "/base/data/home/apps/e~<my-project>/20180124t154459.407164278206739455/lib/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/base/data/home/apps/e~<my-project>/20180124t154459.407164278206739455/lib/googleapiclient/http.py", line 844, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 400 when requesting https://www.googleapis.com/gmail/v1/users/<email-I-need-to-watch>/watch?alt=json returned "Bad Request">
In regards to the authentication and authorization, here is what I have done so far:
I've created a Pub/Sub topic and this is the one I'm passing into the watch() request
I use G-Suite and the email inbox I intend to watch is part of my G-Suite business domain.
For this task I use a service account with enabled G-Suite Domain-wide Delegation. I've downloaded the .json service account file which I'm supplying in order to acquire the oauth2client.service_account.Credentials object (I see the access and refresh tokens being exchanged successfully in the logs). The json service file is placed in the same folder as my main.py script (root of my project).
In my G-Suite administration panel I've enabled the api access to the service account from 2. with the scope of https://www.googleapis.com/auth/gmail.modify. I'm using the gmail.modify access level as I intend to read, write and send both emails and drafts.
Is there something I'm missing out in my code or in the authentication and authorization steps?
Problem solved. I was missing the part of the code for impersonating a user from my domain in order to read his/her mailbox (as explained here).
The corrected code looks like this:
import googleapiclient.discovery
from google.oauth2 import service_account
SCOPES = ['https://www.googleapis.com/auth/gmail.modify']
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES
)
credentials = credentials.with_subject('<email-I-need-to-watch>')
gmail = googleapiclient.discovery.build('gmail', 'v1', credentials=credentials)
watchRequest = {
'labelIds' : ['INBOX'],
'topicName' : 'projects/<my-project>/topics/<my-topic>'
}
gmail.users().watch(userId='me', body=watchRequest).execute()
I have a gmail account in my g-suite organization for which I want to automatically read its gmail messages.
Since this needs to be run periodically with an automated process, the regular OAuth flow is not useful, since someone needs to open a browser and give permission to the automated access.
So far I've created a service account with domain-wide authority, as documented here.
The code I am using is:
import httplib2
from apiclient import discovery
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', ]
CLIENT_SECRET_FILE = '/path/to/client_secrets.json'
def get_credentials():
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_json_keyfile_name(CLIENT_SECRET_FILE, SCOPES)
delegated_credentials = credentials.create_delegated('myproject#myproject-123456.iam.gserviceaccount.com')
return delegated_credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('gmail', 'v1', http=http)
results = service.users().messages().list(userId='user_to_impersonate#mycompany.com').execute()
if __name__ == '__main__':
main()
But I get a 400 Bad Request error:
Traceback (most recent call last):
File "gmail.py", line 77, in <module>
main()
File "gmail.py", line 65, in main
results = service.users().messages().list(userId='user_to_impersonate#mycompany.com').execute()
File "/usr/local/lib/python3.6/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 842, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/gmail/v1/users/user_to_impersonate%40mycompany.com/messages?alt=json returned "Bad Request">
Is it possible at all to access a specific gmail account without having to grant permission from the browser? Do I need to perform any special step first? Or, is there any way get more information for debugging?
Based on the docs, you need to use delegated_credentials = credentials.create_delegated('user_to_impersonate#mycompany.com') instead of delegated_credentials = credentials.create_delegated('myproject#myproject-123456.iam.gserviceaccount.com').
I'm trying to use OAuth2 with my App Engine application, but I keep getting the following error:
Encountered unexpected error from ProtoRPC method implementation: IOError ([Errno 30] Read-only file system: 'credentials.dat')
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/protorpc-1.0/protorpc/wsgi/service.py", line 181, in protorpc_service_app
response = method(instance, request)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/endpoints-1.0/endpoints/api_config.py", line 1332, in invoke_remote
return remote_method(service_instance, request)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/protorpc-1.0/protorpc/remote.py", line 412, in invoke_remote_method
response = method(service_instance, request)
File "/base/data/home/apps/s~art-everywhere/7.385356767964097452/main.py", line 438, in upload_putphoto
gd_client = PicasaWA.login()
File "/base/data/home/apps/s~art-everywhere/7.385356767964097452/main.py", line 1653, in login
storage.put(credentials)
File "/base/data/home/apps/s~art-everywhere/7.385356767964097452/oauth2client/client.py", line 325, in put
self.locked_put(credentials)
File "/base/data/home/apps/s~art-everywhere/7.385356767964097452/oauth2client/file.py", line 113, in locked_put
f = open(self._filename, 'wb')
IOError: [Errno 30] Read-only file system: 'credentials.dat'
It looks like there's a problem with the write used in the library.
Here's a snippet of the code I used:
def login(cls):
scope = 'https://picasaweb.google.com/data/'
user_agent = 'picasawebuploader'
# credential_store = os.path.join(os.path.split(__file__)[0], "credentials.dat")
storage = Storage("credentials.dat")
# storage = Storage(credential_store)
credentials = storage.get()
# user = users.get_current_user()
# storage = StorageByKeyName(CredentialsModel, user.user_id(), 'credentials')
# credentials = storage.get()
if credentials is None or credentials.invalid:
flow = flow_from_clientsecrets("client_secrets.json", scope=scope, redirect_uri='urn:ietf:wg:oauth:2.0:oob')
uri = flow.step1_get_authorize_url()
logging.info("uri: %s", uri)
webbrowser.open(uri)
code = "Here I posted the code retrived by the autentication"
credentials = flow.step2_exchange(code)
storage.put(credentials)
if (credentials.token_expiry - datetime.utcnow()) < timedelta(minutes=5):
http = httplib2.Http()
http = credentials.authorize(http)
credentials.refresh(http)
gd_client = gdata.photos.service.PhotosService(source=user_agent,
email=USERNAME,
additional_headers={'Authorization' : 'Bearer %s' % credentials.access_token})
return gd_client
What am I doing wrong?
Thanks everyone for the help!
You can't create files in AppEngine like you would on a regular filesystem, it's read-only. You need to use special AppEngine credentials and storage objects.
Ran into the same issue running the devserver on my local machine. The solution was not pretty, but this change solved it for me. I'm not sure if you'd be able run storage.put(credentials) from within appengine. This might be where a lot of the issue came from. Doing the oauth flow as part of a separate script, and then loading the credentials from the file actually worked for me after modifying the devserver.
I highly doubt that this would actually work in production though, so you might want to look into Delegating domain-wide authority to the service account