We Keep Coding

Use of token code when accessing OneDrive using Python

I am writing some code to move files over to OneDrive (enterprise account). My app is authenticated in Azure AD and should have the correct accesses (Files.ReadWrite.All in MS Graph, Sites.ReadWrite.All in Office365 SPO and User.Read in Azure AD).
The code to receive the app token works fine:
import msal
client_id = 'dc185bb*************6bcda94'
authority_host_uri = 'https://login.microsoftonline.com'
discovery_uri = 'https://api.office.com/discovery/'
client_secret = 'VsY7vV**************ToiA0='
tenant = '4a6*********************65079'
authority_uri = authority_host_uri + '/' + tenant
scopes=['https://graph.microsoft.com/.default']
app = msal.ConfidentialClientApplication(
client_id=client_id, authority=authority_uri,
client_credential=client_secret)
result = app.acquire_token_for_client(scopes=scopes)
print(result)
However, when I try to use this token with the OneDrive SDK library it seems like I am not able pass it through:
def __init__(self, http_provider, client_id=None, scopes=None, access_token=None, session_type=None, loop=None,
auth_server_url=None, auth_token_url=None):
"""Initialize the authentication provider for authenticating
requests sent to OneDrive
Args:
http_provider (:class:`HttpProviderBase<onedrivesdk.http_provider_base>`):
The HTTP provider to use for all auth requests
client_id (str): Defaults to None, the client id for your
application
scopes (list of str): Defaults to None, the scopes
that are required for your application
access_token (str): Defaults to None. Not used in this implementation.
The above is from the auth_provider.py part of the onedrivesdk, and clearly states the access_token is not used in the implementation.
Is there another way around this? Or other libraries to use?

You could try to use this Authentication of OneDrive for Business.
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = your_client_id
client_secret = your_client_secret
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
Upload an Item:
returned_item = client.item(drive='me', id='root').children['newfile.txt'].upload('./path_to_file.txt')
For more examples, you can refer to this link.

requests library with googleapiclient

Following is the code to access a google storage bucket using the httplib2 library
import json
from httplib2 import Http
from oauth2client.client import SignedJwtAssertionCredentials
from googleapiclient.discovery import build
from pprint import pprint
client_email = 'my.iam.gserviceaccount.com'
json_file = 'services.json'
cloud_storage_bucket = 'my_bucket'
files = 'reviews/reviews_myapp_201603.csv'
private_key = json.loads(open(json_file).read())['private_key']
credentials = SignedJwtAssertionCredentials(client_email,
private_key,'https://www.googleapis.com/auth/devstorage.read_only')
storage = build('storage', 'v1', http=credentials.authorize(Http()))
pprint(storage.objects().get(bucket=cloud_storage_bucket, object=files).execute())
Can someone tell me if I can make the http request using the Python Requests library here?
If yes, how?

Yes, you can use the HTTP header Authorization: Bearer <access_token> with requests or any library you want.
Service account
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(
'services.json',
scopes=['https://www.googleapis.com/auth/devstorage.read_only'],
)
# Copy access token
bearer_token = credentials.token
User account credentials
import json
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'test.json',
'https://www.googleapis.com/auth/devstorage.read_only'
)
# Construct cache path for oauth2 token
oauth2_cache_path = 'test-oauth2.json'
credentials = None
try:
# Try to load existing oauth2 token
with open(oauth2_cache_path, 'r') as f:
credentials = Credentials(**json.load(f))
except (OSError, IOError) as e:
pass
if not credentials or not credentials.valid:
credentials = flow.run_console()
with open(oauth2_cache_path, 'w+') as f:
f.write(json.dumps({
'token': credentials.token,
'refresh_token': credentials.refresh_token,
'token_uri': credentials.token_uri,
'client_id': credentials.client_id,
'client_secret': credentials.client_secret,
'scopes': credentials.scopes,
}))
# Copy access token
bearer_token = credentials.token
Use requests lib
import requests
# Send request
response = requests.get(
'https://www.googleapis.com/storage/v1/<endpoint>?access_token=%s'
% bearer_token)
# OR
response = requests.get(
'https://www.googleapis.com/storage/v1/<endpoint>',
headers={'Authorization': 'Bearer %s' % bearer_token})
Use googleapiclient lib
I recommend you to use build() method and not requests directly because the google library do some checks before sending your API call (like checking params, endpoint, auth and the method you use). This library also raise exceptions when error is detected.
from googleapiclient.discovery import build
storage = build('storage', 'v1', credentials=credentials)
print(storage.objects().get(bucket='bucket', object='file_path').execute())
More informations here : https://developers.google.com/identity/protocols/OAuth2WebServer#callinganapi (click on "HTTP/REST" tab)

I suggest to use the official Google Auth library which is already implementing Requests Library. See this link for more information.
Here is a code to try (given that you have a service account file with required permissions):
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
service_account_file = 'service_account.json'
scopes = ['https://www.googleapis.com/auth/devstorage.full_control']
credentials = service_account.Credentials.from_service_account_file(
service_account_file, scopes=scopes)
session = AuthorizedSession(credentials)
bucket_name = 'YOUR-BUCKET-NAME'
response = session.get(f'https://storage.googleapis.com/storage/v1/b/{bucket_name}')
print(response.json())

How do I call an API Gateway with Cognito credentials in Python

I've managed to setup an API Gateway secured with Cognito. The unauthenticated user role has an access policy that should grant it access to the gateway. I've also managed to use boto3 to retrieve an identity ID from the pool and obtain the associated open ID token, as well as the associated secret and access keys.
How do I now make a call to the gateway using these credentials? Is there a way to use boto3 to handle signing a request to a particular method on the API?

My code is based largely on the questioner's own answer, but I've tried to make it clearer where all the values come from.
import boto3
import requests
from requests_aws4auth import AWS4Auth
# Use 'pip install boto3 requests requests-aws4auth' to get these
region_name = 'ap-southeast-2' # or 'us-west-1' or whatever
# 12 decimal digits from your AWS login page
account_id = '123456789012'
# I've only found this in the sample code for other languages, e.g. JavaScript
# Services→Cognito→Manage Federated Identities→(your-id-pool)→Sample code
identity_pool_id = 'ap-southeast-2:fedcba98-7654-3210-1234-56789abcdef0'
# Create a new identity
boto3.setup_default_session(region_name = region_name)
identity_client = boto3.client('cognito-identity', region_name=region_name)
identity_response = identity_client.get_id(AccountId=account_id,
IdentityPoolId=identity_pool_id)
# We normally wouldn't log this, but to illustrate:
identity_id = identity_response['IdentityId']
print ('identity_id:', identity_id) # good idea not to log this
# Get the identity's credentials
credentials_response = identity_client.get_credentials_for_identity(IdentityId=identity_id)
credentials = credentials_response['Credentials']
access_key_id = credentials['AccessKeyId']
secret_key = credentials['SecretKey']
service = 'execute-api'
session_token = credentials['SessionToken']
expiration = credentials['Expiration']
# Again, we normally wouldn't log this:
print ('access_key_id', access_key_id)
print ('secret_key', secret_key)
print ('session_token', session_token)
print ('expiration', expiration)
# The access_key_id will look something like 'AKIABC123DE456FG7890', similar to
# Services→IAM→Users→(AWS_USER_NAME)→Security credentials→Access key ID
# Get the authorisation object
auth = AWS4Auth(access_key_id, secret_key, region_name, service,
session_token=session_token)
current_app['auth'] = auth
# Just an illustration again:
print ('auth: %(service)s(%(date)s) %(region)s:%(access_id)s' % auth.__dict__)
# We'll use that object to send a request to our app. This app doesn't
# exist in real life, though, so you'll need to edit the following quite
# heavily:
# Services→Cognito→Manage your User Pools→(your-user-pool)→Apps→App name
app_name = 'my-app-name'
api_path = 'dev/helloworld'
method = 'GET'
headers = {}
body = ''
url = 'https://%s.%s.%s.amazonaws.com/%s' % (app_name, service, region_name,
api_path)
response = requests.request(method, url, auth=auth, data=body, headers=headers)

The following code (and the requests-aws4auth library) did the job:
import boto3
import datetime
import json
from requests_aws4auth import AWS4Auth
import requests
boto3.setup_default_session(region_name='us-east-1')
identity = boto3.client('cognito-identity', region_name='us-east-1')
account_id='XXXXXXXXXXXXXXX'
identity_pool_id='us-east-1:YYY-YYYY-YYY-YY'
api_prefix='ZZZZZZZZZ'
response = identity.get_id(AccountId=account_id, IdentityPoolId=identity_pool_id)
identity_id = response['IdentityId']
print ("Identity ID: %s"%identity_id)
resp = identity.get_credentials_for_identity(IdentityId=identity_id)
secretKey = resp['Credentials']['SecretKey']
accessKey = resp['Credentials']['AccessKeyId']
sessionToken = resp['Credentials']['SessionToken']
expiration = resp['Credentials']['Expiration']
print ("\nSecret Key: %s"%(secretKey))
print ("\nAccess Key %s"%(accessKey))
print ("\nSession Token: %s"%(sessionToken))
print ("\nExpiration: %s"%(expiration))
method = 'GET'
headers = {}
body = ''
service = 'execute-api'
url = 'https://%s.execute-api.us-east-1.amazonaws.com/dev/helloworld' % api_prefix
region = 'us-east-1'
auth = AWS4Auth(accessKey, secretKey, region, service, session_token=sessionToken)
response = requests.request(method, url, auth=auth, data=body, headers=headers)
print(response.text)

Next code is working really well.
Hope to help:
from pprint import pprint
import requests
from pycognito import Cognito
USER_POOL_ID = 'eu-central-1_XXXXXXXXXXX'
CLIENT_ID = 'XXXXXXXXXXXX'
CLIENT_SECRET = 'XXXXXXXXXXX'
u = Cognito(USER_POOL_ID,CLIENT_ID, client_secret=CLIENT_SECRET, username='cognito user name')
u.authenticate('cognito user password')
id_token = u.id_token
headers = {'Authorization': 'Bearer ' + id_token}
api_url = 'https://XXXXXXXXXXX.execute-api.eu-central-1.amazonaws.com/stage/XXXXXXXXXXX'
r = requests.get(api_url, headers=headers)
pprint(dict(r.headers))
print(r.status_code)
print(r.text)

Here is an example from our public docs: http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Cognito creds are no different than any other temporary creds, and the signing process is also the same. If you want to move back to Python the example above should be good, or I would guess that there are third-party libraries out there to do the signature for you.

identity_pool_id how to get
If you have not federated pool which could give you "identity_pool_id" ,
execution code below will give you identity_pool_id
import boto3
boto3.setup_default_session(
aws_access_key_id='AKIAJ7TBC72BPWNEWIDQ',
aws_secret_access_key='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',
region_name='ap-southeast-2')
client = boto3.client('cognito-identity')
response = client.list_identity_pools(MaxResults=3,)
print("IdentityPoolId-- ", response)

python oauth2 client issues when trying to get authorization token

I am trying to use OAuth2 to get an authorization token using Python to a REST API. I am successful doing so using CURL but not with python. I am using the examples provided at the following docs:
https://requests-oauthlib.readthedocs.org/en/latest/oauth2_workflow.html
The following is my code:
#!/usr/bin/python
import requests
import requests_oauthlib
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
client_id = 'AAAAAA'
client_secret = 'BBBBBB'
client = BackendApplicationClient(client_id=client_id)
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url='https://example.com/as/token.oauth2', client_id=client_id, client_secret=client_secret)
print token
I am getting the following error:
oauthlib.oauth2.rfc6749.errors.InvalidClientError: (invalid_client) client_id value doesn't match HTTP Basic username value
This is a very basic API that only needs client_id and client_credentials to get an authorization token.
All information would be greatly appreciated.

The documentation specifies the following items:
client_id = r'your_client_id'
client_secret = r'your_client_secret'
redirect_uri = 'https://your.callback/uri'
By client key do you perhaps mean client key?
token = oauth.fetch_token(token_url='https://example.com/as/token.oauth2', client_id=client_id, client_secret=client_secret)
Try changing it to the above and give it a spin. using r'' for raw input instead and the token given.

I have found myself in a similar circumstance.
I am writing a Django app.
I was getting unauthorized_client and invalid_client exceptions.
In my case the post request in "Exchange the code" ("step 3" in the OAuth2 protocol) wasn't being formulated correctly.
Through much searching and trial and error I found it is possible to essentially customise the request. You can do this by specifying the optional arguments of auth, header and/or body.
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import WebApplicationClient, BackendApplicationClient
from requests.auth import HTTPBasicAuth
client_id = CLIENT_ID
client_secret = CLIENT_SECRET
authorization_base_url = AUTHORIZE_URI
token_url = TOKEN_URI
redirect_uri = REDIRECT_URI
auth = HTTPBasicAuth(client_id, client_secret)
scope = SCOPE
# Create the Authorization URI
# Not included here but store the state in a safe place for later
the_first_session = OAuth2Session(client_id=client_id, redirect_uri=redirect_uri, scope=scope)
authorization_url, state = the_first_session.authorization_url(authorization_base_url)
# Browse to the Authorization URI
# Login and Auth with the OAuth provider
# Now to respond to the callback
the_second_session = OAuth2Session(client_id, state=state)
body = 'grant_type=authorization_code&code=%s&redirect_uri=%s&scope=%s' % (request.GET.get('code'), redirect_uri, scope)
token = the_second_session.fetch_token(token_url, code=request.GET.get('code'), auth=auth, body=body)

how to set up Drive SDK so you dont need a success token each time you run .py

At the moment i am needing to get a "success code" each time i want to run my .py app. to access my googdrive files which is a pain.
I saw Ali Afshars great vid on https://developers.google.com/drive/search-parameters where he uses:
from auth import http to streamline this process.
Not sure what this function should contain... Can you point me in the right direction, so that i can set up my .py to do this automatically ...or at least only once.
many thanks
Dav-o
EDIT relevant current snip of code follows:
import logging
logging.basicConfig()
import httplib2
import pprint
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.client import *
from apiclient import errors
CLIENT_ID = "864350......ps.googleusercontent.com"
CLIENT_SECRET = "sw0yb.....-zR6XWzEgM"
OAUTH_SCOPE = 'https://www.googleapis.com/auth/drive'
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oo...ooo' # Redirect URI for installed apps
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
http = httplib2.Http()
http = credentials.authorize(http)
drive_service = build('drive', 'v2', http=http)

What he probably does in auth is:
import httplib2
from oauth2client.client import OAuth2Credentials
http = httplib2.Http()
credentials = OAuth2Credentials(access_token, client_id, client_secret, refresh_token, None, '', '')
credentials.authorize(http)
You can wrap this code in a module/method and use the authenticate your requests. For multiple user scenarios, preserve access and refresh tokens (most likely in a db) and init credentials object with the stored tokens.