So I have been using powershell for quite a while now and am somewhat familiar with getting a token from an Azure application using MS Graph configured with application API permissions. I am now attempting to perform the same in a python console application and am getting flummoxed as I constantly get a 400 error. here's the snippet of my code...
import requests
import json
app_id='<appid>'
client_secret='<client secret>'
token_url='https://login.microsoftonline.com/<tenant id>/oauth2 /v2.0/token'
token_data = {
'grant_type': 'client_credentials',
'client_id': app_id,
'client_secret': client_secret,
'resource': 'https://graph.microsoft.com',
'scope':'https://graph.microsoft.com/.default'
}
headers = {'content-type':'application/json'}
token_r = requests.post(token_url, json=token_data)
token = token_r.json().get('access_token')
any ideas?
Figured it out. I needed to add the oAuth2 requests library. See sample code below:
import requests
import json
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
app_id='<app id>'
client_secret='<client Secret>'
token_url='https://login.microsoftonline.com/tennantname.onmicrosoft.com/oauth2/v2.0/token'
scope='https://graph.microsoft.com/.default'
client = BackendApplicationClient(client_id=app_id, scope=scope, grant_type="client_credentials")
session = OAuth2Session(client=client, scope=scope)
# fill access token
token = session.fetch_token(token_url=token_url,client_id=app_id,scope=scope,client_secret=client_secret)
I've managed to setup an API Gateway secured with Cognito. The unauthenticated user role has an access policy that should grant it access to the gateway. I've also managed to use boto3 to retrieve an identity ID from the pool and obtain the associated open ID token, as well as the associated secret and access keys.
How do I now make a call to the gateway using these credentials? Is there a way to use boto3 to handle signing a request to a particular method on the API?
My code is based largely on the questioner's own answer, but I've tried to make it clearer where all the values come from.
import boto3
import requests
from requests_aws4auth import AWS4Auth
# Use 'pip install boto3 requests requests-aws4auth' to get these
region_name = 'ap-southeast-2' # or 'us-west-1' or whatever
# 12 decimal digits from your AWS login page
account_id = '123456789012'
# I've only found this in the sample code for other languages, e.g. JavaScript
# Services→Cognito→Manage Federated Identities→(your-id-pool)→Sample code
identity_pool_id = 'ap-southeast-2:fedcba98-7654-3210-1234-56789abcdef0'
# Create a new identity
boto3.setup_default_session(region_name = region_name)
identity_client = boto3.client('cognito-identity', region_name=region_name)
identity_response = identity_client.get_id(AccountId=account_id,
IdentityPoolId=identity_pool_id)
# We normally wouldn't log this, but to illustrate:
identity_id = identity_response['IdentityId']
print ('identity_id:', identity_id) # good idea not to log this
# Get the identity's credentials
credentials_response = identity_client.get_credentials_for_identity(IdentityId=identity_id)
credentials = credentials_response['Credentials']
access_key_id = credentials['AccessKeyId']
secret_key = credentials['SecretKey']
service = 'execute-api'
session_token = credentials['SessionToken']
expiration = credentials['Expiration']
# Again, we normally wouldn't log this:
print ('access_key_id', access_key_id)
print ('secret_key', secret_key)
print ('session_token', session_token)
print ('expiration', expiration)
# The access_key_id will look something like 'AKIABC123DE456FG7890', similar to
# Services→IAM→Users→(AWS_USER_NAME)→Security credentials→Access key ID
# Get the authorisation object
auth = AWS4Auth(access_key_id, secret_key, region_name, service,
session_token=session_token)
current_app['auth'] = auth
# Just an illustration again:
print ('auth: %(service)s(%(date)s) %(region)s:%(access_id)s' % auth.__dict__)
# We'll use that object to send a request to our app. This app doesn't
# exist in real life, though, so you'll need to edit the following quite
# heavily:
# Services→Cognito→Manage your User Pools→(your-user-pool)→Apps→App name
app_name = 'my-app-name'
api_path = 'dev/helloworld'
method = 'GET'
headers = {}
body = ''
url = 'https://%s.%s.%s.amazonaws.com/%s' % (app_name, service, region_name,
api_path)
response = requests.request(method, url, auth=auth, data=body, headers=headers)
The following code (and the requests-aws4auth library) did the job:
import boto3
import datetime
import json
from requests_aws4auth import AWS4Auth
import requests
boto3.setup_default_session(region_name='us-east-1')
identity = boto3.client('cognito-identity', region_name='us-east-1')
account_id='XXXXXXXXXXXXXXX'
identity_pool_id='us-east-1:YYY-YYYY-YYY-YY'
api_prefix='ZZZZZZZZZ'
response = identity.get_id(AccountId=account_id, IdentityPoolId=identity_pool_id)
identity_id = response['IdentityId']
print ("Identity ID: %s"%identity_id)
resp = identity.get_credentials_for_identity(IdentityId=identity_id)
secretKey = resp['Credentials']['SecretKey']
accessKey = resp['Credentials']['AccessKeyId']
sessionToken = resp['Credentials']['SessionToken']
expiration = resp['Credentials']['Expiration']
print ("\nSecret Key: %s"%(secretKey))
print ("\nAccess Key %s"%(accessKey))
print ("\nSession Token: %s"%(sessionToken))
print ("\nExpiration: %s"%(expiration))
method = 'GET'
headers = {}
body = ''
service = 'execute-api'
url = 'https://%s.execute-api.us-east-1.amazonaws.com/dev/helloworld' % api_prefix
region = 'us-east-1'
auth = AWS4Auth(accessKey, secretKey, region, service, session_token=sessionToken)
response = requests.request(method, url, auth=auth, data=body, headers=headers)
print(response.text)
Next code is working really well.
Hope to help:
from pprint import pprint
import requests
from pycognito import Cognito
USER_POOL_ID = 'eu-central-1_XXXXXXXXXXX'
CLIENT_ID = 'XXXXXXXXXXXX'
CLIENT_SECRET = 'XXXXXXXXXXX'
u = Cognito(USER_POOL_ID,CLIENT_ID, client_secret=CLIENT_SECRET, username='cognito user name')
u.authenticate('cognito user password')
id_token = u.id_token
headers = {'Authorization': 'Bearer ' + id_token}
api_url = 'https://XXXXXXXXXXX.execute-api.eu-central-1.amazonaws.com/stage/XXXXXXXXXXX'
r = requests.get(api_url, headers=headers)
pprint(dict(r.headers))
print(r.status_code)
print(r.text)
Here is an example from our public docs: http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
Cognito creds are no different than any other temporary creds, and the signing process is also the same. If you want to move back to Python the example above should be good, or I would guess that there are third-party libraries out there to do the signature for you.
identity_pool_id how to get
If you have not federated pool which could give you "identity_pool_id" ,
execution code below will give you identity_pool_id
import boto3
boto3.setup_default_session(
aws_access_key_id='AKIAJ7TBC72BPWNEWIDQ',
aws_secret_access_key='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',
region_name='ap-southeast-2')
client = boto3.client('cognito-identity')
response = client.list_identity_pools(MaxResults=3,)
print("IdentityPoolId-- ", response)
I am trying to use OAuth2 to get an authorization token using Python to a REST API. I am successful doing so using CURL but not with python. I am using the examples provided at the following docs:
https://requests-oauthlib.readthedocs.org/en/latest/oauth2_workflow.html
The following is my code:
#!/usr/bin/python
import requests
import requests_oauthlib
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
client_id = 'AAAAAA'
client_secret = 'BBBBBB'
client = BackendApplicationClient(client_id=client_id)
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url='https://example.com/as/token.oauth2', client_id=client_id, client_secret=client_secret)
print token
I am getting the following error:
oauthlib.oauth2.rfc6749.errors.InvalidClientError: (invalid_client) client_id value doesn't match HTTP Basic username value
This is a very basic API that only needs client_id and client_credentials to get an authorization token.
All information would be greatly appreciated.
The documentation specifies the following items:
client_id = r'your_client_id'
client_secret = r'your_client_secret'
redirect_uri = 'https://your.callback/uri'
By client key do you perhaps mean client key?
token = oauth.fetch_token(token_url='https://example.com/as/token.oauth2', client_id=client_id, client_secret=client_secret)
Try changing it to the above and give it a spin. using r'' for raw input instead and the token given.
I have found myself in a similar circumstance.
I am writing a Django app.
I was getting unauthorized_client and invalid_client exceptions.
In my case the post request in "Exchange the code" ("step 3" in the OAuth2 protocol) wasn't being formulated correctly.
Through much searching and trial and error I found it is possible to essentially customise the request. You can do this by specifying the optional arguments of auth, header and/or body.
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import WebApplicationClient, BackendApplicationClient
from requests.auth import HTTPBasicAuth
client_id = CLIENT_ID
client_secret = CLIENT_SECRET
authorization_base_url = AUTHORIZE_URI
token_url = TOKEN_URI
redirect_uri = REDIRECT_URI
auth = HTTPBasicAuth(client_id, client_secret)
scope = SCOPE
# Create the Authorization URI
# Not included here but store the state in a safe place for later
the_first_session = OAuth2Session(client_id=client_id, redirect_uri=redirect_uri, scope=scope)
authorization_url, state = the_first_session.authorization_url(authorization_base_url)
# Browse to the Authorization URI
# Login and Auth with the OAuth provider
# Now to respond to the callback
the_second_session = OAuth2Session(client_id, state=state)
body = 'grant_type=authorization_code&code=%s&redirect_uri=%s&scope=%s' % (request.GET.get('code'), redirect_uri, scope)
token = the_second_session.fetch_token(token_url, code=request.GET.get('code'), auth=auth, body=body)
I am using googles official oauth2client.client to access the google
plus api. I have a refresh token (that does not expire) stored in a database, and need
to recreate the temporary "Credentials" (access token) from that.
But I could not find a way to do this with to official library supplied by google.
So I hacked around it: used urllib to access the API that gives me a new
access_token from the refresh_token. Using the access_token I can then use the library.
I must be missing somthing!
from apiclient import discovery
from oauth2client.client import AccessTokenCredentials
from urllib import urlencode
from urllib2 import Request , urlopen, HTTPError
import json
# ==========================================
def access_token_from_refresh_token(client_id, client_secret, refresh_token):
request = Request('https://accounts.google.com/o/oauth2/token',
data=urlencode({
'grant_type': 'refresh_token',
'client_id': client_id,
'client_secret': client_secret,
'refresh_token': refresh_token
}),
headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Accept': 'application/json'
}
)
response = json.load(urlopen(request))
return response['access_token']
# ==========================================
access_token = access_token_from_refresh_token(CLIENT_ID, CLIENT_SECRET, REFRESH_TOKEN)
# now I can use the library properly
credentials = AccessTokenCredentials(access_token, "MyAgent/1.0", None)
http = credentials.authorize(httplib2.Http())
service = discovery.build('plus', 'v1', http=http)
google_request = service.people().get(userId='me')
result = google_request.execute(http=http)
I use: oauth2client.client.GoogleCredentials
cred = oauth2client.client.GoogleCredentials(access_token,client_id,client_secret,
refresh_token,expires_at,"https://accounts.google.com/o/oauth2/token",some_user_agent)
http = cred.authorize(httplib2.Http())
cred.refresh(http)
self.gmail_service = discovery.build('gmail', 'v1', credentials=cred)
You can construct an OAuth2Credentials instance directly like this:
import httplib2
from oauth2client import GOOGLE_REVOKE_URI, GOOGLE_TOKEN_URI, client
CLIENT_ID = '<client_id>'
CLIENT_SECRET = '<client_secret>'
REFRESH_TOKEN = '<refresh_token>'
credentials = client.OAuth2Credentials(
access_token=None, # set access_token to None since we use a refresh token
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
refresh_token=REFRESH_TOKEN,
token_expiry=None,
token_uri=GOOGLE_TOKEN_URI,
user_agent=None,
revoke_uri=GOOGLE_REVOKE_URI)
credentials.refresh(httplib2.Http()) # refresh the access token (optional)
print(credentials.to_json())
http = credentials.authorize(httplib2.Http()) # apply the credentials
I solved this quite easily (you certainly miss this documentation). This is a snippet of my code that tries to use Picasa API to get all of album from active user:
http = httplib2.Http(ca_certs=os.environ['REQUESTS_CA_BUNDLE'])
try:
http = self.oauth.credentials.authorize(http)
response, album_list = http.request(Picasa.PHOTOS_URL, 'GET')
if response['status'] == '403':
self.oauth.credentials.refresh(http)
response, album_list = http.request(Picasa.PHOTOS_URL, 'GET')
album_list = json.load(StringIO(album_list))
except Exception as ex:
Logger.debug('Picasa: error %s' % ex)
return {}
Use the refresh method coming from oauth2client.client.OAuth2Credentials. I think it's even okay to use if response['status'] != '200'. Got to check that!
You can also use the requests library as well:
import google.auth.transport.requests
import requests
request = google.auth.transport.requests.Request()
credentials.refresh(request)
Here is my sample code on an active project:
acct_creds = {
'token': self.attachment.account.google_drive_access_token,
'refresh_token': self.attachment.account.google_drive_refresh_token,
'client_id': settings.GOOGLE_CLIENT_ID,
'client_secret': settings.GOOGLE_CLIENT_SECRET,
'token_uri': 'https://37947.ngrok.io/authenticate/google/callback/',
'scopes': 'https://www.googleapis.com/auth/drive.appdata https://www.googleapis.com/auth/drive.file https://www.googleapis.com/auth/drive.install',
}
credentials = google.oauth2.credentials.Credentials(**acct_creds)
if credentials.valid:
print("Credentials valid")
else:
request = google.auth.transport.requests.Request()
credentials.refresh(request)
google.auth.transport.requests module
In case anyone is looking for the answer for how use a refresh token with google_auth_oauthlib, the following works for me:
flow.oauth2session.refresh_token(flow.client_config['token_uri'],
refresh_token=refresh_token,
client_id=<MY_CLIENT_ID>,
client_secret=flow.client_config['client_secret'])
creds = google_auth_oauthlib.helpers.credentials_from_session(
flow.oauth2session, flow.client_config)
I cannot find anywhere where this is documented though.
If you are using the 2018 Youtube Python Quickstart demo app using google-auth, you can't use oauth2client's storage.
So here is the correct way of storing the credentials
Here is a partially working solution for google-auth, missing the correct handling of the case where the token expires:
import os
import json
import os.path
import google.oauth2.credentials
from google.oauth2.credentials import Credentials
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from google_auth_oauthlib.flow import InstalledAppFlow
CLIENT_SECRETS_FILE = "client_secret.json"
SCOPES = ['https://www.googleapis.com/auth/youtube.force-ssl']
API_SERVICE_NAME = 'youtube'
API_VERSION = 'v3'
def get_authenticated_service():
if os.path.isfile("credentials.json"):
with open("credentials.json", 'r') as f:
creds_data = json.load(f)
creds = Credentials(creds_data['token'])
else:
flow = InstalledAppFlow.from_client_secrets_file(CLIENT_SECRETS_FILE, SCOPES)
creds = flow.run_console()
creds_data = {
'token': creds.token,
'refresh_token': creds.refresh_token,
'token_uri': creds.token_uri,
'client_id': creds.client_id,
'client_secret': creds.client_secret,
'scopes': creds.scopes
}
print(creds_data)
with open("credentials.json", 'w') as outfile:
json.dump(creds_data, outfile)
return build(API_SERVICE_NAME, API_VERSION, credentials = creds)
def channels_list(service, **kwargs):
results = service.channels().list(**kwargs).execute()
print('This channel\'s ID is %s. Its title is %s, and it has %s views.' %
(results['items'][0]['id'],
results['items'][0]['snippet']['title'],
results['items'][0]['statistics']['viewCount']))
if __name__ == '__main__':
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
service = get_authenticated_service()
channels_list(service, part='snippet,contentDetails,statistics', forUsername='GoogleDevelopers')
# or if the above doesn't work
channels_list(service, part='snippet,contentDetails,statistics', id='YOUR_YOUTUBE_CHANNEL_ID')
I recommend this method.
from oauth2client import client, GOOGLE_TOKEN_URI
CLIENT_ID = "client_id"
CLIENT_SECRET = "client_secret"
REFRESH_TOKEN = "refresh_token"
credentials = client.OAuth2Credentials(
access_token = None,
client_id = CLIENT_ID,
client_secret = CLIENT_SECRET,
refresh_token = REFRESH_TOKEN,
token_expiry = None,
token_uri = GOOGLE_TOKEN_URI,
token_ id = None,
revoke_uri= None)
http = credentials.authorize(httplib2.Http())
Even if the access token has expired, the credential is still authorize because of the refresh token.
If you have a refresh token then you can generate credentials for use by using OAuth2Credentials as below
from googleapiclient.discovery import build
import httplib2
from oauth2client import client, GOOGLE_TOKEN_URI
client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
refresh_token = "YOUR_REFRESH_TOKEN"
creds = client.OAuth2Credentials(
access_token = None,
client_id = client_id,
client_secret = client_secret,
refresh_token = refresh_token,
token_expiry = None,
token_uri = GOOGLE_TOKEN_URI,
user_agent="pythonclient")
creds.refresh(httplib2.Http())
I don't know what goes in the user agent but I put a random word in there
Now you can use it to build service object and use google APIs like
service = build("drive", "v3", credentials=creds)
In case someone wants to generate and use a offline refresh token for use without having to handle the autorization since it's for your testing then use google oauth playground to generate one. Checkout this video for more information.
You could store the entire credentials rather than only the refresh token:
json = credentials.to_json()
credentials = Credentials.new_from_json(json)
Look at the Storage object which does it this way.
Wow.. 2 years old question and not a good answer.. No surprise given that Google documentation is crap regarding this.
The correct way to do this is by extending the Storage class oauth2client.client.Storage
An example implementation(using mongodb collection _google_credentials) would be something like:
class Storage(oauth2client.client.Storage):
def __init__(self, key):
super(Storage, self).__init__()
self._key = key
def locked_get(self):
if not self._key: return None
data = _google_credentials.find_one({'_id': self._key})
if not data: return None
credentials = oauth2client.client.Credentials.new_from_json(json.dumps(data))
credentials.set_store(self)
return credentials
def locked_put(self, credentials):
data = json.loads(credentials.to_json())
_google_credentials.update_one({'_id': self._key}, {'$set': data},
upsert=True)
credentials.set_store(self)
def locked_delete(self):
bucket.delete(self._key)
Then when you initially get the credentials after step2_exchange, you need to store them using Storage().put:
e.g:
credentials = flow.step2_exchange(code)
Storage(user_id).put(credentials)
When you need the credentials again, just do:
credentials = Storage(user_id).get()
If you already have a Credentials object then you can refresh it like so:
if refresh:
import google_auth_httplib2
# credentials instanceof google.oauth2.credentials.Credentials
credentials.refresh(google_auth_httplib2.Request(httplib2.Http()))
I had created the Credentials object from an old token JSON file like so:
credentials = google.oauth2.credentials.Credentials(
token=token_json['access_token'],
refresh_token=token_json['refresh_token'],
id_token=token_json['id_token'],
token_uri=token_json['token_uri'],
client_id=token_json['client_id'],
client_secret=token_json['client_secret'],
scopes=token_json['scopes'])
In this way I was able to adapt some old oauth2client code.
At the moment i am needing to get a "success code" each time i want to run my .py app. to access my googdrive files which is a pain.
I saw Ali Afshars great vid on https://developers.google.com/drive/search-parameters where he uses:
from auth import http to streamline this process.
Not sure what this function should contain... Can you point me in the right direction, so that i can set up my .py to do this automatically ...or at least only once.
many thanks
Dav-o
EDIT relevant current snip of code follows:
import logging
logging.basicConfig()
import httplib2
import pprint
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.client import *
from apiclient import errors
CLIENT_ID = "864350......ps.googleusercontent.com"
CLIENT_SECRET = "sw0yb.....-zR6XWzEgM"
OAUTH_SCOPE = 'https://www.googleapis.com/auth/drive'
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oo...ooo' # Redirect URI for installed apps
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
http = httplib2.Http()
http = credentials.authorize(http)
drive_service = build('drive', 'v2', http=http)
What he probably does in auth is:
import httplib2
from oauth2client.client import OAuth2Credentials
http = httplib2.Http()
credentials = OAuth2Credentials(access_token, client_id, client_secret, refresh_token, None, '', '')
credentials.authorize(http)
You can wrap this code in a module/method and use the authenticate your requests. For multiple user scenarios, preserve access and refresh tokens (most likely in a db) and init credentials object with the stored tokens.