I am currently working on some code that connects to our Google apps domain, makes some administrative changes and then impersonates different users in our domain and uploads data to their drive, email and calendar. What I'm trying to do is figure out the proper way to use SignedJwtAssertionCredentials and change the "sub" kwarg. Do I need to redo the whole oauth process for each user I want to impersonate, i.e...
http = httplib2.Http()
credentials = SignedJwtAssertionCredentials(service_account, key, scope=api_scopes, sub=current_user)
http = credentials.authorize(http)
service = build(api_service, api_version, http=http)
or is there a way to just update/modify the sub kwarg and then re-authorize the httplib2 object?
I tried doing it the first way I described and it doesn't seem to work, about half way through the programs execution, I start getting odd errors saying,
WARNING:oauth2client.util:new_request() takes at most 1 positional argument (2 given)
Does anyone have any ideas on why I'm getting this error and or the best way I can implement the oauth process to do what I need it to?
Thanks!
Related
I'm building one of my first Python apps and am looking for some advice understanding google oauth and related app design patterns. My primary goal is to build an app that polls my Nest for data and learn to do so securely. Using google's API and oauth2 Python libraries, I've successfully created a tiny Flask app that will return data from a google API along with an auth token + refresh token. The core functions look something like this:
sdm = googleapiclient.discovery.build(
API_SERVICE_NAME, API_VERSION, credentials=credentials)
response = sdm.api.endpoint.call(API_PARAMS)
This app uses the client_secret.json info generated during ouath ID creation. However, in my travels across google's documentation and various tutorials I get the impression I should use the auth token (bearer token?) to make these API calls instead of client_secrets.json.
My original idea was to build one app that uses the refresh_token to refresh the auth_token and store it somewhere. I would then have a second app that reads the auth_token and makes the API calls, stores data, etc. In trying to create the first app that obtains and refreshes the auth token, I've created an app that can do both.
Building two functions (poll data and auth) seems like the right way to go. It allows me to separate the token, refresh token and client_secrets and keep them in different places as the API polling function only needs to know the auth token.
My question is, am I over complicating things in trying to separate auth_refresh and api_calls in to separate apps? What is the most common way to securely store and use oauth creds? Where can I find more information on common google API + oauth design patterns? Chances are I've read too many tutorials at once and have melted my brain in the google documentation labyrinth. Many thanks in advance.
After some more perusing of the docs and stackoverflow code snippets I landed on the below using the google API and oauth Python libraries. Separating these duties seems like the right way to go. In theory you could accomplish the same using curl or the Python requests library. This approach seemed "cleanest" to me. Eye of the beholder and all that...
In any case, here are the core parts one would need to independently rotate and use a google api auth token in a back end process (here, specifically, I want to periodically poll my Nest and store the data). It assumes you've already handled requesting client authorization and the google user has authorized the app. The portion outlined here is mainly for using the credentials post authorization.
import google.oauth2.credentials
import google.auth.transport.requests
import googleapiclient.discovery
# the first function refreshes the auth token every hour and writes it to secrets storage
# it can access the more sensitive bits like refresh_token
credentials = google.oauth2.credentials.Credentials(
token=TOKEN
refresh_token=REFRESH_TOKEN,
token_uri=TOKEN_URI,
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET
)
# refresh the token
request = google.auth.transport.requests.Request()
credentials.refresh(request)
TOKEN = credentials.token
write_to_secrets_store(TOKEN)
# the second function does the API call and related stuffs
# it can see the token but none of the sensitive bits like refresh_token and client_secret
credentials = google.oauth2.credentials.Credentials(token=TOKEN)
# make an api call
sdm = googleapiclient.discovery.build(
API_SERVICE_NAME, API_VERSION, credentials=credentials)
api_results = sdm.enterprises().structures().list(parent=ENTERPRISE_PARENT).execute()
do_stuff_with(api_results)
I can successfully authenticate with the pattern outlined here:
https://developers.google.com/youtube/v3/docs/videos/list?apix=true
for a simple prototype in Google Colab. However I cannot for the life of me figure out how to authenticate in a Cloud Function as there is no user to complete the flow. I'm sure it's a standard pattern, however I'm more familiar with the GCP APIs and the googleapiclient is a new one for me.
I have set up service account credentials with the right access, but I'm not even sure whether it makes sense to use these in a Cloud Function (maybe stored on GCS), or whether there is (as I hope) a more elegant solution.
Any help would be hugely appreciated, thanks!
Not really familiar with Youtube Data API, however based on the API reference for some details you need to issue a call as a video owner (e.g. processingDetails). This means that you need to use three-legged OAuth2 flow. For that you can either:
Setup another function that will generate an authorization URL ->
present it to the user -> setup another function as redirect_uri to
obtain authorization code -> exchange it for access and refresh
tokens -> store refresh token someplace safe where original function
can fetch it.
Obtain refresh token outside of Cloud Functions and hardcode it.
Hardcoding credentials is generally not a good practice, hence a better option would be to have a service account make requests on behalf of a user, but this is only possible for GSuite users via domain-wide delegation. With this, after setup, you would use sub claim with the email address of the impersonated user. You can see more here (make sure to switch to HTTP/REST to understand how JWT is created or, if you're not interested in details, just select Python.
I am attempting to get access to a customer's Google Drive to download their files. When testing the code on my own google drive, I am able to successfully download files. However, when I get the oauth code from them, I get the error:
oauth2client.client.FlowExchangeError: invalid_grant
After looking at some of the other answers, it has been suggested that you ensure access_type='offline', which seems to be the default and I can see in the generated url that this parameter is set. It also sounds like the code they give back may only be valid for an hour, however I have tried to use it within the hour and still no luck. Any other suggestions for how to avoid this problem?
Here is the code I have been running:
from oauth2client import client
import webbrowser
flow = client.flow_from_clientsecrets(
'client_secrets.json',
scope='https://www.googleapis.com/auth/drive.readonly',
redirect_uri='urn:ietf:wg:oauth:2.0:oob')
auth_uri = flow.step1_get_authorize_url()
webbrowser.open(auth_uri)
print auth_uri
auth_code = raw_input('Enter the auth code: ')
credentials = flow.step2_exchange(auth_code)
http_auth = credentials.authorize(httplib2.Http())
It errors on the flow.step2_exchange line.
There are three standard causes for this error that I am aware of.
The time is off on your PC. Solution: check it.
You are trying to use an authentication code more then once.
You can only use the authentication code once to get the refresh token.
You are testing and authenticating the code more then once. Each time you authenticate the code you get a valid refresh token. You can get up to 25 refresh tokens and then they stop working. Solution: make sure that every time you refresh the access you are using that refresh token and not an older one.
Also google as a Python tutorial that shows you how to access drive using the Python Client library it may make things easer for you. Python QuickStart
I'm using a Google Cloud Backends Project in Python to support an Android App. I want the server to be able to make some API calls on the user's accounts
(like Contacts, Maps, etc) rather than have the Android client take this overhead.
I am not using Webapp at all. I'd prefer not to - I don't see the need per se, unless it is the only way to make this work.
Wondering how I'd go forward:
#endpoints.api(name='data_api',
version='v1',
description='Access, create or delete data for meetups.',
audiences=client_ids.allowed_client_ids,
scopes=[ # Get email and Details
endpoints.EMAIL_SCOPE,
# Get Contacts
client_ids.CONTACTS_SCOPE
])
class DataApi(remote.Service):
#UserModel.method(name='user.fetch',
user_required=True)
def get_all_contacts(self, query):
#Now what?
I suppose need to relay the Auth token onto another custom request from the server to Google API's, but I don't know how to extract it and push it forward. Couldn't find it in the documentation or any questions here.
Thanks for your help.
EDIT:
I do know how to check if a user is authenticated, but that doesn't tell me how to hit the Google API's with the same auth token.
if endpoints.get_current_user() is not None:
#Do secret authenticated business...
The final step of the OAuth2 flow, the retrieved token sits in the API request headers. We can access it so:
import os
self.bearer = os.getenv('HTTP_AUTHORIZATION')
This provides the bearer string that can be relayed to hit other APIs. With the right scopes, when done using the atom and gdata Python libraries, we need to dress this up in a new object found here: https://github.com/ca9/meetup-backend/blob/master/atom/auth.py
in the EndpointsAuth class.
Use with gdata as follows:
if e_user:
gd_client = gdata.contacts.client.ContactsClient(source='<var>intense-terra-821</var>', auth_token=EndpointsAuth())
under any endpoint function.
I'm working on converting a Python script using the Google gdata API client + user/pass authentication to something more suitable for production (an API key). I am pretty frustrated with the muddled state of their documentation on authentication. I admittedly don't have a great grasp of OAuth2, but it seems like it's way more complicated for my usage case, which is: Hit Google Analytics every 24 hours to get the X most popular articles on our site.
In this scenario, we're not dealing with modifying someone's personal data, and all activity is centered on one account. It doesn't seem like OAuth2 is worth the complexity for something so simple.
I see that on the Google API Console (https://code.google.com/apis/console/), I've registered there and notice that there's a "Simple API Access" section with one key beneath the "Client ID for web applications" (which appears to be OAuth2). There's also the Google domain update page, https://www.google.com/accounts/UpdateDomain, but that appears to be OAuth related.
Is there any way to use this Simple API Access key (not OAuth) for retrieving analytics data with the Python gdata client, and if so, does anyone have any authentication examples? I already have the data retrieval stuff working once authenticated, but I'm using the user/pass approach, which is not appropriate for production.
Greg,
If you are already using the library gdata-python-client, this is relatively easy to do if you are the only user that your application will be authorizing.
The general mechanisms were detailed in a blog post in September, 2011, but I'll describe them here for completeness.
Part 1: Go to the APIs console and start a new project.
Part 2: From the project, go to "Services" and enable "Analytics API"
Part 3: From the project, go to "API Access" and click "Create an OAuth 2.0 client ID..." (you'll need to provide a product name, though the value you provide won't matter). When asked for the application type, select "Installed Application" and then "Create client ID". Since you will be the only user, you will only need one refresh token, and you can get this by authorizing from a desktop application a single time.
Part 4: Get your client id and client secret from the APIs console and then create an empty token:
import gdata.gauth
CLIENT_ID = 'id-from-apis-console'
CLIENT_SECRET = 'secret-from-apis-console'
SCOPE = 'https://www.google.com/analytics/feeds/' # Default scope for analytics
token = gdata.gauth.OAuth2Token(
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPE,
user_agent='application-name-goes-here')
I got the scope from GData FAQ, though I'm not sure if it is correct.
Part 5: Use the token to create authorization URL for you to visit:
url = token.generate_authorize_url(redirect_uri='urn:ietf:wg:oauth:2.0:oob')
Since your application is an "Installed Application", your redirect URI is the default 'urn:ietf:wg:oauth:2.0:oob'. (Also note, the blog post had a typo and used the keyword argument redirect_url.)
Part 6: Visit the url and authorize your application to make requests on behalf of your account. After authorizing, you'll be redirected to a page with a code on it. This code will be used to exchange for an access token and a long-lived refresh token. The code has a life of 10 minutes and the access token has a life of an hour. The refresh token will allow you to get new access tokens for signing requests in perpetuity (or until you revoke the permission from your account).
Part 7: Use the code to get an access token:
code = 'random-string-from-redirected-page'
token.get_access_token(code) # This returns the token, but also changes the state
This again differs slightly from the blog post, because we are using an installed application.
Part 8: With the token you can now make all requests you want to make to the analytics client:
import gdata.analytics.client
client = gdata.analytics.client.AnalyticsClient()
token.authorize(client)
This is the big money right here. When an access token expires, the API requests signed with that token are rejected. However, by authorizing the client as above, when the said requests fail, the token attempts to use the refresh token to obtain a new access token. If it successfully obtains a new access token, the client resends the original API request, signed with the new access token.
I don't know anything about the Analytics API so I won't provide any more details there.
Future Use Note 1: Saving information for future use. You can re-use this from different places and after this use very easily. There are methods called token_to_blob and token_from_blob provided by the library that allow turning a token into a string and converting out of a string:
saved_blob_string = gdata.gauth.token_to_blob(token)
Once you have done this, you can store the string in a file and kill your running Python process. When you'd like to use it again:
saved_blob_string = retrieve_string_from_file() # You'll need to implement this
token = gdata.gauth.token_from_blob(saved_blob_string)
Future Use Note 2: This token will be able to be used to authorize a client and perform all your magic again and again, so long as you have the refresh token around. If for some reason you would like to get an access token again without calling token.generate_authorize_url, you'll need to manually set this on the object:
token.redirect_uri = 'urn:ietf:wg:oauth:2.0:oob'
Future Use Note 3: Also, if you lose your refresh token and would like to get another one without having to go to the browser to revoke the original, you can use the approval_prompt parameter to get a new refresh token by visiting the url generated by:
url = token.generate_authorize_url(
redirect_uri='urn:ietf:wg:oauth:2.0:oob',
approval_prompt='force')