I created an python application that is using the Youtube api (so examples are in python, but doesn't really matter, the concepts should be the same). I managed to get it working where I can connect and make api calls. However, when I connect to the api, I have to define a flow that checks if a the credentials storage file exists. If it doesn't, then I have to manually sign in using the flow. After sign in the file (main.py-oauth2.json), is created with the token. I would like to be able to download the credentials without having to sign manually sign in. I was hoping there was a way to make a POST request for that token, like I have seen here, but I have been able to do this with Youtube api. Does anyone know how to implement the desired feature ?
main.py
flow = flow_from_clientsecrets(CLIENT_SECRETS_FILE,
scope=YOUTUBE_UPLOAD_SCOPE,
message=MISSING_CLIENT_SECRETS_MESSAGE)
storage = Storage(OAUTH_CREDENTIALS)
credentials = storage.get()
if credentials is None or credentials.invalid:
# manual / UI login
credentials = run_flow(flow, storage, args)
Trying to use a google service account throws 401 errors on upload.
credentials = Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=YOUTUBE_UPLOAD_SCOPES)
if credentials is None or credentials.expired:
raise ValueError('Invalid credentials')
return build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION,
credentials=credentials)
...
status, response = insert_request.next_chunk()
# <HttpError 401 "Unauthorized">
Evidence this can be done
The oauth2client.service_account.ServiceAccountCredentials class is
only used with OAuth 2.0 Service Accounts. No end-user is involved
for these server-to-server API calls, so you can create this object
directly without using a Flow object.
youtube api
Oauth flow docs
https://developers.google.com/identity/protocols/OAuth2#serviceaccount
The problem is that most YouTube data is private user data. Being that it is private user data you must be authenticated as a user who has access to the data in question in order to access it. To do that we use Oauth2 and login to our account and get an access token and a refresh token returned.
The access token can be used to request data from the Youtube Api, the refresh token can be used to request a new access token when ever the access token expires (After an hour)
Normally i would say that you should consider using a service account. Services accounts are dummy users who can be preconfigured with access to user data. Unfortunately the Youtube api does not support service accounts.
What you should be doing and what i have done a number of times in the past is to authenticate your code once. Get the refresh token and save it. In the future whenever you wish to run your application you simply use the refresh token to request a new access token and you will be able to access the api. You wont have to manually type your login and password and consent to the access anymore everything can be done in the background using the refesh token.
Note: You will need to watch it there are some cases that can cause a refresh token to expire but you shouldn't worry for the most part they are good for as long as you continue to use them regularly.
I am not a python dev but found this
from oauth2client import client, GOOGLE_TOKEN_URI
CLIENT_ID = "client_id"
CLIENT_SECRET = "client_secret"
REFRESH_TOKEN = "refresh_token"
credentials = client.OAuth2Credentials(
access_token = None,
client_id = CLIENT_ID,
client_secret = CLIENT_SECRET,
refresh_token = REFRESH_TOKEN,
token_expiry = None,
token_uri = GOOGLE_TOKEN_URI,
token_ id = None,
revoke_uri= None)
http = credentials.authorize(httplib2.Http())
Related
I am trying to use google cloud to get the stats of my youtube channel, but I don't want to have to complete an OAuth every time and enter key into console.. I have an API key and am wondering if its possible to just get the Stats using the API key instead of having to go through OAuth process, as I plan to hook this up to a display and don't want to constantly have to do the OAuth.
Here is the code I copied from the Github, but once again, this does the OAuth and its kinda a pain and I wish to just use API key so I don't have to interact with it to get my stats.
import google.oauth2.credentials
import google_auth_oauthlib.flow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from google_auth_oauthlib.flow import InstalledAppFlow
SCOPES = ['https://www.googleapis.com/auth/yt-analytics.readonly']
API_SERVICE_NAME = 'youtubeAnalytics'
API_VERSION = 'v2'
CLIENT_SECRETS_FILE = 'YTClientSecrets.json'
def get_service():
flow = InstalledAppFlow.from_client_secrets_file(CLIENT_SECRETS_FILE, SCOPES)
credentials = flow.run_console()
return build(API_SERVICE_NAME, API_VERSION, credentials = credentials)
def execute_api_request(client_library_function, **kwargs):
response = client_library_function(
**kwargs
).execute()
print(response)
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
youtubeAnalytics = get_service()
execute_api_request(
youtubeAnalytics.reports().query,
ids='channel==MINE',
startDate='2017-01-01',
endDate='2021-12-31',
metrics='estimatedMinutesWatched,views,likes,subscribersGained',
dimensions='day',
sort='day'
)
print(response.text)```
Answer: no you can not use an API key to access private user data.
How to see if you need authorization for a method.
If you check the documentation for jobs.reports.get you will notice the following
All methods which request private user data will have an authorization section. This tells you which authorization scope is required in order to access the private data.
API keys only allow you to access public data, for exampmle public videos uploaded to youtube are public data so you do not need to be authorized to access them you can use an api key.
Service accounts
There is another type of authorization called service account authorization which allows you to pre authorize a service account to access private user data.
However there are limitations to which APIs support service account authentication.
The YouTube Analytics API does not support the service account flow.
The YouTube Reporting API only supports the service account flow for YouTube content owners that own and manage multiple YouTube channels. Specifically, content owners can use service accounts in API requests that set a value for the onBehalfOfContentOwner request parameter.
If you can not use Service account Authentication your only option is to make a single user type system where you authorize your script store your refresh token so that your script can use the refresh token in the future to request a new access token.
I'm trying to generate a id_token to make authenticated calls to Google Cloud Functions using this following code, that I got from here:
# IAP audience is the ClientID of IAP-App-Engine-app in
# the API->credentials page
# Cloud Function and Cloud Run need the base URL of the service
audience = 'my_cloud_function_url'
# #1 Get the default credential to generate the access token
credentials, project_id = google.auth.default()
# #2 To use the current service account email
service_account_email = credentials.service_account_email
# Don't work with user account, so define manually the email
# service_account_email = 'MY SERVICE ACCOUNT EMAIL'
# #3 prepare the call the the service account credentials API
sa_credentials_url = f'https://iamcredentials.googleapis.com/' \
f'v1/projects/-/serviceAccounts/' \
f'{service_account_email}:generateIdToken'
headers = {'Content-Type': 'application/json'}
# Create an AuthorizedSession that includes
# automatically the access_token based on your credentials
authed_session = AuthorizedSession(credentials)
# Define the audience in the request body
# add the parameter "'includeEmail':true" for IAP access
body = json.dumps({'audience': audience})
# Make the call
token_response = authed_session.request('POST',sa_credentials_url,
data=body, headers=headers)
jwt = token_response.json()
id_token = jwt['token']
I'm not running it into a Google Cloud environment, so I set my GOOGLE_APPLICATION_CREDENTIALS to a json file from a service account I generated in the console. I asked my network manager to authorize the service account within the ['https://www.googleapis.com/auth/cloud-platform'] scope.
When I run the script, I get the following error:
{'error': {'code': 403, 'message': 'The caller does not have permission', 'status': 'PERMISSION_DENIED'}}
I imagine that this error has something to do with not having some intern access with this service account, but I'm not sure, can anybody help me with this?
Edit:
As stated by #guillaume blaquiere on the comments, you can get the id_token without calling the Service Account Credential API, And I could obtain the id_token, but was unable to call my service account with that id_token
import google.oauth2.id_token
import google.auth.transport.requests
request = google.auth.transport.requests.Request()
audience = 'my_cloud_function_url'
id_token = google.oauth2.id_token.fetch_id_token(request, audience)
headers = {'Authorization': f'bearer {id_token}'}
service_response = requests.get(audience, headers=headers)
Does anyone knows why thes id_token is invalid?
You don't need to call the Service Account Credential API, you can use the fetch_id_token method (but you have to know which it exists! It's not very well known and documented!)
import google.oauth2.id_token
import google.auth.transport.requests
request = google.auth.transport.requests.Request()
audience = 'my_cloud_function_url'
id_token = google.oauth2.id_token.fetch_id_token(request, audience)
The library use the default credential that you set in the GOOGLE_APPLICATION_CREDENTIALS env vars. So, it's convenient to get an ID token with this python lib!
The 403 error The caller does not have permission means that the running service account of your Cloud Function doesn't have the required role to invoke the function. Could you go to IAM and check if your service account has Cloud Functions Invoker role? If the service account doesn't have the permission, then please follow this step:
Check the role of your Service Account in IAM, look for Role column and verify if it doesn't have the role mentioned above.
To add permission, hover to your Service Account then Click edit in the right side column named Inheritance.
Select the role Cloud Functions > Cloud Functions Invoker from the Select a role drop-down menu.
Click Save.
I'm trying to implement a simple python client for Spotify api. According to the Spotify's Authorization Guide, the app can be authorized in two ways:
App Authorization: Spotify authorizes your app to access the Spotify Platform (APIs, SDKs and Widgets).
User Authorization: Spotify, as well as the user, grant your app permission to access and/or modify the user’s own data. For information about User Authentication, see User Authentication with OAuth 2.0. Calls to the Spotify Web API require authorization by your application user. To get that authorization, your application generates a call to the Spotify Accounts Service /authorize endpoint, passing along a list of the scopes for which access permission is sought.
CLIENT CREDENTIALS
My first attempt used the app authorization using the oauth2 module from Spotipy, because it requires no token passed, but only client id and client secret, which belong to the app developer.
client.py
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
class SpotifyWrapper(spotipy.Spotify):
def category_playlists(self, category, limit=50, offset=0):
return self._get('browse/categories/%s/playlists' % category,
limit=limit,
offset=offset)
def get_api_client():
# create a client authentication request
client_cred = SpotifyClientCredentials(
client_id=DevelopmentConfig.SPOTIFY_CLIENT_ID,
client_secret=DevelopmentConfig.SPOTIFY_CLIENT_SECRET
)
# create a spotify client with a bearer token,
# dynamically re-created if necessary
return SpotifyWrapper(auth=client_cred.get_access_token())
Then I would import and declare it here:
spotify_utilities.py
from app.resources.spotify.client import get_api_client
sp = get_api_client()
And in order to make requests and get user playlists, pass it like so:
def get_user_playlist(username, sp):
ids=[]
playlists = sp.user_playlists(username)
for playlist in playlists['items']:
ids.append(playlist['id'])
print("Name: {}, Number of songs: {}, Playlist ID: {} ".
format(playlist['name'].encode('utf8'),
playlist['tracks']['total'],
playlist['id']))
return ids
This works and will get user content, where the user is the app developer.
IMPLICIT FLOW
Now I want to move on to Implicit Flow, whereby the app asks ANY user who uses for access and scopes, and for that a token will be required.
Once I fetch the token using Javascript, I know I can use it to get user data hitting the API with simple requests:
GET_USER_PROFILE_ENDPOINT = 'https://api.spotify.com/v1/users/{user_id}'
GET_USER_PLAYLISTS_ENDPOINT = 'https://api.spotify.com/v1/users/{user_id}/playlists'
def get_user_profile(token, user_id):
url = GET_USER_PROFILE_ENDPOINT.format(id=user_id)
resp = requests.get(url, headers={"Authorization": "Bearer {}".format(token)})
print (len(resp.json()))
return resp.json()
def get_user_playlists(token, user_id):
url = GET_USER_PLAYLISTS_ENDPOINT..format(id=user_id)
resp = requests.get(url, headers={"Authorization": "Bearer {}".format(token)})
print (len(resp.json()))
return resp.json()
but in order to get (and change) user data first I need to use this token to fetch user ID.
Also, by the following example form Spotipy docs, user must provide his username at terminal:
if __name__ == '__main__':
if len(sys.argv) > 1:
username = sys.argv[1]
else:
print("Whoops, need your username!")
print("usage: python user_playlists.py [username]")
sys.exit()
token = util.prompt_for_user_token(username)
if token:
sp = spotipy.Spotify(auth=token)
playlists = sp.user_playlists(username)
After reading the docs from Spotify and Spotify, some things that are still not clear:
Is it possible to get this USER ID from passing the token only?
Must the app user necessarily provide his Spotify username via a form in a browser, besides authorizing the app when authentication is prompted?
Is it possible to tweak the wrapper above and implement a client which contemplates the parameters required for implicit flow? Would simply spotify = spotipy.Spotify(auth=token) work and get current usr data?
Also, by the following example form Spotipy docs, user must provide
his username at terminal:
That's because Spotipy caches tokens on disk. When no cache path is specified by the user the username simply gets appended to the files file extension as seen here. So the username specified is never being transmitted to any Spotify API endpoint.
1) Is it possible to get this USER ID from passing the token only?
Yes, using /v1/me instead of /v1/users/{user_id} will do exactly that assuming you are using an access token generated by Authorization Code flow or Implicit Grant flow.
2) Must the app user necessarily provide his Spotify username via a
form in a browser, besides authorizing the app when authentication is
prompted?
No, as seen in the first paragraph of my answer.
3) Is it possible to tweak the wrapper above and implement a client
which contemplates the parameters required for implicit flow? Would
simply spotify = spotipy.Spotify(auth=token) work and get current usr
data?
Spotipy seems to only use Authorization Code Flow right now. Due to you said you are
trying to implement a simple python client for Spotify api.
you should just implement Implicit Grant flow in your application. This has examples for all three Spotify authorization flows.
I followed the Google Sheet Python API Quickstart guide (https://developers.google.com/sheets/api/quickstart/python) and was able to get it working using their supplied code:
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
# If modifying these scopes, delete your previously saved credentials
# at ~/.credentials/sheets.googleapis.com-python-quickstart.json
SCOPES = 'https://www.googleapis.com/auth/spreadsheets'
CLIENT_SECRET_FILE = 'my/path/client_secret.json'
APPLICATION_NAME = 'Google Sheets API Python Quickstart'
credential_path = 'my/path/sheets.googleapis.com-python-quickstart.json'
store = Storage(credential_path)
credentials = store.get()
## !!!!! Is this needed?
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
In the default setup I downloaded two JSON files:
client_secret.JSON
downloaded to project directory.
sheets.googleapis.com-python-quickstart.JSON
downloaded to ~/.credentials directory
The sheets.googleapis.com JSON file starts with:
"_module": "oauth2client.client".
Question 1: What is the purpose for each of these JSON files?
Question 2: Are both of these JSON files needed to successfully use the Google Sheets API?
I am thinking no, as I am able to get the API working without the client_secret.JSON file.
How about this answer? I think when you know the OAuth2 process for retrieving access token and refresh token, you can understand the meaning of both files. The flow for retrieving access token and refresh token using OAuth2 is as follows.
Flow :
Download client_secret.JSON from the API Console.
client_secret.JSON includes client_id, client_secret and redirect_uris.
Retrieve an authorization code using scopes and client_id from client_secret.JSON.
Retrieve access token and refresh token using the authorization code, client_id, client_secret and redirect_uris.
Retrieved access token, refresh token and other parameters are saved to the file of sheets.googleapis.com-python-quickstart.JSON.
Note :
When you run the Quickstart for the first time, the authorization process using your browser is launched. At that time, the script of Quickstart retrieves the authorization code using client_id and scopes, and then the access token and refresh token are retrieved using the authorization code, client_id, client_secret and redirect_uris.
After the first run of the Quickstart, the access token is retrieved by the refresh token from sheets.googleapis.com-python-quickstart.JSON. By this, retrieving the authorization code using browser is not required to do. So when there is sheets.googleapis.com-python-quickstart.JSON, client_secret.JSON is not required.
I think that this leads to an answer for your Question 2.
But, if you want to change scopes and/or credentials of client_secret.JSON, the authorization process using browser and retrieving the authorization code are required to do. For this, you have to remove sheets.googleapis.com-python-quickstart.JSON and authorize again. At that time, at Quickstart, client_secret.JSON is used again.
References :
Using OAuth 2.0 to Access Google APIs
Authorization for Google Services
If this is not useful for you, I'm sorry.
I'm using the Python API that google provides. What I want to do is just make sure that the access token doesn't expire. I have the refresh_token stored in the credentials file. I'm just not sure how to 'check' that the token is still good before making the call to the API and if need be refreshing it and re-storing it in the credentials file.
I did a test that even if I delete the access tokens from the credentials file that it rewrites them into it using the refresh token. I'm hoping that will work for expired access tokens as well.
Thanks
storage = Storage('cred_storage.txt')
credentials = storage.get()
if not credentials:
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
storage.put(credentials)
http = httplib2.Http()
http = credentials.authorize(http)
print http
service = build('admin', 'reports_v1', http=http)
print service
data_query = service.customerUsageReports().get(**{'date':'2015-01-07'})
feed = data_query.execute()
print feed
Simply check the case of expired access token and refresh your expired access token like this:
if credentials.access_token_expired:
credentials.refresh(httplib2.Http())
Tip: While developing this, you can test by editing the access token expiry date in the credentials text file and forcing it to be older than an hour
Also, in your code on the line where you are checking if not credentials:, you can better handle that case with:
if credentials is None or credentials.invalid:
I came across this question while trying to find a way to refresh an access token when construction a credentials object when using from_authorized_user_info. Unfortunately, the following code did not work for me:
credentials.refresh(httplib2.Http())
But I found this documentation from the Oauth library that works wonder. Shared below:
import google.auth.transport.requests
import requests
request = google.auth.transport.requests.Request()
credentials.refresh(request)