We Keep Coding

How can I retrieve an id_token to access a Google Cloud Function?

I'm trying to generate a id_token to make authenticated calls to Google Cloud Functions using this following code, that I got from here:
# IAP audience is the ClientID of IAP-App-Engine-app in
# the API->credentials page
# Cloud Function and Cloud Run need the base URL of the service
audience = 'my_cloud_function_url'
# #1 Get the default credential to generate the access token
credentials, project_id = google.auth.default()
# #2 To use the current service account email
service_account_email = credentials.service_account_email
# Don't work with user account, so define manually the email
# service_account_email = 'MY SERVICE ACCOUNT EMAIL'
# #3 prepare the call the the service account credentials API
sa_credentials_url = f'https://iamcredentials.googleapis.com/' \
f'v1/projects/-/serviceAccounts/' \
f'{service_account_email}:generateIdToken'
headers = {'Content-Type': 'application/json'}
# Create an AuthorizedSession that includes
# automatically the access_token based on your credentials
authed_session = AuthorizedSession(credentials)
# Define the audience in the request body
# add the parameter "'includeEmail':true" for IAP access
body = json.dumps({'audience': audience})
# Make the call
token_response = authed_session.request('POST',sa_credentials_url,
data=body, headers=headers)
jwt = token_response.json()
id_token = jwt['token']
I'm not running it into a Google Cloud environment, so I set my GOOGLE_APPLICATION_CREDENTIALS to a json file from a service account I generated in the console. I asked my network manager to authorize the service account within the ['https://www.googleapis.com/auth/cloud-platform'] scope.
When I run the script, I get the following error:
{'error': {'code': 403, 'message': 'The caller does not have permission', 'status': 'PERMISSION_DENIED'}}
I imagine that this error has something to do with not having some intern access with this service account, but I'm not sure, can anybody help me with this?
Edit:
As stated by #guillaume blaquiere on the comments, you can get the id_token without calling the Service Account Credential API, And I could obtain the id_token, but was unable to call my service account with that id_token
import google.oauth2.id_token
import google.auth.transport.requests
request = google.auth.transport.requests.Request()
audience = 'my_cloud_function_url'
id_token = google.oauth2.id_token.fetch_id_token(request, audience)
headers = {'Authorization': f'bearer {id_token}'}
service_response = requests.get(audience, headers=headers)
Does anyone knows why thes id_token is invalid?

You don't need to call the Service Account Credential API, you can use the fetch_id_token method (but you have to know which it exists! It's not very well known and documented!)
import google.oauth2.id_token
import google.auth.transport.requests
request = google.auth.transport.requests.Request()
audience = 'my_cloud_function_url'
id_token = google.oauth2.id_token.fetch_id_token(request, audience)
The library use the default credential that you set in the GOOGLE_APPLICATION_CREDENTIALS env vars. So, it's convenient to get an ID token with this python lib!

The 403 error The caller does not have permission means that the running service account of your Cloud Function doesn't have the required role to invoke the function. Could you go to IAM and check if your service account has Cloud Functions Invoker role? If the service account doesn't have the permission, then please follow this step:
Check the role of your Service Account in IAM, look for Role column and verify if it doesn't have the role mentioned above.
To add permission, hover to your Service Account then Click edit in the right side column named Inheritance.
Select the role Cloud Functions > Cloud Functions Invoker from the Select a role drop-down menu.
Click Save.

Spotify - get and change user data with Implicit Flow

I'm trying to implement a simple python client for Spotify api. According to the Spotify's Authorization Guide, the app can be authorized in two ways:
App Authorization: Spotify authorizes your app to access the Spotify Platform (APIs, SDKs and Widgets).
User Authorization: Spotify, as well as the user, grant your app permission to access and/or modify the user’s own data. For information about User Authentication, see User Authentication with OAuth 2.0. Calls to the Spotify Web API require authorization by your application user. To get that authorization, your application generates a call to the Spotify Accounts Service /authorize endpoint, passing along a list of the scopes for which access permission is sought.
CLIENT CREDENTIALS
My first attempt used the app authorization using the oauth2 module from Spotipy, because it requires no token passed, but only client id and client secret, which belong to the app developer.
client.py
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
class SpotifyWrapper(spotipy.Spotify):
def category_playlists(self, category, limit=50, offset=0):
return self._get('browse/categories/%s/playlists' % category,
limit=limit,
offset=offset)
def get_api_client():
# create a client authentication request
client_cred = SpotifyClientCredentials(
client_id=DevelopmentConfig.SPOTIFY_CLIENT_ID,
client_secret=DevelopmentConfig.SPOTIFY_CLIENT_SECRET
)
# create a spotify client with a bearer token,
# dynamically re-created if necessary
return SpotifyWrapper(auth=client_cred.get_access_token())
Then I would import and declare it here:
spotify_utilities.py
from app.resources.spotify.client import get_api_client
sp = get_api_client()
And in order to make requests and get user playlists, pass it like so:
def get_user_playlist(username, sp):
ids=[]
playlists = sp.user_playlists(username)
for playlist in playlists['items']:
ids.append(playlist['id'])
print("Name: {}, Number of songs: {}, Playlist ID: {} ".
format(playlist['name'].encode('utf8'),
playlist['tracks']['total'],
playlist['id']))
return ids
This works and will get user content, where the user is the app developer.
IMPLICIT FLOW
Now I want to move on to Implicit Flow, whereby the app asks ANY user who uses for access and scopes, and for that a token will be required.
Once I fetch the token using Javascript, I know I can use it to get user data hitting the API with simple requests:
GET_USER_PROFILE_ENDPOINT = 'https://api.spotify.com/v1/users/{user_id}'
GET_USER_PLAYLISTS_ENDPOINT = 'https://api.spotify.com/v1/users/{user_id}/playlists'
def get_user_profile(token, user_id):
url = GET_USER_PROFILE_ENDPOINT.format(id=user_id)
resp = requests.get(url, headers={"Authorization": "Bearer {}".format(token)})
print (len(resp.json()))
return resp.json()
def get_user_playlists(token, user_id):
url = GET_USER_PLAYLISTS_ENDPOINT..format(id=user_id)
resp = requests.get(url, headers={"Authorization": "Bearer {}".format(token)})
print (len(resp.json()))
return resp.json()
but in order to get (and change) user data first I need to use this token to fetch user ID.
Also, by the following example form Spotipy docs, user must provide his username at terminal:
if __name__ == '__main__':
if len(sys.argv) > 1:
username = sys.argv[1]
else:
print("Whoops, need your username!")
print("usage: python user_playlists.py [username]")
sys.exit()
token = util.prompt_for_user_token(username)
if token:
sp = spotipy.Spotify(auth=token)
playlists = sp.user_playlists(username)
After reading the docs from Spotify and Spotify, some things that are still not clear:
Is it possible to get this USER ID from passing the token only?
Must the app user necessarily provide his Spotify username via a form in a browser, besides authorizing the app when authentication is prompted?
Is it possible to tweak the wrapper above and implement a client which contemplates the parameters required for implicit flow? Would simply spotify = spotipy.Spotify(auth=token) work and get current usr data?

Also, by the following example form Spotipy docs, user must provide
his username at terminal:
That's because Spotipy caches tokens on disk. When no cache path is specified by the user the username simply gets appended to the files file extension as seen here. So the username specified is never being transmitted to any Spotify API endpoint.
1) Is it possible to get this USER ID from passing the token only?
Yes, using /v1/me instead of /v1/users/{user_id} will do exactly that assuming you are using an access token generated by Authorization Code flow or Implicit Grant flow.
2) Must the app user necessarily provide his Spotify username via a
form in a browser, besides authorizing the app when authentication is
prompted?
No, as seen in the first paragraph of my answer.
3) Is it possible to tweak the wrapper above and implement a client
which contemplates the parameters required for implicit flow? Would
simply spotify = spotipy.Spotify(auth=token) work and get current usr
data?
Spotipy seems to only use Authorization Code Flow right now. Due to you said you are
trying to implement a simple python client for Spotify api.
you should just implement Implicit Grant flow in your application. This has examples for all three Spotify authorization flows.

Youtube API request credentials

I created an python application that is using the Youtube api (so examples are in python, but doesn't really matter, the concepts should be the same). I managed to get it working where I can connect and make api calls. However, when I connect to the api, I have to define a flow that checks if a the credentials storage file exists. If it doesn't, then I have to manually sign in using the flow. After sign in the file (main.py-oauth2.json), is created with the token. I would like to be able to download the credentials without having to sign manually sign in. I was hoping there was a way to make a POST request for that token, like I have seen here, but I have been able to do this with Youtube api. Does anyone know how to implement the desired feature ?
main.py
flow = flow_from_clientsecrets(CLIENT_SECRETS_FILE,
scope=YOUTUBE_UPLOAD_SCOPE,
message=MISSING_CLIENT_SECRETS_MESSAGE)
storage = Storage(OAUTH_CREDENTIALS)
credentials = storage.get()
if credentials is None or credentials.invalid:
# manual / UI login
credentials = run_flow(flow, storage, args)
Trying to use a google service account throws 401 errors on upload.
credentials = Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=YOUTUBE_UPLOAD_SCOPES)
if credentials is None or credentials.expired:
raise ValueError('Invalid credentials')
return build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION,
credentials=credentials)
...
status, response = insert_request.next_chunk()
# <HttpError 401 "Unauthorized">
Evidence this can be done
The oauth2client.service_account.ServiceAccountCredentials class is
only used with OAuth 2.0 Service Accounts. No end-user is involved
for these server-to-server API calls, so you can create this object
directly without using a Flow object.
youtube api
Oauth flow docs
https://developers.google.com/identity/protocols/OAuth2#serviceaccount

The problem is that most YouTube data is private user data. Being that it is private user data you must be authenticated as a user who has access to the data in question in order to access it. To do that we use Oauth2 and login to our account and get an access token and a refresh token returned.
The access token can be used to request data from the Youtube Api, the refresh token can be used to request a new access token when ever the access token expires (After an hour)
Normally i would say that you should consider using a service account. Services accounts are dummy users who can be preconfigured with access to user data. Unfortunately the Youtube api does not support service accounts.
What you should be doing and what i have done a number of times in the past is to authenticate your code once. Get the refresh token and save it. In the future whenever you wish to run your application you simply use the refresh token to request a new access token and you will be able to access the api. You wont have to manually type your login and password and consent to the access anymore everything can be done in the background using the refesh token.
Note: You will need to watch it there are some cases that can cause a refresh token to expire but you shouldn't worry for the most part they are good for as long as you continue to use them regularly.
I am not a python dev but found this
from oauth2client import client, GOOGLE_TOKEN_URI
CLIENT_ID = "client_id"
CLIENT_SECRET = "client_secret"
REFRESH_TOKEN = "refresh_token"
credentials = client.OAuth2Credentials(
access_token = None,
client_id = CLIENT_ID,
client_secret = CLIENT_SECRET,
refresh_token = REFRESH_TOKEN,
token_expiry = None,
token_uri = GOOGLE_TOKEN_URI,
token_ id = None,
revoke_uri= None)
http = credentials.authorize(httplib2.Http())

Making a request to the Google Analtycis API responds with Login Required

I get data from the Google Analytics API v3
https://www.googleapis.com/analytics/v3/data/ga?ids=ga:181335694&metrics=ga:sessions&start-date=7daysAgo&end-date=today
Once I run the browser is throwing an error
{
"error":{
"errors":[
{
"domain":"global",
"reason":"required",
"message":"Login Required",
"locationType":"header",
"location":"Authorization"
}
],
"code":401,
"message":"Login Required"
}
}
How to resolve this error?
Once I run python code I get the Google Analytics api, but I run in browser is throwing error for login is required how to resolve it.

There are two types of data when we are talking about Google APIs.
Public data data that is not owned by anyone and everyone can look at
Private data which is owned by a user and you must have permission to access it.
"Login Required",
Means exactly that you must be authenticated in order to access the data you are trying to access. You need the permission of the owner of that data. You cant just take that get string and kick it off in a browser you need an access token in order to do that. You get an access token from the authentication flow.
Since you mentioned python you should be following the hello analytics tutorial which will show you how to set up your project and authenticate your script so that you can get access to the data you need.
import argparse
from apiclient.discovery import build
import httplib2
from oauth2client import client
from oauth2client import file
from oauth2client import tools
def get_service(api_name, api_version, scope, client_secrets_path):
"""Get a service that communicates to a Google API.
Args:
api_name: string The name of the api to connect to.
api_version: string The api version to connect to.
scope: A list of strings representing the auth scopes to authorize for the
connection.
client_secrets_path: string A path to a valid client secrets file.
Returns:
A service that is connected to the specified API.
"""
# Parse command-line arguments.
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
parents=[tools.argparser])
flags = parser.parse_args([])
# Set up a Flow object to be used if we need to authenticate.
flow = client.flow_from_clientsecrets(
client_secrets_path, scope=scope,
message=tools.message_if_missing(client_secrets_path))
# Prepare credentials, and authorize HTTP object with them.
# If the credentials don't exist or are invalid run through the native client
# flow. The Storage object will ensure that if successful the good
# credentials will get written back to a file.
storage = file.Storage(api_name + '.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = tools.run_flow(flow, storage, flags)
http = credentials.authorize(http=httplib2.Http())
# Build the service object.
service = build(api_name, api_version, http=http)
return service
def get_first_profile_id(service):
# Use the Analytics service object to get the first profile id.
# Get a list of all Google Analytics accounts for the authorized user.
accounts = service.management().accounts().list().execute()
if accounts.get('items'):
# Get the first Google Analytics account.
account = accounts.get('items')[0].get('id')
# Get a list of all the properties for the first account.
properties = service.management().webproperties().list(
accountId=account).execute()
if properties.get('items'):
# Get the first property id.
property = properties.get('items')[0].get('id')
# Get a list of all views (profiles) for the first property.
profiles = service.management().profiles().list(
accountId=account,
webPropertyId=property).execute()
if profiles.get('items'):
# return the first view (profile) id.
return profiles.get('items')[0].get('id')
return None
def get_results(service, profile_id):
# Use the Analytics Service Object to query the Core Reporting API
# for the number of sessions in the past seven days.
return service.data().ga().get(
ids='ga:' + profile_id,
start_date='7daysAgo',
end_date='today',
metrics='ga:sessions').execute()
def print_results(results):
# Print data nicely for the user.
if results:
print 'View (Profile): %s' % results.get('profileInfo').get('profileName')
print 'Total Sessions: %s' % results.get('rows')[0][0]
else:
print 'No results found'
def main():
# Define the auth scopes to request.
scope = ['https://www.googleapis.com/auth/analytics.readonly']
# Authenticate and construct service.
service = get_service('analytics', 'v3', scope, 'client_secrets.json')
profile = get_first_profile_id(service)
print_results(get_results(service, profile))
if __name__ == '__main__':
main()

Google Sheets API json files - What is the difference between CLIENT_SECRET and oauth2client credentials?

I followed the Google Sheet Python API Quickstart guide (https://developers.google.com/sheets/api/quickstart/python) and was able to get it working using their supplied code:
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
# If modifying these scopes, delete your previously saved credentials
# at ~/.credentials/sheets.googleapis.com-python-quickstart.json
SCOPES = 'https://www.googleapis.com/auth/spreadsheets'
CLIENT_SECRET_FILE = 'my/path/client_secret.json'
APPLICATION_NAME = 'Google Sheets API Python Quickstart'
credential_path = 'my/path/sheets.googleapis.com-python-quickstart.json'
store = Storage(credential_path)
credentials = store.get()
## !!!!! Is this needed?
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
In the default setup I downloaded two JSON files:
client_secret.JSON
downloaded to project directory.
sheets.googleapis.com-python-quickstart.JSON
downloaded to ~/.credentials directory
The sheets.googleapis.com JSON file starts with:
"_module": "oauth2client.client".
Question 1: What is the purpose for each of these JSON files?
Question 2: Are both of these JSON files needed to successfully use the Google Sheets API?
I am thinking no, as I am able to get the API working without the client_secret.JSON file.

How about this answer? I think when you know the OAuth2 process for retrieving access token and refresh token, you can understand the meaning of both files. The flow for retrieving access token and refresh token using OAuth2 is as follows.
Flow :
Download client_secret.JSON from the API Console.
client_secret.JSON includes client_id, client_secret and redirect_uris.
Retrieve an authorization code using scopes and client_id from client_secret.JSON.
Retrieve access token and refresh token using the authorization code, client_id, client_secret and redirect_uris.
Retrieved access token, refresh token and other parameters are saved to the file of sheets.googleapis.com-python-quickstart.JSON.
Note :
When you run the Quickstart for the first time, the authorization process using your browser is launched. At that time, the script of Quickstart retrieves the authorization code using client_id and scopes, and then the access token and refresh token are retrieved using the authorization code, client_id, client_secret and redirect_uris.
After the first run of the Quickstart, the access token is retrieved by the refresh token from sheets.googleapis.com-python-quickstart.JSON. By this, retrieving the authorization code using browser is not required to do. So when there is sheets.googleapis.com-python-quickstart.JSON, client_secret.JSON is not required.
I think that this leads to an answer for your Question 2.
But, if you want to change scopes and/or credentials of client_secret.JSON, the authorization process using browser and retrieving the authorization code are required to do. For this, you have to remove sheets.googleapis.com-python-quickstart.JSON and authorize again. At that time, at Quickstart, client_secret.JSON is used again.
References :
Using OAuth 2.0 to Access Google APIs
Authorization for Google Services
If this is not useful for you, I'm sorry.