We Keep Coding

How to get Google OAUTH Token without using Gcloud Command Line

I am currently using the following code to get the OAUTH Token
command = 'gcloud auth print-access-token'
result = str(subprocess.Popen(command, universal_newlines=True, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate())
The result variable has the OAUTH Token. This technique uses my current logged in gcloud config.
However, I am looking out for a way to get the OAUTH Token without using command line.
I am using this OAUTH Token to make CDAP calls to get the Google Dataflow Pipeline Execution Details.
I checked some google blogs. This is the one I think should try but it asks to create consent screen and it will require one time activity to provide consent to the scopes defined and then it should work.
Google Document
Shall I follow steps in above document and check OR is there any other way we can get the OAUTH Token?
Is there a way to get authentication done by service account instead of google user account and get the OAUTH Token?

For automated process, service account is the recommended way. You can use the google-oauth library for this. You can generate an access token like this
# With default credential (your user account or the Google Cloud Component service account.
# Or with the service account key file defined in the GOOGLE_APPLICATION_CREDENTIALS env var -> for platform outside GCP)
credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
# With service account key file (not recommended)
# credentials = service_account.Credentials.from_service_account_file('service-account.json',
# scopes=["https://www.googleapis.com/auth/cloud-platform"])
from google.auth.transport import requests
credentials.refresh(requests.Request())
print(credentials.token)
However, if you want to call Google cloud APIs, I recommend you to use authorized request object
Here an example of BigQuery call. You can use service account key file to generate your credential as in my previous example.
base_url = 'https://bigquery.googleapis.com'
credentials, project_id = google.auth.default(scopes=['https://www.googleapis.com/auth/cloud-platform'])
project_id = 'MyProjectId'
authed_session = AuthorizedSession(credentials)
response = authed_session.request('GET', f'{base_url}/bigquery/v2/projects/{project_id}/jobs')
print(response.json())
EDIT
When you want to use Google APIs, a service account key file is not needed (and I recommend you to not use it) on your computer and on GCP component. The Application Default Credential is always sufficient.
When you are in your local environment, you must run the command gcloud auth application-default login. With this command, you will register your personal account as default credential when you run locally your app. (of course, you need to have your user account email authorized on the component that you call)
When you are on GCP environment, each component have a default service account (or you can specify one with you configure your component). Thanks to the component "identity", you can use the default credential. (of course, you need to have the service account email authorized on the component that you call)
ONLY when you run an app automatically and outside GCP, you need a service account key file (for example, in your CI/CD other that Cloud Build, or in an app deployed on other Cloud Provider or on premise)
Why service account key file is not recommended? It's at least my recommendation because this file is ..... a file!! That's the problem. You have a way to authenticate a service account in a simple file: you have to store it securely (it's a secret and an authentication method!!), you can copy it, you can send it by email, you can even commit it in a public GIT repository... In addition, Google recommend to rotate them every 90 days, so it's a nightmare to rotate, to trace and to manage

How can I trigger an HTTP Cloud Function from within Google App Engine?

I have a Flask app in Google App Engine Standard Environment Python, and I also have a Cloud Function with an HTTP trigger which accepts a JSON body including the URL of a file. The CF downloads the file at that URL then saves it to a GCS bucket. The GAE service account has Cloud Function Invoker permissions, yet when using urlfetch.fetch() in my GAE code to trigger CF, the App Engine code gets a 403 Forbidden error unless I make the CF trigger callable by anyone.
How do I successfully call/trigger CF's from GAE in Python? I assume the answer is one of these:
Set IAM permissions on GAE service account to {enlighten me here}
Add authentication headers in urlfetch.fetch() like so {different enlightenment}
Make CF triggerable from anywhere, but hard code some secret key so the CF code itself handles authentication.

It's well documented here: Cloud Functions Authentication
In short you have to provide your service account credentials in the authentication header.
To get your credentials use the Google Auth Client library. If you are testing from local you should create a service account JSON and load it to the environment variable GOOGLE_APPLICATION_CREDENTIALS but on App Engine it will work from scratch.
After you have gotten your token, pass it as an auth header like so:
auth_req = google.auth.transport.requests.Request()
auth_token = google.oauth2.id_token.fetch_id_token(auth_req, cloud_function_url)
response = requests.post(cloud_function_url, json=payload, headers={"Authorization" : f"Bearer {auth_token}"})

How do you refresh an authentication token with Authlib and GSpread?

I'm using gspread in a python app to access some backend Google Sheets that act as a database for the application. The user is required to log into the app using an in app browser and an authlib OAuth2 session is created using the token returned in the url via this method. The initial log in and access works fine, but when the token times out after an hour I access the sheets anymore and the app breaks.
I can't work out how to refresh the token using authlib, the reason we picked the library is because it integrated with gspread and the it was supposed to autorefresh auth tokens for you. The code that I use for login is below but after an hour the gspread function open_by_key() fails because of the authentication problem.
I have tried recreating the gspread client, but that just uses the same token. I feel like there should be a way to refresh the token to extend its life for another hour but I can't find how to do that with authlib.
creds = OAuth2Session(clientId, clientSecret, scope=scope, redirect_uri=redirectUrl)
authUrl, state = creds.create_authorization_url('https://accounts.google.com/o/oauth2/v2/auth', response_type='token')
Load the authURL in the browser and get user to log in. Browser returns the authentication response as urlString
creds.fetch_access_token(authorization_response=urlString)
gc = gspread.Client(None, creds)
ss = gc.open_by_key(sheetKey)
todaysSheetName = datetime.datetime.now().strftime('%d-%m-%Y')
wks = ss.worksheet(todaysSheetName)

authUrl, state = creds.create_authorization_url('https://accounts.google.com/o/oauth2/v2/auth', response_type='token')
According to this code you are using, response_type=token is an implicit grant type flow. I guess there won't be a refresh_token in token with implicit flow. Here are my suggestions:
try change it to authorization code flow
use a database to save tokens
if the token is expired, use your previously saved token's refresh_token to exchange a new token.
The refresh_token method is simple:
refresh_token(url=None, refresh_token=None, body='', auth=None, headers=None, **kwargs)
Using your code, it should be:
refresh_token = your_previous_token.refresh_token
new_token = creds.refresh_token('https://accounts.google.com/...', refresh_token=refresh_token)
The problem is that there is no documentation on this part. But according to the API, it would be used like this. If it is not working, you can report it to Authlib issue tracker.

How to get authenticated identity response from AWS Cognito using boto3

I would like to use boto3 to get temporary credentials for access AWS services. The use case is this: A user in my Cognito User Pool logs in to my server and I want the server code to provide that user with temporary credentials to access other AWS services.
I have a Cognito User Pool where my users are stored. I have a Cognito Identity Pool that does NOT allow unauthorized access, only access by users from the Cognito User Pool.
So here is the code I am starting with:
import boto3
client = boto3.client('cognito-identity','us-west-2')
resp = client.get_id(AccountId='<ACCNTID>',
IdentityPoolId='<IDPOOLID>')
However, just running these three lines of code throws an exception:
botocore.errorfactory.NotAuthorizedException: An error
occurred (NotAuthorizedException) when calling
the GetId operation: Unauthenticated access is not
supported for this identity pool.
Since my Cognito Identity Pool is not set up for unauthenticated access, it seems that I cannot call get_id until I somehow authenticate somewhere.
How do I solve this? What exactly do I need to do to authenticate so I can call get_id?
UPDATE: Looks like I need to pass a Logins field and data to the get_id function call, but to do that I need the login JWT token. If I am running this inside a webapp (eg a Django backend) where I use the AWS Cognito prepackaged login screens, then yes I can get this from the homepage URL after redirection from successful login. But now I am writing some test scripts that have nothing to do with a website. Is there a way to use boto or boto3 or some other python package to login with username and password and get JWT token?

Just to add to the answer from Arka Mukherjee above, to get the token I do this:
auth_data = { 'USERNAME':username , 'PASSWORD':password }
provider_client=boto3.client('cognito-idp', region_name=region)
resp = provider_client.admin_initiate_auth(UserPoolId=user_pool_id, AuthFlow='ADMIN_NO_SRP_AUTH', AuthParameters=auth_data, ClientId=client_id)
token = resp['AuthenticationResult']['IdToken']
Here I have to use the username and password of the Cognito user, client_id is the app client id for the app client that I set up thru Cognito, and user_pool_id is the user pool id.
Note that my app client has this option checked/selected: Enable sign-in API for server-based authentication (ADMIN_NO_SRP_AUTH) and I created that app client with no secret key (apparently that is important for web clients especially).

To pass the Cognito User Pool JWT Token, you would need to use the Logins Map in the GetId API call. You could try the following Python code out on your end, after replacing the necessary placeholders.
response = client.get_id(
AccountId='string',
IdentityPoolId='string',
Logins={
'cognito-idp.<region>.amazonaws.com/<YOUR_USER_POOL_ID>': '<JWT ID Token>'
}
)
If you do not provide a Logins Map, Amazon Cognito treats the authentication event as Unauthenticated, and hence, you are facing this error.

how to obtain GCR access token with python / listing docker images

The access token im getting with gcloud auth print-access-token is obviously a different access token than the one i can get with some basic python code:
export GOOGLE_APPLICATION_CREDENTIALS=/the-credentials.json
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
credentials.get_access_token()
What i am trying to do is get a token that would work with:
curl -u _token:<mytoken> https://eu.gcr.io/v2/my-project/my-docker-image/tags/list
I'd prefer not to install gcloud utility as a dependency for my app, hence my tries to obtain the access token progrmatically via oath google credentials

I know this is a very old question, but I just got faced with the exact same problem of requiring an ACCESS_TOKEN in Python and not being able to generate it, and managed to make it work.
What you need to do is to use the variable credentials.token, except it won't exist once you first create the credentials object, returning None. In order to generate a token, the credentials must be used by a google.cloud library, which in my case was done by using the googleapiclient.discovery.build method:
sqladmin = googleapiclient.discovery.build('sqladmin', 'v1beta4', credentials=credentials)
response = sqladmin.instances().get(project=PROJECT_ID, instance=INSTANCE_ID).execute()
print(json.dumps(response))
After which the ACCESS_TOKEN could be properly generated using
access_token = credentials.token
I've also tested it using google.cloud storage as a way to test credentials, and it also worked, by just trying to access a bucket in GCS through the appropriate Python library:
from google.oauth2 import service_account
from google.cloud import storage
PROJECT_ID = your_project_id_here
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = '/path/to/service.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
try:
list(storage.Client(project=PROJECT_ID, credentials=credentials).bucket('random_bucket').list_blobs())
except:
print("Failed because no bucket exists named 'random_bucket' in your project... but that doesn't matter, what matters is that the library tried to use the credentials and in doing so generated an access_token, which is what we're interested in right now")
access_token = credentials.token
print(access_token)

So I think there are a few questions:
gcloud auth print-access-token vs GoogleCredentials.get_application_default()
gcloud doesn't set application default credentials by default anymore when performing a gcloud auth login, so the access_token you're getting from gcloud auth print-access-token is going to be the one corresponding to the used you used to login.
As long as you follow the instructions to create ADC's for a service account, that account has the necessary permissions, and the environment from which you are executing the script has access to the ENV var and the adc.json file, you should be fine.
How to make curl work
The Docker Registry API specifies that a token exchange should happen, swapping your Basic auth (i.e. Authorization: Basic base64(_token:<gcloud_access_token>)) for a short-lived Bearer token. This process can be a bit involved, but is documented here under "How to authenticate" and "Requesting a Token". Replace auth.docker.io/token with eu.gcr.io/v2/token and service=registry.docker.io with service=eu.gcr.io, etc. Use curl -u oauth2accesstoken:<mytoken> here.
See also: How to list images and tags from the gcr.io Docker Registry using the HTTP API?
Avoid the question entirely
We have a python lib that might be relevant to your needs:
https://github.com/google/containerregistry