I have been trying (with little success) to have a google cloud function be triggered via an http request from a google sheet (google apps script) and it seemingly won't work. A few important things:
The function should only run if the user comes from my organization
The user should not have to be invited to the GCP project
I know this can be done very easily in google colabs and Python. The following script will let a user in my organization who is not in the GCP project trigger the cloud function:
import requests
import google.auth
from google.auth.transport.requests import Request
from google.colab import auth
credentials, project_id = google.auth.default()
request = Request()
credentials.refresh(request=request)
GCF_URL = ''https://project_location-project_id.cloudfunctions.net/name-of-your-funciton''
resp = requests.get(GCF_URL, headers={'Authorization': f'Bearer {credentials.id_token}'})
This will work and trigger the cloud function for any users inside my organization but does not work for my personal email for example.
Now, I would like to replicate this behvaiour inside a google apps script such that an end user with access to that sheet can trigger the cloud function as long as they are a member of my organization.
I have tried some things I have seen online such as this example:
function callExternalUrl() {
var url = 'https://project_location-project_id.cloudfunctions.net/name-of-your-funciton';
var oauthToken = ScriptApp.getOAuthToken(); // get the user who's logged into Google Sheet's OAuth token
const data = {
oauthToken, // 1. add the oauth token to the payload
activeUser: param.user // 2. this here is important as it adds the userinfo.email scope to the token
// any other data you need to send to the Cloud Function can be added here
};
var options = {
'method' : 'get', // or post, depending on how you set up your Cloud Function
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
'payload' : JSON.stringify(data)
};
const response = UrlFetchApp.fetch(url, options);
Logger.log('Response Code: ' + response.getResponseCode());
}
This gives a 403 error but if I change it up so that it gives the OAuth on the correct format like this:
function callExternalUrl() {
var url = 'https://project_location-project_id.cloudfunctions.net/name-of-your-funciton';
var oauthToken = ScriptApp.getOAuthToken(); // get the user who's logged into Google Sheet's OAuth token
var response = UrlFetchApp.fetch(url, {
headers: {
Authorization: 'Bearer ' + oauthToken
}
});
// const response = UrlFetchApp.fetch(url, options);
Logger.log('Response Code: ' + response.getResponseCode());
}
I get a 401 (i.e. the authorization failed). Now, it seems that I simply have to get the correct authentication from the users to send in this request for it to work. I have seen this github repo that focuses on getting OAuth2 from google apps scripts (https://github.com/gsuitedevs/apps-script-oauth2), but I can't seem to get that to work either, it would have to be adapted to cloud in some way I am unaware of.
I have read
Securely calling a Google Cloud Function via a Google Apps Script
which is very similar but it did not seem to get to the root of the problem, any input on how to make this process possible?
Related
Do we have a solution in python to list job runs that are created using Azure VM managed identity in Azure Databricks.
Appreciate the help!
I am getting http 403 error when using managed identity library in python
from azure.identity import ManagedIdentityCredential
credential = ManagedIdentityCredential()
# Obtain an access token
from azure.identity import DefaultAzureCredential
credentials = DefaultAzureCredential()
access_token = credentials.get_token("management.azure.com/")
headers = { 'Authorization': 'Bearer ' + access_token,
'Content-Type': 'application/json' }
# Set the URL for the Databricks REST API
endpoint url = "databricks_url" + '/api/2.0/clusters/list'
# Make the REST API call to the Databricks endpoint
response = requests.get(url, headers=headers)
print(response.json())
If managed identity isn't added into the Databricks workspace yet, then having only access token isn't enough - you also need to provide an additional access token for accessing Azure management API (the https://management.core.windows.net/ resource URL), and it should be provided as the X-Databricks-Azure-SP-Management-Token header, together with Databricks Workspace Resource ID as the X-Databricks-Azure-Workspace-Resource-Id.
And then you need to generate the access token to the for Databricks workspace resource (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) that should be sent as bearer token.
This specific scenario is described in the details the official documentation.
I have a public Cloud Run, authenticated by JWT Token. Working 100%.
The logic inside the Cloud Run to decode the token is in python:
def decode_jwt(token: str) -> dict:
try:
decoded_token = jwt.decode(
token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
return decoded_token if decoded_token["expires"] >= time.time() else None
except Exception as e:
raise InvalidTokenError
The Cloud Run is publicly available using a custom domain.
Now, I want to do some requests to the Cloud Run, using Cloud Tasks (each request have different parameters, created previously by a Cloud Functions).
In the Cloud Tasks, I create each task with a "Bearer {token}" parameter
Cloud Task Headers Code:
task["http_request"]["headers"] = \
{"Authorization": f"Bearer {token}",
"Accept": "application/json"}
First situation:
When I create the task without the "oidc_token" parameter in the http_request creation.
Cloud Run returns "403 Forbidden", and never reach the decode_jwt function inside cloud run.
Cloud Task http_request Code:
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": url,
}
}
Second situation:
I add an "oidc_token".
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": url,
"oidc_token": {
"service_account_email": "service-task#xxxxx.iam.gserviceaccount.com",
}
}
Now, the request reach the Cloud Run decode_jwt function, and the log in Cloud Run returns "InvalidTokenError".
Extra: I added a logging.info to expose the token received in Cloud Run, and is not the token I passed in the Cloud Task Creation.
Problem Summary:
you have a public (allUsers) Cloud Run service.
you have created your own authorization mechanism (HS256 - HMAC with SHA-256).
you want to assign a custom token for the HTTP Authorization Bearer value.
Cloud Run authorization is managed by IAP.
Authorization for the Cloud Run service is managed by the Identity Aware Proxy (IAP). If you add an HTTP Authorization Bearer token, IAP will verify that token. That step fails for your custom token which results in an HTTP 403 Forbidden error.
Cloud Tasks supports two types of HTTP Authorization Bearer tokens. OAuth Access tokens and OIDC Identity tokens. You cannot use your own token value to replace the supported types.
That leaves you with two options:
Enhance your code to support Google signed OIDC Identity Tokens.
Use a custom HTTP header that supports your custom token format.
Note: I do not recommend using HS256. HS256 is a symmetric algorithm which means the secret must be known to both sides in order to validate the payload. RS256 is an asymmetric algorithm which uses private/public key pairs. To verify only requires the public key. This is one of the strong design features of Google's use of private keys for service accounts and identities. If you switch to Google's method, all of the hard work is done for you.
You have to specificy the audience of your Cloud Run service, like that
task = {
"http_request": { # Specify the type of request.
"http_method": tasks_v2.HttpMethod.POST,
"url": url, # The full url path that the task will be sent to.
"oidc_token": {
"service_account_email": "service-task#xxxxx.iam.gserviceaccount.com",
"audience": base url of Cloud Run, no /sub/path
}
}
How can I authenticate to Azure DevOps REST API in a python script?
I found that there are 2 methods :
Using personal access token (PAT)
Using OAuth 2.0
I am using the second method. Followed the steps in this documentation:
https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/oauth?view=azure-devops
I wrote this function to autherize to azure DevOps using OAuth 2.0:
def get_authenticated():
client_id = < my client ID as a string >
state = "user1"
scope = "vso.graph_manage%20vso.identity_manage%20vso.profile_write%20vso.project_manage%20vso.tokenadministration%20vso.tokens"
callback_URL = < Callback URL to my azure devops account >
# Azure DevOps Services authorization endpoint
Auth_URL = "https://app.vssps.visualstudio.com/oauth2/authorize?client_id=" + client_id + "&response_type=Assertion&state=" + state + "&scope=" + scope + "&redirect_uri=" + callback_URL
headers = {'Accept': 'application/json;api-version=1.0'}
print(Auth_URL)
response = requests.get(Auth_URL,headers = headers)
print(response)
print(response.status_code)
print(response.headers['content-type'])
response.raise_for_status()
But when calling this function, output I am getting is:
<Response [203]>
203
text/html; charset=utf-8
The auth URL is correct because when I tried to access the same URL in a browser it successfully redirects to a form to enter azure user credentials.
The expected behavior of the script is, when the auth_url is requested, Azure DevOps Services should ask the user to authorize. I think that should be done by prompting for username&password in terminal/via a browser.
I am totally new to python scripting and REST APIs.
Can someone help me by pointing out the faults in my code or pointing to some samples?
The http error 203 indicates that the returned metainformation is not a definitive set of the object from a server with a copy of the object, but is from a private overlaid web. In your code,you added headers = {'Accept': 'application/json;api-version=1.0'}, but in fact the content type should be application/x-www-form-urlencoded.
You can use some OAuth2 library for python to authenticate to Azure DevOps REST API, such as OAuthLib. It includes sevelral samples.
Also, you can refer to following topic, hope it is helpful for you.
Tutorial for using requests_oauth2
Based on this Google documentation I can generate the token for Computer Vision API request by calling this in terminal gcloud auth application-default print-access-token. However, I am going to call the request from my python code and I try to generate from Python code, something like below...
The code is based on this documentation page
with open( environ.get(KEY_ENV_VARIABLE) ) as f:
key = json.load(f)
iat = time.time()
exp = iat + 3600
payload = {
'iss': key.get('client_email'),
'sub': key.get('client_email'),
'aud': 'https://vision.googleapis.com/google.cloud.automl_v1beta1',
'iat': iat,
'exp': exp
}
additional_headers = { "kid": key.get("private_key_id") }
signed_jwt = jwt.encode(payload, key.get("private_key"), headers=additional_headers, algorithm='HS256')
return signed_jwt.decode('utf-8')
It does generate the token however it is different in terms of length, compared to the token generated by gcloud tool.
I know that the easiest and quick dirty fix would be calling os.system('gcloud auth application-default print-access-token'). However, I do not want to do the dirty way if possible and want to generate the token in correct way.
Try following this documentation to download a service account. After you get a key, you'll want to set GOOGLE_APPLICATION_CREDENTIALS to the file path of the key.
I want to retrieve realtime user data from Google Analytics and this article by Google outlines how to this (and works as expected).
gapi.client.analytics.data.realtime.get({
"ids": "ga:21660971",
"metrics": "rt:activeUsers"
})
However I am following the server-side authentication because I want this to be always visible to users who do not have access. But overnight this access token has become invalid. What can I do or rework to make this work with the need to constantly have manual intervention to refresh the token all of the time!
Python script (as is in the Google article).
# service-account.py
import time
from oauth2client.service_account import ServiceAccountCredentials
# The scope for the OAuth2 request.
SCOPE = 'https://www.googleapis.com/auth/analytics.readonly'
# The location of the key file with the key data.
KEY_FILEPATH = 'MY-JSON-FILE.json'
# Defines a method to get an access token from the ServiceAccount object.
def get_access_token():
return ServiceAccountCredentials.from_json_keyfile_name(
KEY_FILEPATH, SCOPE).get_access_token().access_token
print(get_access_token())
JS Authentification
/**
* Authorize the user with an access token obtained server side.
*/
gapi.analytics.auth.authorize({
'serverAuth': {
'access_token': 'TOKEN-FROM-PY-SCRIPT-ABOVE'
}
});
This is the error given in the console, and it only does this after 60 minutes or so.
{domain: "global", reason: "authError", message: "Invalid
Credentials", locationType: "header", location: "Authorization"}
Array(0) message
"Invalid Credentials"