I am deploying a Google Cloud Function from another Cloud Function with Python. See my code below:
import requests
import json
def make_func(request):
# Get the access token from the metadata server
metadata_server_token_url = 'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token?scopes=https://www.googleapis.com/auth/cloud-platform'
token_request_headers = {'Metadata-Flavor': 'Google'}
token_response = requests.get(metadata_server_token_url, headers=token_request_headers)
token_response_decoded = token_response.content.decode("utf-8")
jwt = json.loads(token_response_decoded)['access_token']
# Use the api to create the function
response = requests.post('https://cloudfunctions.googleapis.com/v1/projects/myproject/locations/us-central1/functions',
json={"name":"projects/my-project/locations/us-central1/functions/funct","runtime":"python37","sourceArchiveUrl":"gs://bucket/main.zip","entryPoint":"hello_world","httpsTrigger": {} },
headers={'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': 'Bearer {}'.format(jwt)} )
if response:
return 'Success! Function Created'
else:
return str(response.json())
However this function does not have "allow unauthenticated" on automatically. Thus, no requests from outside are allowed. How can I change my Python code to add this functionality when deploying the new function?
Thanks
You'll need to additionally give the allUsers member the Cloud Functions Invoker role:
from googleapiclient.discovery import build
service = build('cloudfunctions', 'v1')
project_id = ...
location_id = ...
function_id = ...
resource = f'projects/{project_id}/locations/{location_id}/functions/{function_id}'
set_iam_policy_request_body = {
'policy': {
"bindings": [
{
"role": "roles/cloudfunctions.invoker",
"members": ["allUsers"],
},
],
},
}
request = service.projects().locations().functions().setIamPolicy(
resource=resource,
body=set_iam_policy_request_body,
)
response = request.execute()
print(response)
This uses the google-api-python-client package.
In addition of Dustin answer, you have to know that the --allow-unauthenticated is for developer convenience. Under the hood it perform 2 things
Deploy your function in private mode
Add allUsers as member with Cloudfunction.invoker role
gcloud functions add-iam-policy-binding --member=allUsers --role=roles/cloudfunctions.invoker function-1
So, indeed, use the google-cloud-iam library for doing this.
In addition, your current code don't work because you use an access token to reach Cloud Function.
Indeed, you have an authorized error (401) -> You present an authorization header, but it's not authorize.
Without the header, you get a 403 error -> unauthenticated.
Anyway, you need to have a signed identity token. You have description and python code snippet here
Related
Problem :
I need to get a list of certificates of apps registered under Azure AD and renew the ones which are expiring.
I was able to get the apps related details through Microsoft Graph API > applications. But, the issue is the bearer token refreshes every time in 1 hr. Since I want this task to be automated, I need to create a fresh token always.
I got some reference of Azure SDK for identity-based authentication but the package function is returning a credential, not a token (bearer token) to be used inside the rest API header Authorization
Code:
from azure.identity import DefaultAzureCredential
default_credential = DefaultAzureCredential()
References:
Azure api or sdk to get list of app registrations and the certificates associated with them
Ok after a lot of debugging and surfing the internet, I was able to find the RestAPI way to get the bearer token.
data = {
"client_id":"add your client id",
"scope": "add scope ex: User.read Directory.read.All",
"grant_type": "password", [don't modify this one since you are providing the password]
"username": "your username",
"password": "your password",
"client_secret": "client secret"
}
headers = {
"Host": "login.microsoftonline.com",
"Content-Type": "application/x-www-form-urlencoded"
}
data = requests.post(f'https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token', data=data, headers=headers)
You will receive a json consisting of access token and related details.
Do remember to provide the permissions in the azure portal> Azure AD > app registrations > your app > API permissions (grant consent)
: )
I'm trying to create a python script which takes a (.csv with access tokens and a file) as input and uploads that file to multiple google drives whose access tokens are in that csv
but after sometime access tokens get expired and I have to get them again...just saw there's something called refresh and it refreshes access token
Is it possible to do this from python script, please explain.
Do refresh token expire?
import json
import requests
import pandas as pd
headers = {}
para = {
"name": "update",
}
files = {
'data': ('metadata', json.dumps(para), 'application/json; charset=UTF-8'),
'file': open("./update.txt", "rb")
}
tokens = pd.read_csv('tokens.csv')
for i in tokens.token:
headers={"Authorization": i}
r = requests.post(
"https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart",
headers=headers,
files=files
)
print(r.text)
In order to be able to get a new access_token programmatically using a refresh_token, you must have set access_type to offline when redirecting the user to Google's OAuth 2.0 server.
If you did that, you can get a new access_token if you do the following POST request to https://oauth2.googleapis.com/token:
POST /token HTTP/1.1
Host: oauth2.googleapis.com
Content-Type: application/x-www-form-urlencoded
client_id=your_client_id&
client_secret=your_client_secret&
refresh_token=refresh_token&
grant_type=refresh_token
The corresponding response would be something like:
{
"access_token": "1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in": 3920,
"scope": "https://www.googleapis.com/auth/drive",
"token_type": "Bearer"
}
Note:
You can find code snippets for several languages in the reference I provide below, including Python, but considering you are not using the Python library, I think the HTTP/REST snippet I provided might be more useful in your situation.
Reference:
Refreshing an access token (offline access)
I'm following the Get access without a user guide to write a Python script that will call Microsoft Graph.
This script will be scheduled from cron so it cannot get admin consent (therefore authorize using Client Credentials). I am able to successfully obtain a token using this call:
request_url = "https://login.microsoftonline.com/mytenant.onmicrosoft.com/oauth2/v2.0/token"
data = {
'Host' : 'login.microsoftonline.com',
'Content-Type' : 'application/x-www-form-urlencoded',
'client_id' : 'my-client-id-1234',
'scope' : 'https://graph.microsoft.com/.default',
'client_secret' : client_secret,
'grant_type' : 'client_credentials'
}
response = requests.post(url = request_url, data = data)
I then try to get a user listing with this call, using the valid token:
request_url = "https://graph.microsoft.com/v1.0/users"
headers = {
'Authorization' : 'Bearer ' + token,
'Host' : 'graph.microsoft.com'
}
response = requests.get(url = request_url, headers = headers)
The problem is that I get an Authorization_IdentityNotFound error:
<Response [401]>
{
"error": {
"code": "Authorization_IdentityNotFound",
"message": "The identity of the calling application could not be established.",
"innerError": {
"request-id": "2257f532-abc4-4465-b19f-f33541787e76",
"date": "2018-03-27T19:11:07"
}
}
}
These are the permissions I've selected:
Any idea how to fix this error?
For others running into this issue, I was also getting this error until found out the documentation omits a very important caveat:
For client credentials, if the app belongs to a work or school (organization) context then for https://login.microsoftonline.com/common/oauth2/token replace common with a tenantId or domain name
See
Authorization_IdentityNotFound on Microsoft Graph API request
First things first, you can go ahead an remove all those Delegated Permission scopes. If you're using the Client Credentials Grant, you will only be using Application Permission scopes.
Second, you need to execute the Admin Consent flow before you can use Client Credentials. This is done by having a Global Admin from the tenant authenticate and accept your scope request:
https://login.microsoftonline.com/common/adminconsent?client_id=[APPLICATION ID]&redirect_uri=[REDIRECT URI]
You can read more about Admin Consent here: v2 Endpoint and Admin Consent
I want to access details from Github using Github GraphQl v4 API. I found Graphene library, but I'm not sure how to authenticate with a personal access token in Python.
I tried to search on Google but couldn't found any example. It's Python library that can create graphical schema's and are not for consuming them, I tried with `requests' but failed. How can i authenticate and can find list of repositories?
I have used Github GraphQl explorer to find list of repositories via this code:
viewer {
repositories(first: 30) {
totalCount
pageInfo {
hasNextPage
endCursor
}
edges {
node {
name
}
}
}
Unlike rest, graphql has only one end point. You just need to do a POST with your query as a json object. You should provide your api_token you get from github as part of the headers.
import requests
url = 'https://api.github.com/graphql'
json = { 'query' : '{ viewer { repositories(first: 30) { totalCount pageInfo { hasNextPage endCursor } edges { node { name } } } } }' }
api_token = "your api token here..."
headers = {'Authorization': 'token %s' % api_token}
r = requests.post(url=url, json=json, headers=headers)
print (r.text)
Graphene is for building GraphQL APIs not for consuming them.
Did you see that: https://github.com/graphql-python/gql ?
It's a GraphQL client for Python.
Hope that's helpful.
As previous answers mentioned, calling GraphQL is as simple has making a POST request with the query string.
However, if you're on Python3 want something more advanced that'll also verify your queries during build and generate typed data-class response classes for you check out the new GQL library:
https://github.com/ekampf/gql
Exactly for GitHub, there is an example on using the Github GraphQL API with Python 3
https://gist.github.com/gbaman/b3137e18c739e0cf98539bf4ec4366ad
(check link as it has a lot of comments including better code for authentication)
# An example to get the remaining rate limit using the Github GraphQL API.
import requests
headers = {"Authorization": "Bearer YOUR API KEY"}
def run_query(query): # A simple function to use requests.post to make the API call. Note the json= section.
request = requests.post('https://api.github.com/graphql', json={'query': query}, headers=headers)
if request.status_code == 200:
return request.json()
else:
raise Exception("Query failed to run by returning code of {}. {}".format(request.status_code, query))
# The GraphQL query (with a few aditional bits included) itself defined as a multi-line string.
query = """
{
viewer {
login
}
rateLimit {
limit
cost
remaining
resetAt
}
}
"""
result = run_query(query) # Execute the query
remaining_rate_limit = result["data"]["rateLimit"]["remaining"] # Drill down the dictionary
print("Remaining rate limit - {}".format(remaining_rate_limit))
And there are many Python GraphQL client libraries:
https://github.com/graphql-python/gql (aka https://github.com/ekampf/gql)
https://github.com/graphql-python/gql-next
https://github.com/prodigyeducation/python-graphql-client
Official list is at https://graphql.org/code/#python
(just scroll down, client libraries are after server libraries)
When you create an instance A in Google Compute Engine, it'll get predefined, "default" service account attached to it (this basically means, that you can query google API from A, being authenticated with 'default' service account).
What I'd like to do, is to setup GCE instance with service account, that's different than a default one. This should be conceptually possible, given GCE API, but fails with exception:
{
"name": "operation-1400060483459-4f958fbc7d7b9-cd817778-b80d1cad",
"operationType": "insert",
"status": "DONE",
"user": "some_name#developer.gserviceaccount.com",
"error": {
"errors": [ {
"code": "SERVICE_ACCOUNT_ACCESS_DENIED",
"message": "The user does not have access to service account 'some_name#developer.gserviceaccount.com'"
} ] } }
Here's my code in python, which setups the instance:
discovery_service = discovery.build('compute',
config['compute_api_version'],
http=SignedJwtAssertionCredentials(
service_account_name="some_name#developer.gserviceaccount.com",
private_key=key_data,
scope='https://www.googleapis.com/auth/compute')
.authorize(httplib2.Http()))
instance = {}
# sets instance configuration details here
# ...
# ...
instance['serviceAccounts'] = [{
'email': "some_name#developer.gserviceaccount.com",
'scopes': ['https://www.googleapis.com/auth/devstorage.full_control',
'https://www.googleapis.com/auth/compute',
'https://www.googleapis.com/auth/userinfo.email', ]
}]
discovery_service.instances().insert(project=project, zone=zone, body=instance)
The weirdest part of it, is that exception says "The user does not have access to service account 'some_name#developer.gserviceaccount.com'", but the "user" it refers to is the 'some_name#developer.gserviceaccount.com' itself! Which means 'some_name#developer.gserviceaccount.com' does not have access to 'some_name#developer.gserviceaccount.com', which makes no sense.
I believe you'll need to create a new service account to use the API from a non-GCE instance. The service account you're referencing works within a GCE instance only.
To do that go to the Cloud Console > Project > APIs & Auth > Credentials.
Create new Client ID
Service Account
Download the .p12 file and load that as the private key. (See example below)
Also you'll need to create an instance from a boot disk which is typically created from one of the GCE supplied images.
Here's an example using JSON Web Tokens that worked for me. It was adapted from the docs located here: https://cloud.google.com/compute/docs/tutorials/python-guide#addinganinstance.
from apiclient import discovery
from oauth2client.file import Storage
from oauth2client.client import SignedJwtAssertionCredentials
import httplib2
import os.path
INSTANCE_NAME = 'my-instance'
API_VERSION = 'v1'
GCE_URL = 'https://www.googleapis.com/compute/%s/projects/' % (API_VERSION)
PROJECT_ID = '***'
SERVICE_ACOUNT_CLIENT_ID = '***.apps.googleusercontent.com'
SERVICE_ACCOUNT_EMAIL_ADDRESS = '***#developer.gserviceaccount.com'
GCE_SCOPE = 'https://www.googleapis.com/auth/compute'
ZONE = 'us-central1-a'
DEFAULT_SERVICE_EMAIL = 'default'
DEFAULT_SCOPES = ['https://www.googleapis.com/auth/devstorage.full_control',
'https://www.googleapis.com/auth/compute']
SOURCE_IMAGE_URL = 'projects/ubuntu-os-cloud/global/images/ubuntu-1410-utopic-v20141217'
def main():
f = file('private-key.p12', 'rb')
oauth_key_data = f.read()
f.close()
http = httplib2.Http()
oauth_storage = Storage('compute-creds.dat')
oauth_credentials = oauth_storage.get()
if oauth_credentials is None or oauth_credentials.invalid:
oauth_credentials = SignedJwtAssertionCredentials(
service_account_name=SERVICE_ACCOUNT_EMAIL_ADDRESS,
private_key=oauth_key_data,
scope=GCE_SCOPE)
oauth_storage.put(oauth_credentials)
else:
oauth_credentials.refresh(http)
http = oauth_credentials.authorize(http)
gce_service = discovery.build('compute', 'v1', http=http)
project_url = '%s%s' % (GCE_URL, PROJECT_ID)
image_url = '%s%s/global/images/%s' % (
GCE_URL, 'ubuntu-os-cloud', 'ubuntu-1410-utopic-v20141217')
machine_type_url = '%s/zones/%s/machineTypes/%s' % (
project_url, ZONE, 'n1-standard-1')
network_url = '%s/global/networks/%s' % (project_url, 'default')
instance = {
'name': INSTANCE_NAME,
'machineType': machine_type_url,
'disks': [{
'autoDelete': 'true',
'boot': 'true',
'type': 'PERSISTENT',
'initializeParams' : {
'diskName': INSTANCE_NAME,
'sourceImage': SOURCE_IMAGE_URL
}
}],
'networkInterfaces': [{
'accessConfigs': [{
'type': 'ONE_TO_ONE_NAT',
'name': 'External NAT'
}],
'network': network_url
}],
'serviceAccounts': [{
'email': DEFAULT_SERVICE_EMAIL,
'scopes': DEFAULT_SCOPES
}]
}
# Create the instance
request = gce_service.instances().insert(
project=PROJECT_ID, body=instance, zone=ZONE)
response = request.execute(http=http)
response = _blocking_call(gce_service, http, response)
print response
def _blocking_call(gce_service, auth_http, response):
"""Blocks until the operation status is done for the given operation."""
status = response['status']
while status != 'DONE' and response:
operation_id = response['name']
# Identify if this is a per-zone resource
if 'zone' in response:
zone_name = response['zone'].split('/')[-1]
request = gce_service.zoneOperations().get(
project=PROJECT_ID,
operation=operation_id,
zone=zone_name)
else:
request = gce_service.globalOperations().get(
project=PROJECT_ID, operation=operation_id)
response = request.execute(http=auth_http)
if response:
status = response['status']
return response
main()
FYI: in GCE you usually get two default service accounts:
-compute#developer.gserviceaccount.com
#cloudservices.gserviceaccount.com
Note the different Email suffix (developer.gserviceaccount.com vs. cloudservices.gserviceaccount.com). It appears that using your own service account, EVEN if it has the Owner role, does not grant you access to the <number>#cloudservices.gserviceaccount.com account, only to the 1st one (<number>-compute#developer.gserviceaccount.com).
In my case, I got the aforementioned error when trying to create an instance with my own service account while specifing that the instance will use the 2nd service account from above. Once I fixed the call to request that the instance will use the 1st account, it worked.