Is there a way to get the email of the Runtime Service Account programmatically from a Cloud Function?
I know I can "guess" the default App Engine account (since it is always <project-id>#appspot.gserviceaccount.com), but this is not what I want.
I was expecting to have some Environment Variable or something with this information, but I was not able to find any.
For older runtimes (Node.js 8, Python 3.7 and Go 1.11), you can use FUNCTION_IDENTITY environment variable, as described here.
Example in Python:
import os
service_account_email = os.environ.get('FUNCTION_IDENTITY')
For newer runtimes, you need to query the Metadata Server, as in this example:
import requests
def query_metadata(entry):
response = requests.get('http://metadata.google.internal/computeMetadata/v1/' + entry, headers={'Metadata-Flavor': 'Google'})
return response.content.decode("utf-8")
service_account_email = query_metadata('instance/service-accounts/default/email')
You can also write a function that supports both runtimes (older and newer) using:
service_account_email = os.environ.get('FUNCTION_IDENTITY') or query_metadata('instance/service-accounts/default/email')
For a list of metadata endpoints that you can use to query for information, check here.
If you're using the newer runtimes, you can get the runtime service account of your Cloud Function programmatically using REST API. Here's an example based from this answer:
import requests
import json
def get_sa():
FUNCTION_NAME = 'func_name'
PROJECT_ID = 'proj_id'
REGION = 'us-central1'
# Get the access token from the metadata server
metadata_server_token_url = 'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token?scopes=https://www.googleapis.com/auth/cloud-platform'
token_request_headers = {'Metadata-Flavor': 'Google'}
token_response = requests.get(metadata_server_token_url, headers=token_request_headers)
token_response_decoded = token_response.content.decode("utf-8")
access_token = json.loads(token_response_decoded)['access_token']
# Call functions.get() to retrieve Cloud Functions information
response = requests.get('https://cloudfunctions.googleapis.com/v1/projects/{}/locations/{}/functions/{}'.format(PROJECT_ID, REGION, FUNCTION_NAME),
headers={
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': 'Bearer {}'.format(access_token)
})
print(response.json()['serviceAccountEmail'])
get_sa()
Note that you can only get an access token from the metadata server if your app is deployed in GCP (Compute Engine, Cloud Functions, etc.). If your app is on your local machine, you need to use a service account and an auth library to generate the access token. Here's the auth library reference if you're using Python.
To learn more about the given REST API method, see projects.locations.functions.get().
Related
I'm hosting a Flask web app on Cloud Run. I'm also using Secret Manager to store Service Account keys. (I previously downloaded a JSON file with the keys)
In my code, I'm accessing the payload then using os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = payload to authenticate. When I deploy the app and try to visit the page, I get an Internal Service Error. Reviewing the logs, I see:
File "/usr/local/lib/python3.10/site-packages/google/auth/_default.py", line 121, in load_credentials_from_file
raise exceptions.DefaultCredentialsError(
google.auth.exceptions.DefaultCredentialsError: File {"
I can access the secret through gcloud just fine with: gcloud secrets versions access 1 --secret="<secret_id>" while acting as the Service Account.
Here is my Python code:
# Grabbing keys from Secret Manager
def access_secret_version():
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the secret version.
name = "projects/{project_id}/secrets/{secret_id}/versions/1"
# Access the secret version.
response = client.access_secret_version(request={"name": name})
payload = response.payload.data.decode("UTF-8")
return payload
#app.route('/page/page_two')
def some_random_func():
# New way
payload = access_secret_version() # <---- calling the payload
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = payload
# Old way
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "service-account-keys.json"
I'm not technically accessing a JSON file like I was before. The payload variable is storing entire key. Is this why it's not working?
Your approach is incorrect.
When you run on a Google compute service like Cloud Run, the code runs under the identity of the compute service.
In this case, by default, Cloud Run uses the Compute Engine default service account but, it's good practice to create a Service Account for your service and specify it when you deploy it to Cloud Run (see Service accounts).
This mechanism is one of the "legs" of Application Default Credentials when your code is running on Google Cloud, you don't specify the environment variable (you also don't need to create a key) and Cloud Run service acquires the credentials from the Metadata service:
import google.auth
credentials, project_id = google.auth.default()
See google.auth package
It is bad practice to define|set an environment variable within code. By their nature, environment variables should be provided by the environment. Doing this with APPLICATION_DEFAULT_CREDENTIALS means that your code always sets this value when it should only do this when the code is running off Google Cloud.
For completeness, if you need to create Credentials from a JSON string rather than from a file contain a JSON string, you can use from_service_account_info (see google.oauth2.service_account)
I was told to move my bash script that reports on VM backup status, also reports VMs that are not being backed up to Azure automation account. I picked python since Automation Account doesn't have bash and I have done python scripts before for system admin purposes. I am not a python developer, and I need help navigate Azure python SDK classes.
I need to find the "Backup Items" in portal from one of the python SDK modules, to retrieve the VM information from backup vault. I've tried azure.mgmt.recoveryservices and azure.mgmt.recoveryservicesbackup. I can get vault information from azure.mgmt.recoveryservices, which I can use to query more information about vault, hopefully the VM information. My guess is azure.mgmt.recoveryservicesbackup. But I am lost in azure.mgmt.recoveryservicesbackup.jobs.models. I can't tell among hundreds of classes, which one would give that information.
I'd like to use the output from vault backup to against the list of VMs, to find out which ones are not being backed up.
I've looked: https://learn.microsoft.com/en-us/python/api/azure-mgmt-recoveryservicesbackup/azure.mgmt.recoveryservicesbackup.activestamp?view=azure-python https://azure.github.io/azure-sdk/releases/latest/all/python.html, https://www.programcreek.com/python/?ClassName=azure.mgmt.recoveryservicesbackup&submit=Search.
any help would much appreciated!
thanks.
Using python to get the backup of VM
You can use the below code snippet to get VM Backup details.
from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.compute import ComputeManagementClient
from azure.mgmt.network import NetworkManagementClient
import requests
SUBSCRIPTION_ID = '<Your Subscription ID>'
VM_NAME = '<VM Name>'
credentials = ServicePrincipalCredentials(
client_id='<Client id>',
secret='<Client Secret>',
tenant='<Your Tenent Id>'
)
# Creating base URL
BASE_API_URL = "https://management.azure.com/Subscriptions/<Subscription>/resourceGroups/<Resourece group name>/providers/Microsoft.RecoveryServices/vaults/your_vault_name/backupProtectedItems?api-version=2019-05-13&"
# Add the required filter to fetch the Exact details
customFilter="$filter=backupManagementType eq 'AzureIaasVM' and itemType eq 'VM' and policyName eq 'DailyPolicy'"
#Adding the Base API url with custom filter
BASE_URL = BASE_API_URL + customFilter
header = {
"Authorization": 'Bearer '+ credentials.token["access_token"]
}
response = requests.get(BASE_URL, headers=header)
# here you can handle the response to know the details of backup
print(response.content)
...
Refer here to achieve using Azure cli.
what I was looking for is in "backup_protected_item", in RecoveryServicesBackupClient constructor, here is sample code.
from azure.mgmt.recoveryservicesbackup import RecoveryServicesBackupClient
backup_items = backup_client.backup_protected_items.list(resource_group_name='rg_xxx', vault_name=var_vault)
print(backup_items.properties)
I need to call other cloud functions/cloud run services from my cloud function. I would like to be authenticated if possible so to that effect I have been looking into how to create an AuthorizedSession using the credentials I get back from google.auth.default. My current code looks like so:
credentials, _ = google.auth.default(scopes=[
SERVICE_A_URL,
SERVICE_B_URL,
SERVICE_C_URL,
])
return AuthorizedSession(credentials)
When running this I get the following:
google.auth.exceptions.RefreshError: ('No access token in response.', {'id_token': '[ID_TOKEN]'})
Does anyone know how to get the AuthorizedSession to accept my credentials?
Authorized session works well with Google Cloud APIs which expect an access token. In the case of Cloud Functions and Cloud Run (and also App Engine behind IAP) you need to provide an identity token.
So, you can't with authorized session, you need to generate and ID token with the correct audience and then to add it in the authorization header of your request.
credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
credentials.with_target_audience("SERVICE_A_URL")
from google.auth.transport import requests
credentials.refresh(requests.Request())
print(credentials.token)
request.get("SERVICE_A_URL+Parameters", headers={'Authorization': 'bearer {}'.format(credentials.token)})
I'm looking for a way to trigger an Azure Function which will run some Python code, each time a new virtual machine is created. I have already done the same thing in AWS using CloudWatch + Lambda, but I can't find where/how achieve the same thing in Azure.
I have tried to use Logic App with Event Grid but there is no trigger to monitor VM state.
Anyone could provide me with some guidance here ?
Many thanks in advance.
Azure service don't have built-in method to achieve your requirement, but I think you can achieve this by your own python code. The main logic is to polling the VM names from your subscription and then store the VM names in somewhere, if they changes, post a request to something like 'HttpTrigger' endpoint(Or just put the logic in the polling algorithm.).
And the for the polling algorithm, you can design by yourself or just use the 'TimeTrigger' to achieve.
I notice you add the 'Python' tag, so just use code like below and put them inside a polling algorithm:
import requests
from azure.identity import ClientSecretCredential
import json
client_id = 'xxx'
tenant_id = 'xxx'
client_secret = 'xxx'
subscription_id = 'xxx'
credential = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret)
accesstoken = str(credential.get_token('https://management.azure.com/.default'))[19:1287]
bearertoken = "Bearer "+accesstoken
r = requests.get("https://management.azure.com/subscriptions/"+subscription_id+"/resources?$filter=resourceType eq 'Microsoft.Compute/virtualMachines'&api-version=2020-06-01",headers={'Authorization': bearertoken})
items = json.loads(r.text)
print(r.text)
for item in items['value']:
print(item['name'])#This line is print, you need to store this in some place such as database, azure blob storage, azure table storage etc.
#check the VM names here. If some VM been added, post a request to the HttpTrigger function.
If you use azure function 'Time Trigger' instead of self-designed algorithm, then you can store the client id, tenent id, client_secret and subscription id to the keyvault and then let your function app configuration settings refer to the keyvault, this will make it safe.
Above code is based on AAD bearer token, you need to create a AAD App and let it have the 'Owner' RBAC role of the subscription. You need to something like this:
This just like a 'custom trigger' that trigger by the VM created in your 'subscription'. And I think your VM will not be many, so it will not consume much computing resources.
The access token im getting with gcloud auth print-access-token is obviously a different access token than the one i can get with some basic python code:
export GOOGLE_APPLICATION_CREDENTIALS=/the-credentials.json
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
credentials.get_access_token()
What i am trying to do is get a token that would work with:
curl -u _token:<mytoken> https://eu.gcr.io/v2/my-project/my-docker-image/tags/list
I'd prefer not to install gcloud utility as a dependency for my app, hence my tries to obtain the access token progrmatically via oath google credentials
I know this is a very old question, but I just got faced with the exact same problem of requiring an ACCESS_TOKEN in Python and not being able to generate it, and managed to make it work.
What you need to do is to use the variable credentials.token, except it won't exist once you first create the credentials object, returning None. In order to generate a token, the credentials must be used by a google.cloud library, which in my case was done by using the googleapiclient.discovery.build method:
sqladmin = googleapiclient.discovery.build('sqladmin', 'v1beta4', credentials=credentials)
response = sqladmin.instances().get(project=PROJECT_ID, instance=INSTANCE_ID).execute()
print(json.dumps(response))
After which the ACCESS_TOKEN could be properly generated using
access_token = credentials.token
I've also tested it using google.cloud storage as a way to test credentials, and it also worked, by just trying to access a bucket in GCS through the appropriate Python library:
from google.oauth2 import service_account
from google.cloud import storage
PROJECT_ID = your_project_id_here
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = '/path/to/service.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
try:
list(storage.Client(project=PROJECT_ID, credentials=credentials).bucket('random_bucket').list_blobs())
except:
print("Failed because no bucket exists named 'random_bucket' in your project... but that doesn't matter, what matters is that the library tried to use the credentials and in doing so generated an access_token, which is what we're interested in right now")
access_token = credentials.token
print(access_token)
So I think there are a few questions:
gcloud auth print-access-token vs GoogleCredentials.get_application_default()
gcloud doesn't set application default credentials by default anymore when performing a gcloud auth login, so the access_token you're getting from gcloud auth print-access-token is going to be the one corresponding to the used you used to login.
As long as you follow the instructions to create ADC's for a service account, that account has the necessary permissions, and the environment from which you are executing the script has access to the ENV var and the adc.json file, you should be fine.
How to make curl work
The Docker Registry API specifies that a token exchange should happen, swapping your Basic auth (i.e. Authorization: Basic base64(_token:<gcloud_access_token>)) for a short-lived Bearer token. This process can be a bit involved, but is documented here under "How to authenticate" and "Requesting a Token". Replace auth.docker.io/token with eu.gcr.io/v2/token and service=registry.docker.io with service=eu.gcr.io, etc. Use curl -u oauth2accesstoken:<mytoken> here.
See also: How to list images and tags from the gcr.io Docker Registry using the HTTP API?
Avoid the question entirely
We have a python lib that might be relevant to your needs:
https://github.com/google/containerregistry