I am trying to list Azure VMs using python code. could someone help me with this issue?
I have already tried to go through the code on the Microsoft website but it is not clear to me.
First, you need to follow the section Register your client application with Azure AD of Azure offical document Azure REST API Reference to register an application with Azure AD on Azure portal for getting the required parameters client_id and secret for Authentication to list VMs.
And, you continue to get the other required parameters subscription_id and tenant_id.
Then, to create a virtual environment in Python to install Azure SDK for Python via pip install azure or just install Azure Compute Management for Python via pip install azure-mgmt-compute.
Here is my sample code.
from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.compute import ComputeManagementClient
credentials = ServicePrincipalCredentials(
client_id='<your client id>',
secret='<your client secret>',
tenant='<your tenant id>'
)
subscription_id = '<your subscription id>'
client = ComputeManagementClient(credentials, subscription_id)
If just list VMs by resource group, to use function list(resource_group_name, custom_headers=None, raw=False, **operation_config) as the code below.
resource_group_name = '<your resource group name>'
vms_by_resource_group = client.virtual_machines.list(resource_group_name)
Or you want to list all VMs in your subscription, to use function list_all(custom_headers=None, raw=False, **operation_config) as the code below.
all_vms = client.virtual_machines.list_all()
As references, there are two SO threads I think which may help for understanding deeper: How to get list of Azure VMs (non-classic/Resource Managed) using Java API and Is it anyway to get ftpsState of azure web app (Azure Function app) using Python SDK?.
Hope it helps.
Related
Using Azure Devops I'm developing a Cloud Run service on PROJECT_A that need to utilize a Service Account of a PROJECT_B to read logs from Stackdriver.
I've successfully deployed the Cloud Run using its gcloud commands.
gcloud run deploy [[SERVICE] --namespace=NAMESPACE] [--service-account=Service_Account#PROJECT_A.iam.gserviceaccount.com]
Since I'm storing my service account as secure file in Azure Devops, I've uploaded PROJECT_B service account to PROJECT_A GCP Secret Manager using
echo $(service_account_PROJECT_B.json) > SA_PROJECT_B.txt
gcloud secrets create SA_PROJECT_B --data-file=SA_PROJECT_B.txt --replication policy=user-managed --project=PROJECT_A
I'm finding Issues while accessing to the Service Account stored in the secret manager.
Locally, when I create the client, I use:
#config.py
if DEPLOY_ENVIRONMENT == "local":
SA_PROJECT_B = os.path.join(BASE_DIR / "SA_PROJECT_B.json")
os.environ["SA_PROJECT_B"] = str(SA_PROJECT_B)
.
#client.py
from google.cloud import logging
from config import SA_PROJECT_B
logging_client = logging.Client.from_service_account_json(
SA_PROJECT_B
)
And it works.
When I execute the code from the Cloud Run, I get an error message stating that it can't cannot import name 'SA_PROJECT_B'
So here is my question:
how should I reference to a secret stored in secrets manager from the code?
I've tried following this google cloud community tutorial that showed me that the issue happens when I upload the secret to secret manager.
secrets = secretmanager.SecretManagerServiceClient()
SA_PROJECT_B= secrets.access_secret_version(request={"name": "projects/"+"PROJECT_B"+"/secrets/PROJECT_B/versions/1"}).payload.data.decode("utf-8")
print(SA_PROJECT_B) returns '$(service_account_PROJECT_B.json)'
I can't understand what I'm doing wrong.
Is something related with uploading a service account to gcp or something related to accessing correctly the secret manager?
I have a requirement where I will create a azure Http trigger using python function app.
For this function app I need to provide Authorization by using client Id or client Secret. Where the http link has to approved only with the client secret
I couldn't found any reference in python.
If any one has done this method can u show ur code for reference or guide me to solve this
Here's Microsoft documentation that you can use to get an idea of how to implement Azure AD Authentication into your Python Function App via code.
It includes other authentication methods also like Authentication with service principals, user credentials, AzureCliCredential or Managed Identity and DefaultAzureCredential.
Instead of facing difficulties in accessing the Client Id or Secret from Azure AD if you do not have access to it, you can store them in Azure Key Vault and access through Python Code in the same Azure Function App:
import os from azure.identity
import DefaultAzureCredential from azure.keyvault.secrets
import SecretClient
vault_url = os.environ["KEY_VAULT_URL"]
credential = DefaultAzureCredential()
secret_client = SecretClient(vault_url=vault_url, credential=credential)
retrieved_secret = secret_client.get_secret("secret-name-01")
And this Microsoft Blog Article presents you to register the Azure Python Function App with Azure AD using Azure CLI Commands and access the HTTP Trigger URL secured with the Client Id.
How I can get a list of users in account GCP using python. I can't find how I can authorize using python in account and get a list. Can anybody help me?
I am assuming that you are just getting started with Google Cloud and the Python SDKs. If you are already experienced, skip to the bottom of my answer for the actual example code.
The documentation for the Google Cloud Python SDKs can be hard to figure out. The key detail is that Google documents the APIs using automated tools. Google publishes a document that SDKs can read to automatically build APIs. This might appear strange at first, but very clever when you think about it. SDKs that automatically update themselves to support the latest API implementation.
Start with the root document: Google API Client Library for Python Docs
Near the bottom is the link for documentation:
Library reference documentation by API
For your case, listing users with IAM bindings in a project, scroll down to cloudresourcemanager. Sometimes there are multiple API versions. Usually, pick the latest version. In this case, v3.
Knowing which API to use is built from experience. As you develop more and more software in Google Cloud, the logic to the architecture becomes automatic.
Cloud Resource Manager API
The API provides multiple Instance Methods. In your case, the instance method is projects.
Cloud Resource Manager API - projects
Within projects are Instance Methods. In your case, getIamPolicy().
getIamPolicy(resource, body=None, x__xgafv=None)
Sometimes you need to review the REST API to understand parameters and returned values.
Resource Manager REST API: Method: projects.getIamPolicy
For example, to understand the response from the Python SDK API, review the response documented by the REST API which includes several examples:
Resource Manager REST API: Policy
Now that I have covered the basics of discovering how to use the documentation, let's create an example that will list the roles and IAM members.
Import the required Python libraries:
from google.oauth2 import service_account
import googleapiclient.discovery
Create a variable with your Project ID. Note: do not use Project Name.
PROJECT_ID='development-123456'
Note: In the following explanation, I use a service account. Later in this answer, I show an example using ADC (Application Default Credentials) set up by the Google Cloud CLI (gcloud).
Create a variable with the full pathname to your Google Cloud Service Account JSON Key file:
SA_FILE='/config/service-account.json'
Create a variable for the required Google Cloud IAM Scopes. Typically I use the following scope as I prefer to control permissions via IAM Roles assigned to the service account:
SCOPES=['https://www.googleapis.com/auth/cloud-platform']
Create OAuth credentials from the service account:
credentials = service_account.Credentials.from_service_account_file(
filename=SA_FILE,
scopes=SCOPES)
Now we are at the point to start using the API documentation. The following code builds the API discovery document and loads the APIs for cloudresourcemanager:
service = googleapiclient.discovery.build(
'cloudresourcemanager',
'v3',
credentials=credentials)
Now call the API which will return a JSON response details the roles and members with bindings to the project:
resource = 'projects/' + PROJECT_ID
response = service.projects().getIamPolicy(resource=resource, body={}).execute()
The following is simple code to print part of the returned JSON:
for binding in response['bindings']:
print('Role:', binding['role'])
for member in binding['members']:
print(member)
Complete example that uses ADC (Application Default Credentials):
import googleapiclient.discovery
PROJECT_ID='development-123456'
service = googleapiclient.discovery.build('cloudresourcemanager', 'v3')
resource = 'projects/' + PROJECT_ID
response = service.projects().getIamPolicy(resource=resource, body={}).execute()
for binding in response['bindings']:
print('Role:', binding['role'])
for member in binding['members']:
print(member)
Complete example using a service account:
from google.oauth2 import service_account
import googleapiclient.discovery
PROJECT_ID='development-123456'
SA_FILE='/config/service-account.json'
SCOPES=['https://www.googleapis.com/auth/cloud-platform']
credentials = service_account.Credentials.from_service_account_file(
filename=SA_FILE,
scopes=SCOPES)
service = googleapiclient.discovery.build(
'cloudresourcemanager', 'v3', credentials=credentials)
resource = 'projects/' + PROJECT_ID
response = service.projects().getIamPolicy(resource=resource, body={}).execute()
for binding in response['bindings']:
print('Role:', binding['role'])
for member in binding['members']:
print(member)
I am trying to use the documentation on https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Python.html. Right now I am stuck at session = boto3.session(profile_name='RDSCreds'). What is profile_name and how do I find that in my RDS?
import sys
import boto3
ENDPOINT="mysqldb.123456789012.us-east-1.rds.amazonaws.com"
PORT="3306"
USR="jane_doe"
REGION="us-east-1"
os.environ['LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN'] = '1'
#gets the credentials from .aws/credentials
session = boto3.Session(profile_name='RDSCreds')
client = session.client('rds')
token = client.generate_db_auth_token(DBHostname=ENDPOINT, Port=PORT, DBUsername=USR, Region=REGION)
session = boto3.Session(profile_name='RDSCreds')
profile_name here means the name of the profile you have configured to use for your aws cli.
usually when you run aws configure it creates a default profile.But sometime users want to manage aws cli with another account credentials or amange request for another region so they configure separate profile. docs for creating configuring multiple profiles
aws configure --profile RDSCreds #enter your access keys for this profile
in case if you think you have already created RDSCreds profile to check that profile less ~/.aws/config
the documentation which you have mentioned for rds using boto3 also says "The code examples use profiles for shared credentials. For information about the specifying credentials, see Credentials in the AWS SDK for Python (Boto3) documentation."
I need to retrieve secrets from keyvault. This is my code so far:
from azure.mgmt.keyvault import KeyVaultManagementClient
from azure.common.credentials import ServicePrincipalCredentials
subscription_id = 'x'
# See above for details on creating different types of AAD credentials
credentials = ServicePrincipalCredentials(
client_id = 'x',
secret = 'x',
tenant = 'x'
)
kv_client = KeyVaultManagementClient(credentials, subscription_id)
for vault in kv_client.vaults.list():
print(vault)
But I am getting this error:
msrestazure.azure_exceptions.CloudError: Azure Error:
AuthorizationFailed Message: The client 'x' with object id 'x' does
not have authorization to perform action
'Microsoft.Resources/subscriptions/resources/read' over scope
'/subscriptions/x'.
Now I am able to access the same keyvault with same credentials using C# code/ POwershell so there is definitely nothing wrong with authorization. Not sure why it isnt working using SDK. Please help.
If you are looking to access via a ServicePrincipalCredentials instance, you can just use:
from azure.keyvault import KeyVaultClient, KeyVaultAuthentication
from azure.common.credentials import ServicePrincipalCredentials
credentials = None
def auth_callback(server, resource, scope):
credentials = ServicePrincipalCredentials(
client_id = '',
secret = '',
tenant = '',
resource = "https://vault.azure.net"
)
token = credentials.token
return token['token_type'], token['access_token']
client = KeyVaultClient(KeyVaultAuthentication(auth_callback))
secret_bundle = client.get_secret("https://vault_url", "secret_id", "")
print(secret_bundle.value)
This assumes that you don't want to pass a version. If you do, you can substitute the last parameter for it.
I run your code sample above and it is able to list the key vaults without any issue, hence it is not a code issue.
I have assigned the Contributor role to my AD application on the subscription where the key vault is provisioned and set the Access Policies to allow GET & LIST permissions for Key and Secret to the AD application.
The versions of my Azure Python packages used running under Python 3.6.2 runtime environment:
azure.common (1.1.8)
azure.mgmt.keyvault (0.40.0)
msrestazure(0.4.13)
I'll recommend you to try on the Python runtime version and Azure Python packages versions which is verified working.
Addendum:
If the above Python runtime environment version as well as Azure Python packages also does not work for you, you should probably consider creating a new issue in the Azure SDK for Python GitHub as it is working with the same credential with Azure .NET SDK as well as PowerShell.
You can also get secret by the name of the secret instead of ID:
secret_bundle = client.get_secret(<VAULT URL>, "<NAME>", "")
There are some good answers already, but the Azure SDK has since released new packages for working with Key Vault in Python that replace azure-keyvault:
azure-keyvault-certificates (Migration guide)
azure-keyvault-keys (Migration guide)
azure-keyvault-secrets (Migration guide)
azure-identity is also the package that should be used with these for authentication.
Documentation for working with the secrets library can be found on the azure-sdk-for-python GitHub repository, and here's a sample for retrieving secrets as you were doing:
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
credential = DefaultAzureCredential()
secret_client = SecretClient(
vault_url="https://my-key-vault.vault.azure.net/",
credential=credential
)
secret = secret_client.get_secret("secret-name")
You can provide the same credentials that you used for ServicePrincipalCredentials by setting environment variables corresponding to the client_id, secret, and tenant:
export AZURE_CLIENT_ID="client_id"
export AZURE_CLIENT_SECRET="secret"
export AZURE_TENANT_ID="tenant"
(I work on the Azure SDK in Python)
One can use the below class from azure.identity i.e ClientSecretCredential, find the below code ex: snippet
from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient
TENANT= <TenantId-in-string>
CLIENT_ID = <ClientId-in-string>
CLIENT_SECRET= <ClientSecret-in-string>
credential = ClientSecretCredential(TENANT,CLIENT_ID,CLIENT_SECRET)
VAULT_URL= <AzureVault-url-in-string>
client = SecretClient(vault_url=VAULT_URL, credential=credential)
print(client)
example_secret = client.get_secret(<secret_name_in_string>)
print(example_secret.value)