I'm tryng to generate a shared access signature link through python of my files which are already at blob storage, but something goes wrong , I received this message when I put the generate link on web browser:
"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature."
I'm generating the key from my container name on right button get shared access signature, but I can't go further.
from datetime import datetime
from datetime import timedelta
#from azure.storage.blob import BlobService
datetime.utcnow()
from azure.storage.blob import generate_blob_sas, AccountSasPermissions,AccessPolicy
def generate_link():
account_name='my_account_name_storage'
container_name='container_name'
blob_name='file_name.xsl'
account_key='?sv=2019-12-12&ss=bfqt&srt=sco&sp=rwdlacupx&se=2020-09-17T05:49:57Z&st=2020-09-16T21:49:57Z&spr=https&sig=sdfsdhgbjgnbdkfnglfkdnhklfgnhklgf%30'
url = f"https://{account_name}.blob.core.windows.net/{container_name}/{blob_name}"
sas_token = generate_blob_sas(
account_name=account_name,
account_key=account_key,
container_name=container_name,
blob_name=blob_name,
permission=AccountSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=1)
)
print(sas_token)
url_with_sas = f"{url}?{sas_token}"
print(url_with_sas)
generate_link()```
It's wrong about account_key in your code.
To find account_key of your storage account, please nav to azure portal -> your storage account -> Settings -> Access keys, then you can see the account_key. The screenshot as below:
Related
I am new come to the python, but I need to invoke Power BI REST API with python to publish my pbix file in my repo to the workspace.
Based on this document, I could successfully authenticated and get the workspace:
import json, requests, pandas as pd
try:
from azure.identity import ClientSecretCredential
except Exception:
!pip install azure.identity
from azure.identity import ClientSecretCredential
# --------------------------------------------------------------------------------------#
# String variables: Replace with your own
tenant = 'Your-Tenant-ID'
client = 'Your-App-Client-ID'
client_secret = 'Your-Client-Secret-Value' # See Note 2: Better to use key vault
api = 'https://analysis.windows.net/powerbi/api/.default'
# --------------------------------------------------------------------------------------#
# Generates the access token for the Service Principal
auth = ClientSecretCredential(authority = 'https://login.microsoftonline.com/',
tenant_id = tenant,
client_id = client,
client_secret = client_secret)
access_token = auth.get_token(api)
access_token = access_token.token
print('\nSuccessfully authenticated.')
But I do not know how to publish my pbix to one of my workspace and with parameter overwrite by using REST API with python. And if the pbix already existed in the workspace, provide the parameter to overwrite it.
Any advice would be greatly appreciated and a sample will be greate.
I am having a lot of difficulty writing an API response as json to a blob within an Azure Storage Container. I have tried multiple solutions online but have not managed to see any through to success. I would like to share 2 attempts I have made and hopefully there is someone out there that can assist me in getting at least one methodology correct
Attempt/Method 1
I have tried to use a Service Principle to authenticate my BlobServiceClient from Azure-Storage-Blob. My service principal has been assigned the role of Storage Blob Data Contributor for the Container within which I am trying to create the blob. However on execution of the script I receive an error along the lines of "Unsupported Credential". Below is my script and the error:
My script and resulting error are:
import azure.functions as func
import requests
import json
import uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
from msrestazure.azure_active_directory import ServicePrincipalCredentials
from azure.storage.common import TokenCredential
# Initialise parameters to obtain data from Rest API
url = "https://api.powerbi.com/v1.0/myorg/admin/groups?$top=1000&$expand=datasets,dataflows,reports,users,dashboards"
headers = {'Authorization': get_access_token()}
# Get response. I want to save the response output to a blob.
response = requests.get(url, headers=headers)
response = response.json()
# Initialise parameters for credentials
CLIENT = "bxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx7" # Azure App/Service Principal ID
KEY = "Gxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1" # Azure App/Service Principal Key
TENANT_ID = "cxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx7" # Tenant where Storage Account is which is different to the Tenant the App resides
RESOURCE = f"https://storageaccountxxxxxxxxx.blob.core.windows.net"
# Create credentials & token
credentials = ServicePrincipalCredentials(
client_id = CLIENT,
secret = KEY,
#tenant = TENANT_ID,
resource = RESOURCE
)
tokenCre = TokenCredential(credentials.token["access_token"])
# Initialise parameters for BlobServiceClient
ACCOUNT_URL = "https://storageaccountxxxxxxxxx.blob.core.windows.net/pbiactivity" # includes container name at end of url
#Create BlobServiceClient
blobService = BlobServiceClient(account_url = ACCOUNT_URL, token_credential=tokenCre)
#Create blobClient
blobClient = BlobClient(account_url = RESOURCE,container_name=CONTAINER_NAME, blob_name="response.json", credential = tokenCre )
#Upload response json as blob
blobClient.upload_blob(response, blob_type = "BlockBlob")
Click here for the error that comes after the upload_blob method call]1
Attempt/Method 2
In my second attempt I tried to create ,my BlobServiceClient using Azure-Storage-Blob using my storage account connection string. This method actually allows me to create containers, however when I try to upload a blob as in the script below, However I am unable to create blobs within a container as I get a 403 Forbidden response.
My script and resulting error are:
import requests
import json
import uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
# Initialise parameters to obtain data from Rest API
url = "https://api.powerbi.com/v1.0/myorg/admin/groups?$top=1000&$expand=datasets,dataflows,reports,users,dashboards"
headers = {'Authorization': get_access_token()}
# Get response. I want to save the response output to a blob.
response = requests.get(url, headers=headers)
response = response.json()
# Initialise parameters
CONNECTION_STRING = "DefaultEndpointsProtocol=https;AccountName=storageaccountxxxxxxxxx;AccountKey=rxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxQ==;EndpointSuffix=core.windows.net"
# Create blobServiceClient from connection string
blobServiceClient = BlobServiceClient.from_connection_string(conn_str=CONNECTION_STRING)
#Create blobClient
blobClient = blobServiceClient.get_blob_client(container = "pbiactivity", blob = "response.json")
#Upload response json to blob
blobClient.upload_blob(response, blob_type = "BlockBlob")
Click Here for the errors that come after the upload_blob method call]2
Here is one of the workaround that worked for me:-
import os
import logging
from azure.storage.blob import BlobServiceClient, BlobClient
#Initialise parameters
url = "<YourURL>"
headers = {'Authorization': get_access_token()}
#Get response
response = requests.get(url, headers=headers)
response = response.json()
connectionString= "<Your_Connection_String>"
containerName = "<Name_of_your_container>"
blobServiceClient = BlobServiceClient.from_connection_string(connectionString)
blobContainerClient = blobServiceClient.get_container_client(containerName)
#To create Container (If the container has already been created you can ignore this)
#blobContainerClient.create_container()
#Create blobClient
blobClient = blobServiceClient.get_blob_client(container = "<Name_of_your_container>", blob = "response.json")
with open("response", "rb") as blob_file:
blobClient.upload_blob(data=blob_file)
In my Storage Account:-
I am trying to download a client's blob data which is in JSON format from their azure storage.
I have the account URL containing the SAS token, the SAS token itself as well as connection URL.
However, when I try to download a blob from their server, I am getting the following error :
This request is not authorized to perform this operation using this resource type.
RequestId:d6d9d23e-301e-0078-32be-ecd22a000000
Time:2020-02-26T16:02:05.6188260Z
ErrorCode:AuthorizationResourceTypeMismatch
Error:None
Here is the code I am using :
import os, uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
try:
account_url = "account_url/sas_token"
# Open the container containing the relevant blobs
container = ContainerClient(account_url, container_name="container_name")
# Printing the available blobs
blob_list = container.list_blobs()
for blob in blob_list:
print(blob.name + '\n')
blob_client=container.get_blob_client("blob_name")
print(blob_client)
with open("./test.txt", "wb") as my_blob:
blob_data = blob_client.download_blob()
blob_data.readinto(my_blob)
except Exception as ex:
print('Exception:')
print(ex)
Any more information about this error would be appreciated !
Thanks :D
Since your objective is to download blob, please make sure that your signed resource type (srt) should include object (o).
Your srt should be like srt=co (or just srt=o).
Please regenerate the SAS token accordingly.
You can see the Account SAS permissions by operations here: https://learn.microsoft.com/en-us/rest/api/storageservices/create-account-sas#account-sas-permissions-by-operation (Download Blob is basically Get Blob operation).
I am trying to download an Azure Blob Storage file from my storage account, to do so, I have checked what the URL is and I am doing the following:
with urllib.request.urlopen("<url_file>") as resp:
img = np.asarray(bytearray(resp.read()), dtype="uint8")
But I am getting the following error:
urllib.error.HTTPError: HTTP Error 404: The specified resource does not exist.
I have doubled checked that the url is correct. Could this have something to do with not having passed the keys of my subscription or any other info about the Storage Account?
Any idea?
As on Dec 26, 2019 I am unable to import BaseBlobService from azure cloud storage. Neither of BlobPermissions, generate_blob_shared_access_signature worked for me. Below is something I used and it worked in my case and hope it helps
from azure.storage.blob import generate_blob_sas, AccountSasPermissions
def scan_product():
account_name=<account_name>
container_name=<container_name>
blob_name=<blob_name>
account_key=<account_key>
url = f"https://{account_name}.blob.core.windows.net/{container_name}/{blob_name}"
sas_token = generate_blob_sas(
account_name=account_name,
account_key=account_key,
container_name=container_name,
blob_name=blob_name,
permission=AccountSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=1)
)
url_with_sas = f"{url}?{sas_token}"
Actually, you can generate a blob url with sas token in Azure Storage SDK for Python for accessing directly, as my sample code below.
from azure.storage.blob.baseblobservice import BaseBlobService
from azure.storage.blob import BlobPermissions
from datetime import datetime, timedelta
account_name = '<account name>'
account_key = '<account key>'
container_name = '<container name>'
blob_name = '<blob name>'
url = f"https://{account_name}.blob.core.windows.net/{container_name}/{blob_name}"
service = BaseBlobService(account_name=account_name, account_key=account_key)
token = service.generate_blob_shared_access_signature(container_name, blob_name, permission=BlobPermissions.READ, expiry=datetime.utcnow() + timedelta(hours=1),)
url_with_sas = f"{url}?{token}"
Then,
import urllib
import numpy as np
req = urllib.urlopen(url_with_sas)
img = np.asarray(bytearray(req.read()), dtype=np.uint8)
For downloading using url directly, you should put the blob in a public container, or in the private container then you should generate a sas token for the blob(the url looks like : https://xxx.blob.core.windows.net/aa1/0116.txt?sp=r&st=2019-06-26T09:47:04Z&se=2019-06-26xxxxx).
I test your code with the url which contains a sas token, it can be downloaded.
Test result:
How to generate sas token for a blob:
To solve the issue all I needed to do was to change the Blob Storage access level to Blob (anonymous read access for blob only). Once this is done, it will work.
I'm trying to find out how to create a container obect that has SAS permission applied on it without supplying the account or account key. I've done this in C# but looking to do this in Python.
sasToken = "https://samplestoragehotblob.blob.core.windows.net/samplecontainer?sv=2016-05-31&sr=c&sig=dfdLKJ.....kljsdflkjljsd=3027-09-11T17%3A16%3A57Z&sp=racwdl";
CloudBlobContainer cbContainer = new CloudBlobContainer(new Uri(sasToken));
I can then work in the container with all the necessary permissions without specifying an account and key. Is this possible in Python?
Found a way to not use the account key. This was an acceptable compromise.
from azure.storage.blob import BlockBlobService
def AccessTest():
accountName = "Account Name"
containerName = "Container Name"
sasToken = "sv=2016-05-31&sr=c&sig=BhhYbf3............................-10-02T15%3A28%3A59Z&sp=racwdl"
blobService = BlockBlobService(account_name = accountName, account_key = None, sas_token = sasToken)
for blob in blobService.list_blobs(containerName):
print blob.__getattribute__('name')
Just as Summary, If you haven't generated SAS token,you can't avoid using account key.
You could follow the official tutorial and use generate_shared_access_signature,generate_container_shared_access_signature,generate_blob_shared_access_signature to generate SAS Token for azure storage account,container,blob in python.
If you already generated SAS token, you could operate your container with SAS token instead of account key.
Code Snippet:
from datetime import datetime, timedelta
from azure.storage.blob import (
BlockBlobService,
ContainerPermissions,
)
accountName = "***"
accountKey = "***"
containerName = "***"
def GenerateSasToken():
blobService = BlockBlobService(account_name=accountName, account_key=accountKey)
sas_url = blobService.generate_container_shared_access_signature(containerName,ContainerPermissions.READ, datetime.utcnow() + timedelta(hours=1))
# print sas_url
return 'https://' + accountName + '.blob.core.windows.net/' + containerName + '?' + sas_url
def AccessTest(sastoken):
blobService = BlockBlobService(account_name = accountName, account_key = None, sas_token = sastoken)
BlockBlobService
for blob in blobService.list_blobs(containerName):
print blob.__getattribute__('name')
sastoken = GenerateSasToken()
print sastoken
AccessTest(sastoken)
In addition, you could try to use Azure Key Vault.
The Azure Storage Account (ASA) key feature manages secret rotation
for you. It also removes the need for your direct contact with an ASA
key by offering Shared Access Signatures (SAS) as a method.
which mentioned here.
Please refer to the Azure Key Vault official tutorial and it support REST API.