So i am working on azure function my goal is to have an azure function that is triggerd by an http request that have a blob storage binding which i can retrive files through but when i try to add my blob in binding the function stop working even though i didnt even add any code the main function
this is the binding setting i am using
{
"type": "blob",
"direction": "in",
"name": "inputblob",
"path": "test/{name}",
"connection": "MyStorageConnection"
}
this is the main function i have right now
import logging
import azure.functions as func
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
return func.HttpResponse(f"HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.")
but it is not working it give me a 500 error when trying to trigger the function
So, in your bindings you specify a blob trigger, but your code is expecting an http trigger. That is not going to work.
The code for a blob trigger takes an InputStream represeting the blob:
import logging
import azure.functions as func
def main(myblob: func.InputStream):
logging.info('Python Blob trigger function processed %s', myblob.name)
Also, the binding should like this (note: the type is blobTrigger, not blob):
{
"scriptFile": "__init__.py",
"disabled": false,
"bindings": [
{
"name": "myblob",
"type": "blobTrigger",
"direction": "in",
"path": "samples-workitems/{name}",
"connection":"MyStorageAccountAppSetting"
}
]
}
This is also very well explained in the docs
Related
I have a flow on Power Automate. It will post a json-string to my azure function. How can i write this json to a json file in blobs storage using Azure Data Factory or only Azure Function ?
Below is a sample where you can save JSON through the azure function.
For example, Considering this to be my workflow
I am using HTTP trigger in my case and sending the JSON sample to my function app which has output binding as blob storage and My Http function App looks like this:-
init.py
from http.client import HTTPResponse
import logging
import azure.functions as func
def main(req: func.HttpRequest,outputblob: func.Out[str]) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
print(str(req.get_json()))
outputblob.set(str(req.get_json()))
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"name": "outputblob",
"type": "blob",
"path": "outputcontainer/sample.json",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}
RESULT :
In power Automate
In function app
In Storage Account
I have a Python Azure Function that is one file and one main function:
def main(req: func.HttpRequest) -> func.HttpResponse:
[bunch of code]
return func.HttpResponse("the file will be deleted in 10 minutes", status_code=200)
It creates a file inside Azure Blob storage for the user and deletes it in 10 minutes. I use time.sleep(600) to do this. However, the message only arrives at the end of this timer, after the file has already been deleted.
How can I make the HttpResponse show the message before the script ends, then wait 10 minutes before deleting the message?
I've tried adding func.HttpResponse('the file will be deleted in 10 minutes') before the time.sleep(600) but it doesn't return anything.
For a Function with http output binding like this you have to return the http response at the end for the response to work. So with a single Function, you cannot achieve this. Continue reading for the alternate solution.
This problem is typically an 'asynchronous' processing example where you want to respond immediately like "ok, I am going to do this" while it "queues further processing" to be continued in the backend. To achieve this in Azure Function you will need 2 functions as below:
Function 1 : Http trigger, http output and Queue output binding (for simplicity I will use storage queue).
Function 2 : Queue trigger (will get triggered by the message queued by function 1).
Function 1 (update according to your need):
JSON:
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "queue",
"direction": "out",
"name": "msg",
"queueName": "outqueue",
"connection": "AzureStorageQueuesConnectionString"
}
]
}
Code:
import azure.functions as func
def main(req: func.HttpRequest, msg: func.Out[str]) -> func.HttpResponse:
[bunch of code]
input_msg = "<create the message body required by function 2>"
msg.set(input_msg)
return func.HttpResponse("the file will be deleted in 10 minutes", status_code=201)
Function 2 (update according to your need):
JSON:
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "msg",
"type": "queueTrigger",
"direction": "in",
"queueName": "messages",
"connection": "AzureStorageQueuesConnectionString"
}
]
}
Code:
import json
import azure.functions as func
def main(msg: func.QueueMessage):
# below is just an example of parsing the message, in your case it might be taking the blob info required for deleting
message = json.dumps({
'id': msg.id,
'body': msg.get_body().decode('utf-8'),
'expiration_time': (msg.expiration_time.isoformat()
if msg.expiration_time else None),
'insertion_time': (msg.insertion_time.isoformat()
if msg.insertion_time else None),
'time_next_visible': (msg.time_next_visible.isoformat()
if msg.time_next_visible else None),
'pop_receipt': msg.pop_receipt,
'dequeue_count': msg.dequeue_count
})
[bunch of code]
NOTE: You can also look at Durable Functions where you can handle complex workflow and would not need to manage the queueing yourself. But since your scenario in this case is quite simple, I did not cover it.
This is because the actual response isn't sent before the function itself returns something to the pipeline - the pipeline will then return the result to the caller.
And instead of doing this wonky 10-minute waiting inside a function app (which is something you really never should do), I'd create a queue message, set the initial invisibility to 10 minutes, add to e.g. delete-file-queue. Have a QueueTrigger somewhere, listening to delete-file-queue, and do the deletion of the file.
So instead, do something like this (I'm not super familiar with Functions in python, so treat this as pseudo code):
def main(req: func.HttpRequest) -> func.HttpResponse:
# handle whatever you have to, but do NOT include time.sleep
queue_client.send_message("path/to/blob", visibility_timeout=600)
# the message will end up in the back of the queue, and
# it'll stay invisible for 600 seconds
# this is something we don't have to wait for, and thus, the following
# will return immediately
return func.HttpResponse("file will be deleted in 10 minutes")
Your QueueTrigger would then be something like this:
def main(filename: func.QueueMessage, inputblob: func.InputStream) -> None:
# check if inputblob is none, if not, delete it
In your functions.json, you should include bindings for the filename and inputblob:
{
"name": "filename",
"type": "queueTrigger",
"direction": "in",
"queueName": "delete-file-queue",
"connection": "MyStorageConnectionString"
},
{
"name": "inputblob",
"type": "blob",
"path": "{queueTrigger}",
"connection": "MyStorageConnectionString",
"direction": "in"
}
Guide to initializing a queue_client.
And more info here.
I am trying to stream data from hyperlink destination to azure storage. I have to do this via binding since I want to run this from azure function App.
file -- function.json:
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "blob",
"direction": "out",
"name": "outputBlob",
"path": "samples-workitems/{rand-guid}",
"connection": ""
}
]
}
file -- init.py:
import logging
import cdsapi
import azure.functions as func
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
def main(req: func.HttpRequest, outputBlob:func.Out[func.InputStream]) -> func.HttpResponse:
logging.info('Python HTTP trigger function is about to process request.')
try:
source_blob="http://www.africau.edu/images/default/sample.pdf"
with open(source_blob, "rb") as data:
print(data)
outputBlob.set(data)
except Exception as ex:
logging.info(" error!", ex, "occurred.")
return func.HttpResponse(
"This HTTP triggered function executed successfully.",
status_code=200
)
I have tested binding and it works. When I simply do outputBlob.set("sample string") data is streamed as it should be.
I am stuck with converting data from hyperlink to bytes(or blob). While running code above, i get error Exception: TypeError: not all arguments converted during string formatting. Any help in converting this and uploading to azure storage is appreciated.
Problem is you were trying to read the File from URL with open(source_blob, "rb") as data: which of course won't work since open is for local files only. I have changed your code as below using requests module to get the remote URL response and set the content to blob.
import requests
source_url="http://www.africau.edu/images/default/sample.pdf"
with requests.get(source_url, stream=True) as r:
r.raise_for_status()
outputBlob.set(r.content)
I want to write in an azure blob storage using Azure Functions in Python.
I'm using the output blob storage bindings for Azure Functions.
My function.json:
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"name": "inputblob",
"type": "blob",
"path": "{containerName}/{blobName}.json",
"connection": "MyStorageConnectionAppSetting",
"direction": "in"
},
{
"name": "outputblob",
"type": "blob",
"path": "{containerName}/{blobName}.json",
"connection": "MyStorageConnectionAppSetting",
"direction": "out"
},
{
"type": "http",
"direction": "out",
"name": "$return"
}
]
}
And my Python code looks like this:
import logging
import azure.functions as func
import azure.storage.blob
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
import json, os
def main(req: func.HttpRequest, inputblob: func.InputStream, outputblob: func.Out[func.InputStream]) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
# Initialize variable for tracking any changes
anyChanges= False
# Read JSON file
jsonData= json.loads(inputblob.read())
# Make changes to jsonData (omitted for simplicity) and update anyChanges
# Upload new JSON file
if anyChanges:
outputblob.set(jsonData)
return func.HttpResponse(f"Input data: {jsonData}. Any changes: {anyChanges}.")
However, this isn't working at all, with the following error being thrown:
Value 'func.Out' is unsubscriptable
Another guy in December already asked for a resolution to the same issue, but the answer does not solve the issue
Same issue using 3.7.0 switched to 3.6.8 to solve it
I've had the same issue, and I actually found out that's the problem is resolved by removing pylint library:
pip uninstall pylint
I am not sure if that is a problem in pylint or if Azure Functions cannot coexist with pylint in Python 3.7, but removing it did the trick for me.
Instead of uninstalling pylint, you can simply disable that specific warning:
# pylint: disable=unsubscriptable-object
def main(req: func.HttpRequest, inputblob: func.InputStream, outputblob: func.Out[func.InputStream]) -> func.HttpResponse:
# your code
I have a python azure function with a queue output binding. I am successfully using this binding to queue messages from within the function. Is it possible to set the message TTL on the underlying queue or on the message itself? I don't need to set it on a per message basis, but will do it that way if that is the only option.
host.json
{
"version": "2.0",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
}
}
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"name": "req",
"methods": [
"get",
"post"
]
},
{
"type": "http",
"direction": "out",
"name": "$return"
},
{
"type": "queue",
"direction": "out",
"name": "msg",
"queueName": "predictions",
"connection": "AzureWebJobsStorage"
}
]
}
function code
import json
import logging
import azure.functions as func
from graphene import Schema
from .helpers import responses
from .schema.Query import Query
def main(req: func.HttpRequest, msg: func.Out[func.QueueMessage]) -> func.HttpResponse:
logging.info('Executing GraphQL function.')
try:
query = req.get_body().decode()
except ValueError:
pass
if query:
schema = Schema(Query)
results = schema.execute(query)
response = responses.graphql(results)
# Write response to azure queue storage
message = responses.storage(query, response)
if message:
msg.set(message)
return response
else:
return responses.bad_request(
'Please pass a GraphQL query in the request body.')
For now this only supports c# language, you could bind it to CloudQueue type. If it's other laguage you have to use the SDK method to implement. If you insist this feature, you could got to this github issue to comment you requirements.
And below is my test code to set TTL in a HTTP trigger function with azure-storage-queue 2.1.0.
import logging
import azure.functions as func
from azure.storage.queue import QueueService
import os
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
queue_service = QueueService(connection_string=os.environ['AzureWebJobsStorage'])
message = req.params.get('message')
if not message:
try:
req_body = req.get_json()
except ValueError:
pass
else:
message = req_body.get('message')
if message:
queue_service.put_message('myqueue',message,None,300,None)
return func.HttpResponse(f" {message}!")
else:
return func.HttpResponse(
"Please pass message in the request body",
status_code=400
)