We Keep Coding

Make Bulk Requests API Client Programming Pattern / Design

I am writing an API client for a REST API that allows for requests the be batched together in and send as one request
I am struggling to figure out how to structure the client. I've come up with two designs so far (pseudo Python code):
Async Method
When calls to the api are made (make_api_request), the method waits for the bulk request to be made. Once the bulk request is made by calling the resolve coroutine, the request coroutine releases control back to he make_api_request method which processes the response for that specific request and returns it
import asyncio
class FakeAPIClient:
def __init__(self):
"""
"""
pass
async def make_api_request(self):
"""
returns response after request is made
"""
resp = await self.request()
#response processing
return resp
async def request(self):
"""
I block until the bulk request is made
Once it is make, I return the response from the single request
"""
return "RESPONSE AFTER DONE BLOCKING "
async def resolve(self):
"""
Make the bulk request and release the make_api_request calls and return the assocaited responses
"""
return None
async def example():
api = FakeAPIClient()
future1 = api.make_api_request()
future2 = api.make_api_request()
future3 = api.make_api_request()
#3 requests are sent in bulk
response1, response2, response3, _ = asyncio.gather(
future1,
future2,
future3,
api.resolve()
)
asyncio.run(example())
Lookup method
When calls to the api are made (make_api_request), a lookup ID is returned and the request is put into storage. When the resolve method is called, the bulk request is called, the requests in storage are sent as a bulk request and an object is returned that can be used to find the lookup id's corresponding response.
class FakeAPIClient:
def __init__(self):
pass
def make_api_request(self):
"""
adds request to a queue of requests that will be resolved when resolve method is called
also provide a callback for post request processing
returns a unique id of the request
"""
return "UNIQUE ID"
def resolve(self):
"""
makes the bulk request
takes the responses and associates them with the request id
calls the callback associated with the response for post request processing
returns an object with a method that returns the responses when provided the request id
"""
return "RESPONSE LOOKUP"
api = FakeAPIClient()
lookup_id1 = api.make_api_request()
lookup_id2 = api.make_api_request()
lookup_id3 = api.make_api_request()
lookup_object = api.resolve()
response1 = lookup_object.get(lookup_id1)
response2 = lookup_object.get(lookup_id2)
response3 = lookup_object.get(lookup_id3)
I don't really love either of these solutions, but I can't think of any alternatives. I'd assume there are known patterns for solving this problem, what are they?

Azure Function app - Python SDK - How to read Service Bus Message body

I completed Azure solution: Azure Function, Service Bus, Python SDK code. My Function is triggered by Service Bus message. Function is triggered as expected. Function code below
import logging
import json
import requests
import azure.functions as func
def main(msg: func.ServiceBusMessage):
logging.info('Python ServiceBus queue trigger processed message: %s',
msg.get_body().decode('utf-8'))
url = 'WebHookURL'
payload = {"keyA": "valueA"}
requests.post(url, data=json.dumps(payload), headers={'Content-Type': 'application/json'})
Question is how can I get Service bus message nested values.
In Ms documentation I found an example, below, but how to get message body details ('body': msg.get_body().decode('utf-8')). Message body is the core of the solution because in this message body there are a lot of key:value properties delivered by external service.
def main(msg: func.ServiceBusMessage):
logging.info('Python ServiceBus queue trigger processed message.')
result = json.dumps({
'message_id': msg.message_id,
'body': msg.get_body().decode('utf-8'),
'content_type': msg.content_type,
'expiration_time': msg.expiration_time,
'label': msg.label,
'partition_key': msg.partition_key,
'reply_to': msg.reply_to,
'reply_to_session_id': msg.reply_to_session_id,
'scheduled_enqueue_time': msg.scheduled_enqueue_time,
'session_id': msg.session_id,
'time_to_live': msg.time_to_live,
'to': msg.to,
'user_properties': msg.user_properties,
'metadata' : msg.metadata
}, default=str)

I completed Function App Python code, it looks like below, works as expected, verified and tested.
import logging
import json
import azure.functions as func
def main(msg: func.ServiceBusMessage):
result = ({
'body': json.loads(msg.get_body().decode('utf-8'))
})
try:
resource_name = result.get('body', {}).get('resourceName')
logging.info('resourceName: %s', resource_name)
except Exception as e:
logging.info(e)
input json content:
{
"resourceName":"testVM",
"resourceVnet": "Vnet01"
}

Return File/Streaming response from online video URL in FastAPI

I am using FastAPI to return a video response from googlevideo.com. This is the code I am using:
#app.get(params.api_video_route)
async def get_api_video(url=None):
def iter():
req = urllib.request.Request(url)
with urllib.request.urlopen(req) as resp:
yield from io.BytesIO(resp.read())
return StreamingResponse(iter(), media_type="video/mp4")
but this is not working
I want this Nodejs to be converted into Python FastAPI:
app.get("/download-video", function(req, res) {
http.get(decodeURIComponent(req.query.url), function(response) {
res.setHeader("Content-Length", response.headers["content-length"]);
if (response.statusCode >= 400)
res.status(500).send("Error");
response.on("data", function(chunk) { res.write(chunk); });
response.on("end", function() { res.end(); }); }); });

I encountered similar issues but solved all. The main idea is to create a session with requests.Session(), and yield a chunk one by one, instead of getting all the content and yield it at once. This works very nicely without making any memory issue at all.
#app.get(params.api_video_route)
async def get_api_video(url=None):
def iter():
session = requests.Session()
r = session.get(url, stream=True)
r.raise_for_status()
for chunk in r.iter_content(1024*1024):
yield chunk
return StreamingResponse(iter(), media_type="video/mp4")

The quick solution would be to replace yield from io.BytesIO(resp.read()) with the one below (see FastAPI documentation - StreamingResponse for more details).
yield from resp
However, instead of using urllib.request and resp.read() (which would read the entire file contents into memory, hence the reason for taking too long to respond), I would suggest you use the HTTPX library, which, among other things, provides async support as well. Also, it supports Streaming Responses (see async Streaming Responses too), and thus, you can avoid loading the entire response body into memory at once (especially, when dealing with large files). Below are provided examples in both synchronous and asynchronous ways on how to stream a video from a given URL.
Note: Both versions below would allow multiple clients to connect to the server and get the video stream without being blocked, as a normal def endpoint in FastAPI is run in an external threadpool that is then awaited, instead of being called directly (as it would block the server)—thus ensuring that FastAPI will still work asynchronously. Even if you defined the endpoint of the first example below with async def instead, it would still not block the server, as StreamingResponse will run the code (for sending the body chunks) in an external threadpool that is then awaited (have a look at this comment and the source code here), if the function for streaming the response body (i.e., iterfile() in the examples below) is a normal generator/iterator (as in the first example) and not an async one (as in the second example). However, if you had some other I/O or CPU blocking operations inside that endpoint, it would result in blocking the server, and hence, you should drop the async definition on that endpooint. The second example demonstrates how to implement the video streaming in an async def endpoint, which is useful when you have to call other async functions inside the endpoint that you have to await, as well as you thus save FastAPI from running the endpoint in an external threadpool. For more details on def vs async def, please have a look at this answer.
The below examples use iter_bytes() and aiter_bytes() methods, respectively, to get the response body in chunks. These functions, as described in the documentation links above and in the source code here, can handle gzip, deflate, and brotli encoded responses. One can alternatively use the iter_raw() method to get the raw response bytes, without applying content decoding (if is not needed). This method, in contrast to iter_bytes(), allows you to optionally define the chunk_size for streaming the response content, e.g., iter_raw(1024 * 1024). However, this doesn't mean that you read the body in chunks of that size from the server (that is serving the file) directly. If you had a closer look at the source code of iter_raw(), you would see that it just uses a ByteChunker that stores the byte contents into memory (using BytesIO() stream) and returns the content in fixed-size chunks, depending the chunk size you passed to the function (whereas raw_stream_bytes, as shown in the linked source code above, contains the actual byte chunk read from the stream).
Using HTTPX with def endpoint
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import httpx
app = FastAPI()
#app.get('/video')
def get_video(url: str):
def iterfile():
with httpx.stream("GET", url) as r:
for chunk in r.iter_bytes():
yield chunk
return StreamingResponse(iterfile(), media_type="video/mp4")
Using HTTPX with async def endpoint
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import httpx
app = FastAPI()
#app.get('/video')
async def get_video(url: str):
async def iterfile():
async with httpx.AsyncClient() as client:
async with client.stream("GET", url) as r:
async for chunk in r.aiter_bytes():
yield chunk
return StreamingResponse(iterfile(), media_type="video/mp4")
You can use public videos provided here to test the above. Example:
http://127.0.0.1:8000/video?url=http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
If you would like to return a custom Response or FileResponse instead—which I wouldn't really recommend in case you are dealing with large video files, as you should either read the entire contents into memory, or save the contents to a temporary file on disk that you later have to read again into memory, in order to send it back to the client—please have a look at this answer and this answer.

How can I troubleshoot my AWS Lambda function?

I created a lambda function in AWS and apparently it is throwing an error. Here it is:
import json
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
import boto3
#Create a SSM Client to access parameter store
ssm = boto3.client('ssm')
def lambda_handler(event, context):
# TODO implement
#return {
# 'statusCode': 200,
# 'body': json.dumps('Hello from Lambda!')
#}
slack_message = {
'text' = f'Hello World'
}
#retrieve webhook url from parameter store
webhook_url = ssm.get_parameter(Name='slackwebhookurl', WithDecryption=True)
#make request to the API
req = Request(webhook_url['Parameter']['Value'],
json.dumps(slack_message).encode('utf-8'))
try:
response = urlopen(req)
response.read()
print("Messge posted to Slack")
except HTTPError as e:
print(f'Request failed: {e.code} {e.reason})
except URLError as e:
print(f'Server Connection failed: {e.reason})
It's triggered by an AWS SNS notification. It's supposed to grab a webhook url for a slack channel and then send the notification to Slack.
Can anyone see what the problem is?
If it's not obvious, can someone direct me to a tutorial on how to test AWS Lambda functions?
Thanks.

AWS Lambda has an in-built Test functionality. You can click the Test button and configure an input in case the function uses values from event.
Logs will be displayed within the Lambda console.
Also, ensure that the IAM Role associated with the AWS Lambda function has the AWSLambdaBasicExecutionRole permission policy so that it can write to CloudWatch Logs. Then, you can go to the Monitoring tab of the function and click View logs in CloudWatch to see past logs.
You can add print() statements in your code, which will appear in the Logs.

Using boto to invoke lambda functions how do I do so asynchronously?

SO I'm using boto to invoke my lambda functions and test my backend. I want to invoke them asynchronously. I have noted that "invoke_async" is deprecated and should not be used. Instead you should use "invoke" with an InvocationType of "Event" to do the function asynchronously.
I can't seem to figure out how to get the responses from the functions when they return though. I have tried the following:
payload3=b"""{
"latitude": 39.5732160891,
"longitude": -119.672918997,
"radius": 100
}"""
client = boto3.client('lambda')
for x in range (0, 5):
response = client.invoke(
FunctionName="loadSpotsAroundPoint",
InvocationType='Event',
Payload=payload3
)
time.sleep(15)
print(json.loads(response['Payload'].read()))
print("\n")
Even though I tell the code to sleep for 15 seconds, the response variable is still empty when I try and print it. If I change the invokation InvokationType to "RequestResponse" it all works fine and response variable prints, but this is synchronous. Am I missing something easy? How do i execute some code, for example print out the result, when the async invokation returns??
Thanks.

There is a difference between an 'async AWS lambda invocation' and 'async python code'. When you set the InvocationType to 'Event', by definition, it does not ever send back a response.
In your example, invoke() immediately returns None, and does not implicitly start up anything in the background to change that value at a later time (thank goodness!). So, when you look at the value of response 15 seconds later, it's still None.
It seems what you really want is the RequestResponse invocation type, with asynchronous Python code. You have a bunch of options to choose from, but my favorite is concurrent.futures. Another is threading.
Here's an example using concurrent.futures:
(If you're using Python2 you'll need to pip install futures)
from concurrent.futures import ThreadPoolExecutor
import json
payload = {...}
with ThreadPoolExecutor(max_workers=5) as executor:
futs = []
for x in xrange(0, 5):
futs.append(
executor.submit(client.invoke,
FunctionName = "loadSpotsAroundPoint",
InvocationType = "RequestResponse",
Payload = bytes(json.dumps(payload))
)
)
results = [ fut.result() for fut in futs ]
print results
Another pattern you might want to look into is to use the Event invocation type, and have your Lambda function push messages to SNS, which are then consumed by another Lambda function. You can check out a tutorial for SNS-triggered lambda functions here.

An asynchronously executed AWS Lambda function doesn't return the result of execution. If an asynchronous invocation request is successful (i.e. there were no errors due to permissions, etc), AWS Lambda immediately returns the HTTP status code 202 ACCEPTED and bears no further responsibility for communicating any information about the outcome of this asynchronous invocation.
From the documentation of AWS Lambda Invoke action:
Response Syntax
HTTP/1.1 StatusCode
X-Amz-Function-Error: FunctionError
X-Amz-Log-Result: LogResult
Payload
Response Elements
If the action is successful, the service sends back the following HTTP
response.
StatusCode
The HTTP status code will be in the 200 range for successful request.
For the RequestResponse invocation type this status code will be 200.
For the Event invocation type this status code will be 202. For the DryRun invocation type the status code will be 204.
[...]
The response returns the following as the HTTP body.
Payload
It is the JSON representation of the object returned by the Lambda
function. This is present only if the invocation type is
RequestResponse.

The following are a python function that accepts lambda-function-Name to invoke and payload to send to that function.
It invokes the lambda function by boto3 client.
import boto3, json, typing
def invokeLambdaFunction(*, functionName:str=None, payload:typing.Mapping[str, str]=None):
if functionName == None:
raise Exception('ERROR: functionName parameter cannot be NULL')
payloadStr = json.dumps(payload)
payloadBytesArr = bytes(payloadStr, encoding='utf8')
client = boto3.client('lambda')
return client.invoke(
FunctionName=functionName,
InvocationType="RequestResponse",
Payload=payloadBytesArr
)
And usage:
if __name__ == '__main__':
payloadObj = {"something" : "1111111-222222-333333-bba8-1111111"}
response = invokeLambdaFunction(functionName='myLambdaFuncName', payload=payloadObj)
print(f'response:{response}')