I have two scripts which I need to deploy in AWS lambda, I have never done it before, from the documentation I created kind of a few steps which would summarize the flow:
Create a lambda function
Install boto3
Use invoke function
Lets say I have a simple function:
def first_function():
return print('First function')
When I go to AWS -> Lambda -> Functions -> Create function I get to the configuration part where in the editor I see this:
import json
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
Is this how I should edit this to deploy my function:
import json
def lambda_handler(event, context):
# TODO implement
return {
def first_function():
return print('First function')
first_function()
}
Tha lambda_handler that shows up when you create a function in the console is simply boiler plate code.
You can name your handler anything, or simply place your function code under lambda_handler
def lambda_handler(event, context):
return print('First function')
The name lambda_handler is configurable, meaning you could use the code
def first_function(event, context):
return print('First function')
But you'll need to ensure that the function is configured to use first_function as it's handler.
I'd recommend reading through the docs specific to python handlers
Whatever functionality you need to implement in your lambda, you should write within the lambda_handler. If you want to refer to other smaller function you can define it outside the lambda handler function and can refer to it in the handler. So it might be like below
import x
def functiona():
print(‘something’)
def functionb():
print(‘somethingelse’)
def lambda_handler(event,context)
print(‘lambda entry point)
functiona()
functionb()
Since the module will first be imported, you can still write code outside of functions although it is usually not a good practice since you cannot access the context and parameters you have sent to lambda.
Related
I am having lambdas which use boto3.client() to connect to a dynamoDB.
I tried to test it like this
#mock.patch("boto3.client")
def test(self, mock_client, test):
handler(event, context)
print(mock_client.call_count) # 1
print(mock_client.put_item.call_count) # 0
However, the mock_client.call_count is 1, but not the put_item_call_count.
My handler looks like this:
def handler(event, context):
dynamodb = boto3.client('dynamodb')
response = dynamodb.put_item(// same attributed)
Any suggestion, how to test if the correct item gots putted in the database without using moto?
I believe you're very close, there's just one tiny problem.
When your mocked boto3.client is called, it returns another mock and you want to evaluate that mocks call_count. By accessing the return_value of the original mock, you get access to the created magic mock.
#mock.patch("boto3.client")
def test(self, mock_client, test):
handler(event, context)
print(mock_client.call_count)
# .return_value refers to the magic mock that's
# created when boto3.client is called
print(mock_client.return_value.put_item.call_count)
What you're currently evaluating is the call count of boto3.client.put_item and not boto3.client("dynamodb").put_item().
I am using a library that requires async context(aioboto3).
My issue is that I can't call methods from outside the async with block on my custom S3StreamingFile instance. If I do so, python raises an exception, telling me that HttpClient is None.
I want to access the class methods of S3StreamingFile from an outer function, for example in a API route. I don't want to return anything more(from file_2.py) than the S3StreamingFile class instance to the caller(file_3.py). The aioboto3 related code can't be moved to file_3.py. file_1.py and file_2.py need to contain the aioboto3 related logic.
How can I solve this?
Example of not working code:
# file_1.py
class S3StreamingFile():
def __init__(self, s3_object):
self.s3_object = s3_object
async def size(self):
return await self.s3_object.content_length # raises exception, HttpClient is None
...
# file_2.py
async def get_file():
async with s3.resource(...) as resource:
s3_object = await resource.Object(...)
s3_file = S3StreamingFile(s3_object)
return s3_file
# file_3.py
async def main()
s3_file = await get_file()
size = await s3_file.size() # raises exception, HttpClient is None
Example of working code:
# file_1.py
class S3StreamingFile():
def __init__(self, s3_object):
self.s3_object = s3_object
async def size(self):
return await self.s3_object.content_length
...
# file_2.py
async def get_file():
async with s3.resource(...) as resource:
s3_object = await resource.Object(...)
s3_file = S3StreamingFile(s3_object)
size = await s3_file.size() # works OK here, HttpClient is available
return s3_file
# file_3.py
async def main()
s3_file = await get_file()
I want to access the class methods from an outer function... how do I solve this?
Don't. This library is using async context managers to handle resource acquisition/release. The whole point about the context manager is that things like s3_file.size() only make sense when you have acquired the relevant resource (here the s3 file instance).
But how do you use this data in the rest of your program? In general---since you haven't said what the rest of your program is or why you want this data---there are two approaches:
acquire the resource somewhere else, and then make it available in much larger scopes, or
make your other functions resource-aware.
In the first case, you'd acquire the resource before all the logic runs, and then hold on to it. (This might look like RAII.) This might well make sense in smaller scripts, or when a resource is designed to be held by only one process at a time. It's a poor fit for code which will spend most of its time doing nothing, or has to coexist with other users of the resource. (An extension of this is writing your own code as a context manager, and effectively moving the problem up the calling stack. If each code path only handles one resource, this might well be the way to go.)
In the second, you'd write your higher-level functions to be aware that they're accessing a resource. You might do this by passing the resource itself around:
def get_file(resource: AcquiredResource) -> FileResource:
...
def get_size(thing: AcquirableResource) -> int:
with thing as resource:
s3_file = get_file(resource)
return s3_file.size
(using made-up generic types here to illustrate the point).
Or you might want a static copy of (some) attrs of a particular resource, like a file here, and a step where you build that copy. Personally I would likely store those in a dict or local object to make it clear that I wasn't handling the resource itself.
The basic idea here is that the with block guards access to a potentially difficult-to-acquire resource. That safety is built into the library, but it comes at the cost of having to think about the acquisition and structure it into the flow of your code.
I am writing tests for my azure function, and for some reason - I can't mock a function call. I should also mention this is the first time I'm writing a python test case so be nice :)
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
try:
req_body = req.get_json()
except ValueError as error:
logging.info(error)
download_excel(req_body)
return func.HttpResponse(
"This HTTP triggered function executed successfully.",
status_code=200
)
so thats the initial function. This function calls download_excel and pass the request body. The next function receives the request body, and writes that excel to a blob storage.
def download_excel(request_body: Any):
excel_file = request_body["items_excel"]
#initiate the blob storage client
blob_service_client = BlobServiceClient.from_connection_string(os.environ["AzureWebJobsStorage"])
container = blob_service_client.get_container_client(CONTAINER_NAME)
blob_path = "excel-path/items.xlsx"
blob_client = container.get_blob_client(blob_path)
blob_client.upload_blob_from_url(excel_file)
Those are the two functions. receive a file, save it to blob storage, but i can't mock the download_excel call in the main function. I've tried using mock, patch, went through all sorts of links, and i just can't find a way to achieve this. Any help would be appreciated. here is what i currently have in the test file.
class TestFunction(unittest.TestCase):
##patch('download_excel')
def get_excel_files_main(self):
"""Test main function"""
req = Mock()
resp = main(req)
# download_excel= MagicMock()
self.assertEqual(resp.status_code, 200)
commenting out the function call in the function and in the test makes the test pass, but i need to know how to mock the download_excel call. I'm still going to write a test case for the download_excel function, but will cross that bridge when i get to it.
Figured it out. I'm pretty silly. Main issue was in an azure function, I figured since there was no class I can ignore every example in the doc that had to do with classes.
The trick is to use the function name as a class. so say you have function name - http_trigger, and a init.py file within that function folder. Within that init file - you have your main method, and a second method thats called from the main method - you can use MagicMock.
import function_name
def test_main_function(self):
"""Testing main function"""
function_name.second_method_being_called = MagicMock()
Thats it. Thats how you mock it! *facepalm
I have a lambda function which in turn calls another lambda function synchronously. I am trying to write tests for the parent function using pytest but I am not sure what is the best way to mock the child lambda function using moto and how does all of it come together in a test function. Can anyone please guide me here.
Parent function
app.py
import boto3
def launch_child_lambda(func_name, data):
lambda_client = boto3.client('lambda')
result = lambda_client.invoke(
FunctionName=func_name,
Payload=json.dumps(data)
)
return result
.....
#omitting code for brevity
.....
def handler(event,context):
//some logic to be tested
response=launch_child_lambda(func_name,data)
//some other logic to be tested
I am trying to use bottle.py to build some webpages. It seems like a major part of using bottle is learning to use decorators but I have read the python docs explanation of what decorators are but I am still not sure I understand them.
The docs say:
"A Python decorator is a specific change to the Python syntax that allows us to more conveniently alter functions and methods (and possibly classes in a future version)."
It sounds like you are calling a function with some changes made but I am not sure why you would do it this way or how to read the decorator.
Looking at some bottle code:
if __name__ == '__main__':
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static').replace('\\', '/')
HOST = os.environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(os.environ.get('SERVER_PORT', '5555'))
except ValueError:
PORT = 5555
#bottle.route('/static/<filepath:path>')
def server_static(filepath):
"""Handler for static files, used with the development server.
When running under a production server such as IIS or Apache,
the server should be configured to serve the static files."""
return bottle.static_file(filepath, root=STATIC_ROOT)
# Starts a local test server.
bottle.run(server='wsgiref', host=HOST, port=PORT)
What does this line do #bottle.route('/static/<filepath:path>')?
If its a fancy function call then why do it this way rather than just calling the function?
Thanks for your help! :D
Check out this code:
def my_decorator(func):
return lambda: print("goodbye")
def greet():
print('hello')
result = my_decorator(greet)
result()
--output:--
goodbye
The following is a shortcut to accomplish the same thing:
def my_decorator(func):
return lambda: print("goodbye")
#my_decorator
def greet():
print('hello')
greet()
--output:--
goodbye
The #my_decorator syntax takes the function below it, greet, and makes this call:
greet = my_decorator(greet)
The my_decorator() function has to be defined so that:
It takes a function as an argument.
Returns a function.
A Python decorator is a specific change to the Python syntax that
allows us to more conveniently alter functions and methods (and
possibly classes in a future version).
Okay, so let's say that you want to add to whatever the greet() function does:
def my_decorator(func): # func = greet
def add_to_greet():
func() #<*********This is greet()
print('world') #<***This is additional stuff.
return add_to_greet
#my_decorator
def greet():
print('hello')
greet()
--output:--
hello
world
What does this line do #bottle.route('/static/<filepath:path>')
Okay, are you ready? If the #some_name syntax specifies an argument, for instance:
#wrapper('world')
def do_stuff():
First python will execute the following call:
#wrapper('world')
def do_stuff():
...
#****HERE:
decorator = wrapper('world') #decorator is a newly created variable
The wrapper() function must be defined to:
Take any old argument, e.g. 'world'
Return a function that:
Takes a function as an argument.
Returns a function.
Secondly, python will execute the call:
#wrapper('world')
def do_stuff():
...
decorator = wrapper('world')
#*****HERE:
do_stuff = decorator(do_stuff)
Whew! Here is an example:
def wrapper(extra_greeting):
def my_decorator(func):
def add_to_greet():
func()
print(extra_greeting)
return add_to_greet
return my_decorator
#wrapper('world')
def greet():
print('hello')
greet()
--output:--
hello
world
Now, let's analyze this decorator:
#bottle.route('/static/<filepath:path>')
def server_static(filepath):
bottle -- a module
route -- a function(or other callable) defined in the bottle module
'/static/<filepath:path>' -- a route
So the bottle module might look like this:
#bottle.py
def route(your_route): #your_route <= '/static/<filepath:path>'
def my_decorator(func): #The decorator syntax will cause python to call this function with server_static as the argument
def do_stuff(filepath):
func(filepath) #Call the server_static() function with the part of the url that matched filepath
return do_stuff #This function will be called when your code calls server_static(...)
return my_decorator
If its a fancy function call then why do it this way rather than just
calling the function?
Advanced stuff.
Comment: Perhaps you forgot to explain what specifically that route decorator does?
#route('/hello')
def hello():
return "Hello World!"
The route() decorator binds a piece of code to an URL path. In this
case, we link the /hello path to the hello() function. This is called
a route (hence the decorator name) and is the most important concept
of this framework. You can define as many routes as you want. Whenever
a browser requests a URL, the associated function is called and the
return value is sent back to the browser. It’s as simple as that.
http://bottlepy.org/docs/dev/tutorial.html
A path can include wild cards:
The simplest form of a wildcard consists of a name enclosed in angle
brackets (e.g. <name>)....Each wildcard matches one or more
characters, but stops at the first slash (/).
The rule /<action>/<item> matches as follows:
Path Result
/save/123 {'action': 'save', 'item': '123'}
/save/123/ No Match
/save/ No Match
//123 No Match
Filters are used to define more specific wildcards, and/or transform
the matched part of the URL before it is passed to the callback. A
filtered wildcard is declared as <name:filter>
The following standard filters are implemented:
:path matches all characters including the slash character in a non-greedy way and may be used to match more than one path
segment.
http://bottlepy.org/docs/dev/routing.html
A decorator is just a function wrapper, it takes some computable and surround it with more computables, technically a decorators is a function that returns a function(or an object, in fact there are decorator classes).
Lets say for example you want to make a logger decorator, this logger decorator will execute something and log who is executing it:
def loggger(name):
def wrapper(f):
def retF(*args, **kwargs):
print name, "is executing"
f(*args, **kwargs)
return retF
return wrapper
So, we have our decorator that will print "Daniel is executing" before call our desired function, for example
#logger("Daniel")
def add2Nums(a,b):
return a+b
>>> add2Nums(1,2)
>>> Daniel is executing
>>> 3
Bottle just works the same way, in the
#bottle.route('/static/<filepath:path>')
it just wrapps your server_static call so whenever some acces that route your function is called.