I am using the following Python function in AWs Lambda:
import json
import boto3
from boto3.dynamodb.conditions import Key, Attr
#always start with the lambda_handler
def lambda_handler(event, context):
# make the connection to dynamodb
dynamodb = boto3.resource('dynamodb')
# select the table
table = dynamodb.Table("test")
response = table.query(
KeyConditionExpression=Key('coursename').eq('intro')
)
data = response['Items']
return {'body' : data}
It outputs the following JSON - notice the "body" key? This is creating some issues when I try to use the response in my app because I have to reference the "body" as part of the response.
JSON response from Lambda
{
"body": [{
"coursename": "introto1",
"course-lesson-name": "Welcome to One! "
}, {
"coursename": "introto2",
"course-lesson-name": "What is One?"
}, {
"coursename": "introto2",
"course-lesson-name": "What Can We do with One?"
}]
}
This is the JSON format I need my Python function to output. Can this be done in AWS Lambda?
JSON format I need:
[{
"coursename": "introto1",
"course-lesson-name": "Welcome to One! "
}, {
"coursename": "introto2",
"course-lesson-name": "What is One?"
}, {
"coursename": "introto2",
"course-lesson-name": "What Can We do with One?"
}]
The response is in that format because you are explicitly wrapping it in a JSON object with a body property. Change return {'body' : data} to return data.
Related
When executing a select query against an athena table via boto3, the response object given is in the syntax:
{
"UpdateCount":0,
"ResultSet":{
"Rows":[
{
"Data":[
{
"VarCharValue":"site_name"
},
{
"VarCharValue":"volume_out_capacity"
},
{
"VarCharValue":"region"
},
{
"VarCharValue":"site_ref"
}
]
},
{
"Data":[
{
"VarCharValue":"ASSET 12"
},
{
"VarCharValue":"10"
},
{
"VarCharValue":"NORTH"
},
{
"VarCharValue":"RHW007777000138"
}
]
}
]
}
Is there an additional argument that can be passed so that the response object will contain columns that do not contain values? Something like:
{
"VarCharValue":"xyz"
}
]
},
{
"Data":[
{
"VarCharValue":None
}
I have looked through the documentation extensively but cannot find arguments that can describe how to format the response in get_query_results() or start_query_execution()
I do not see a option to get that data back directly from Athena. Alternatively if you use S3 Select instead of Athena, you'll get back a json object with all the columns whether they have data or are empty.
Sample Data:
name,number,city,state
coin,1,driggs,
michelle,,chicago,
shaniqua,2,,
marcos,3,stlouis,
S3 Select Result:
{"name":"coin","number":"1","city":"driggs","state":""}
{"name":"michelle","number":"","city":"chicago","state":""}
{"name":"shaniqua","number":"2","city":"","state":""}
{"name":"marcos","number":"3","city":"stlouis","state":""}
Code:
import boto3
session = boto3.session.Session(profile_name="<my-profile>")
s3 = session.client('s3')
resp = s3.select_object_content(
Bucket='<my-bucket>',
Key='<my-file>',
ExpressionType='SQL',
Expression="SELECT * FROM s3object",
InputSerialization={'CSV': {"FileHeaderInfo": "Use", "RecordDelimiter": '\r\n'}, 'CompressionType': 'NONE'},
OutputSerialization={'JSON': {}},
)
for event in resp['Payload']:
if 'Records' in event:
records = event['Records']['Payload'].decode('utf-8')
print(records)
elif 'Stats' in event:
statsDetails = event['Stats']['Details']
print("Stats details bytesScanned: ")
print(statsDetails['BytesScanned'])
print("Stats details bytesProcessed: ")
print(statsDetails['BytesProcessed'])
print("Stats details bytesReturned: ")
print(statsDetails['BytesReturned'])
I'm developing an AWS lambda function that is triggered from an event bridge and then putting another event using python
but struggling to retrieve a value from a variable in the Json string
below is the code
import json, boto3
client = boto3.client('events')
def lambda_handler(event, context):
testV2_dict={
"K1" : event['e1'] ,
"K2" : event['e2']
}
#converting python to json as (put_event - Details) section is expecting json
testV2=json.dumps(testV2_dict)
response = client.put_events(
Entries=
[
{
"DetailType": "test",
"Source": "test",
"Detail": "{ \"testK\": \"testV\",\"testK2\": \""+ testV2 +"\" }"
}
]
)
tried to add Details on different ways,
"Detail": "{ \"testK\": \"testV\",\"testK2\": \""+ testV2 +"\" }" and still getting error as Malformated Details
and if i deleted the ++, I'm getting word testV2 itself not the value from the above
How can I retrieve the value of testV2 in the Details inside the event?
You don't have enough escaping in there. If testV2 is supposed to be a JSON string emebedded in a JSON string embedded in as JSON string, then you need more string escapes. I would let json.dumps handle that:
import json
event = {'e1': 99, 'e2': 101}
testV2_dict={
"K1" : event['e1'] ,
"K2" : event['e2']
}
testV2=json.dumps(testV2_dict)
detail = {
"testK": "testV",
"testK2": testV2
}
Entries= [
{
"DetailType": "test",
"Source": "test",
"Detail": json.dumps(detail),
}
]
print(Entries)
Output:
[{'DetailType': 'test', 'Source': 'test', 'Detail': '{"testK": "testV", "testK2": "{\\"K1\\": 99, \\"K2\\": 101}"}'}]
I don't think you need to do any thing the output of boto3 client has always been in JSON string.
Lambda Program
import json
import boto3
from pprint import pprint
def lambda_handler(event, context):
# TODO implement
#instance = event['instanceid']
client = boto3.client("ec2")
status = client.describe_instance_status(InstanceIds=[
'i-0c52lidc87f',
],)
#pprint(status)
for i in status["InstanceStatuses"]:
print("AvailabilityZone :", i["AvailabilityZone"])
print("InstanceId :", i["InstanceId"])
print("InstanceState :", i["InstanceState"])
print("InstanceStatus", i["InstanceStatus"])
return {
'body': ("Instance Status :", i["InstanceState"],i["InstanceId"],i["AvailabilityZone"])
}
Output
{
"statusCode": 200,
"body": [
"Instance Status :",
{
"Code": 16,
"Name": "running"
},
"i-0c52lidc87f",
"ca-central-1a"
]
}
I'm getting the above response from my lambda function on AWS - how can change this to readable format - just Instance ID: i-0c5e8c3c87f and Status: Running
Please help!
Youre JSON isnt formatted properly, if Instance Status is supposed to be a tuple, try:
return {
"body": {
"Instance Status":{
"InstanceState": i["InstanceState"]["Name"],
"InstanceId": i["InstanceId"],
"AvailabilityZone": i["AvailabilityZone"]
}
}
resp = {
"Name": "test",
"os": "windows",
"Agent": {
"id": "2",
"status": [
{
"code": "123",
"level": "Info",
"displayStatus": "Ready",
"message": "running",
"time": "2022-01-18T09:51:08+00:00"
}
]
}
I am trying to get the time value from the JSON.
I tried the below code but faced error with dict
resp1 = json.loads(resp)
resp2 = resp1.values()
creation_time = resp2.get("Agent").get("status")
val= creation_time["time"]
print(val) ## Thrwoing error as dict_values has no 'get'
Any suggestion on python how to take this time values
Few problems I noticed
You are trying to load a Dict type using the json's loads function which is supposed to get a string in json format (ex: '{ "name":"John", "age":30, "city":"New York"}')
You tried to access resp2 before declaration (I guessed you meant "resp1?")
You're using resp3 without declaration.
You are missing }
You don't need the .value() function because it will return a list.
Also creation time is a list with one object, so you need to access it too.
Considering all this, you can change it as follows:
import json
resp = '{ "Name": "test", "os": "windows","Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}'
resp1 = json.loads(resp)
creation_time = resp1.get("Agent").get("status")
val= creation_time[0]["time"]
print(val)
You just need to access the dicts using [] like so:
resp = {"Name": "test", "os": "windows", "Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}
creation_time = resp["Agent"]["status"]
val= creation_time[0]["time"]
print(val)
Output:
2022-01-18T09:51:08+00:00
I have testdata read from excel and want it used to format a nested json string for request post body used, details as below:
import json
kwargs = {"name": "testname", "device": {"plt": "dsk"}}
payload = """
{{
"name": "{name}",
"device": "{device}",
}}
"""
payload = json.loads(payload.format(**kwargs))
And the expected payload should be:
{
"name": "testname",
"device": {"plt": "dsk"}
}
But there's error with json.loads(payload.format(**kwargs)), then how to format nested json string with **dict?
error message
You should do the operation in steps to figure out what goes wrong.
kwargs = {"name": "testname", "device": {"plt": "dsk"}}
payload = """
{{
"name": "{name}",
"device": "{device}",
}}
"""
json_string = payload.format(**kwargs)
print(json_string)
gives
{
"name": "testname",
"device": "{'plt': 'dsk'}",
}
Notice the dictionary {'plt': 'dsk'} was stringified python-style before being interpolated into the payload string. This is not valid json -- the apostrophes need to be replaced with quotation marks.
To get around this, I suggest you create a python dict that looks like payload and then convert that to a json string.
# Some preexisting dictionary
payload_dict = {"oldvalue": 0}
# Add values from kwargs
for key, value in kwargs.items():
payload_dict[key] = value
# Dump it to json
payload = json.dumps(payload_dict, indent=4) # indent=4 is only for pretty-print
print(payload)
which gives the output:
{
"oldvalue": 0,
"name": "testname",
"device": {
"plt": "dsk"
}
}
If payload_dict is identical to kwargs, then all you need to do is
payload = json.dumps(kwargs)
print(payload)
{
"name": "testname",
"device": {
"plt": "dsk"
}
}