We Keep Coding

Return empty columns in athena boto response object

When executing a select query against an athena table via boto3, the response object given is in the syntax:
{
"UpdateCount":0,
"ResultSet":{
"Rows":[
{
"Data":[
{
"VarCharValue":"site_name"
},
{
"VarCharValue":"volume_out_capacity"
},
{
"VarCharValue":"region"
},
{
"VarCharValue":"site_ref"
}
]
},
{
"Data":[
{
"VarCharValue":"ASSET 12"
},
{
"VarCharValue":"10"
},
{
"VarCharValue":"NORTH"
},
{
"VarCharValue":"RHW007777000138"
}
]
}
]
}
Is there an additional argument that can be passed so that the response object will contain columns that do not contain values? Something like:
{
"VarCharValue":"xyz"
}
]
},
{
"Data":[
{
"VarCharValue":None
}
I have looked through the documentation extensively but cannot find arguments that can describe how to format the response in get_query_results() or start_query_execution()

I do not see a option to get that data back directly from Athena. Alternatively if you use S3 Select instead of Athena, you'll get back a json object with all the columns whether they have data or are empty.
Sample Data:
name,number,city,state
coin,1,driggs,
michelle,,chicago,
shaniqua,2,,
marcos,3,stlouis,
S3 Select Result:
{"name":"coin","number":"1","city":"driggs","state":""}
{"name":"michelle","number":"","city":"chicago","state":""}
{"name":"shaniqua","number":"2","city":"","state":""}
{"name":"marcos","number":"3","city":"stlouis","state":""}
Code:
import boto3
session = boto3.session.Session(profile_name="<my-profile>")
s3 = session.client('s3')
resp = s3.select_object_content(
Bucket='<my-bucket>',
Key='<my-file>',
ExpressionType='SQL',
Expression="SELECT * FROM s3object",
InputSerialization={'CSV': {"FileHeaderInfo": "Use", "RecordDelimiter": '\r\n'}, 'CompressionType': 'NONE'},
OutputSerialization={'JSON': {}},
)
for event in resp['Payload']:
if 'Records' in event:
records = event['Records']['Payload'].decode('utf-8')
print(records)
elif 'Stats' in event:
statsDetails = event['Stats']['Details']
print("Stats details bytesScanned: ")
print(statsDetails['BytesScanned'])
print("Stats details bytesProcessed: ")
print(statsDetails['BytesProcessed'])
print("Stats details bytesReturned: ")
print(statsDetails['BytesReturned'])

python unittest function expected called but the result is none called

This is my function :
#File lambda_function.py
from calculate import addition
def lambda_handler(event, context):
try:
v = addition(2)
return {
"statusCode": 200,
"body": {
"message": v,
},
}
except Exception as err:
return {
"statusCode": 500,
"headers": {},
"body": {
"message": "error.generic",
},
}
#File calculate.py
def addition(x):
return x + 1;
File test_lambda_function.py
from unittest.mock import patch
import lambda_function as lambda_function
import calculate
class TestLambdaFunction(unittest.TestCase):
def setUp(self):
self.member_id = 100
self.event = {
"body": "None",
}
#patch('calculate.addition', return_value=3)
def test_lambda_handler(self, mocked_addition):
lambda_function.lambda_handler(self.event, {})
self.assertEqual(mocked_addition.call_count, 1)
Test result:
test_lambda_function.py:114 (TestLambdaFunction.test_lambda_handler)
1 != 0
Expected :0
Actual :1
Please can you explain me why the test result tell me that the function isn't called. did I make a mistake in import?

AWS Pinpoint Request on Lambda Function using Python 3.6

I tried to using Pinpoint services in my Lambda Function using Python Language. But I am facing time out error once Executed my function. Here is my Lambda Function source code.
import json
import boto3
client = boto3.client('pinpoint')
def trigger(event, context):
print("---------1----------------")
payload = {
'TraceId': 'sadfasdf',
'Context': {
'string': 'string'
},
'MessageConfiguration': {
'GCMMessage': {
'RawContent': ""
}
},
'TemplateConfiguration': {
'PushTemplate': {
'Name': 'coll-1-en'
}
},
'Users': {
"id": {
"Substitutions":{
"subject": ["string"],
"id": ["120913"]
}
}
}
}
try:
print("---------2----------------")
response = client.send_users_messages(
ApplicationId='my_aws_pinpoint_services_project.id',
SendUsersMessageRequest=payload
)
print("---------3----------------")
data = response.read()
print(data.decode("utf-8"))
return response
except Exception:
return False
I am not sure what is the issue from my code. So I have inserted print("-------") to find out the error lines.
The codes didn't executed since response = client.send_users_messages( line. So I was sure the client.send_users_messages had an issue. But I am not sure what is wrong in this line. Pls help me to fix this issue. I attached the screenshot of the Test result.

AWS Pinpoint Request on Lambda Function using Python

I tried to using pinpoint services in my Lambda Function using Python Language.
But I am facing time out error once Executed my function.
Here is my lambda function source code.
import json
import boto3
client = boto3.client('pinpoint')
def trigger(event, context):
print("---------1----------------")
payload = {
'TraceId': 'sadfasdf',
'Context': {
'string': 'string'
},
'MessageConfiguration': {
'GCMMessage': {
'RawContent': ""
}
},
'TemplateConfiguration': {
'PushTemplate': {
'Name': 'coll-1-en'
}
},
'Users': {
"id": {
"Substitutions":{
"subject": ["string"],
"id": ["120913"]
}
}
}
}
try:
print("---------2----------------")
response = client.send_users_messages(
ApplicationId='my_aws_pinpoint_services_project.id',
SendUsersMessageRequest=payload
)
print("---------3----------------")
data = response.read()
print(data.decode("utf-8"))
return response
except Exception:
return False
I am not sure what is the issue from my code.
So I have inserted print("string") to find out the error lines.
The codes didn't executed since response = client.send_users_messages( line.
So I was sure the client.send_users_messages had an issue.
But I am not sure what is wrong in this line.
Pls help me to fix this issue.
I attached the screenshot of the Test result.

AWS Python Lambda Function JSON format

I am using the following Python function in AWs Lambda:
import json
import boto3
from boto3.dynamodb.conditions import Key, Attr
#always start with the lambda_handler
def lambda_handler(event, context):
# make the connection to dynamodb
dynamodb = boto3.resource('dynamodb')
# select the table
table = dynamodb.Table("test")
response = table.query(
KeyConditionExpression=Key('coursename').eq('intro')
)
data = response['Items']
return {'body' : data}
It outputs the following JSON - notice the "body" key? This is creating some issues when I try to use the response in my app because I have to reference the "body" as part of the response.
JSON response from Lambda
{
"body": [{
"coursename": "introto1",
"course-lesson-name": "Welcome to One! "
}, {
"coursename": "introto2",
"course-lesson-name": "What is One?"
}, {
"coursename": "introto2",
"course-lesson-name": "What Can We do with One?"
}]
}
This is the JSON format I need my Python function to output. Can this be done in AWS Lambda?
JSON format I need:
[{
"coursename": "introto1",
"course-lesson-name": "Welcome to One! "
}, {
"coursename": "introto2",
"course-lesson-name": "What is One?"
}, {
"coursename": "introto2",
"course-lesson-name": "What Can We do with One?"
}]

The response is in that format because you are explicitly wrapping it in a JSON object with a body property. Change return {'body' : data} to return data.