We Keep Coding

Can't test APIs with pytest using JSON

Here's the code of my test:
#pytest.mark.run(order=18)
def test_post(client):
"""
Test whether the test client has been added or not.
"""
print(f"\n\n {'>>>'*6} TESTING CLIENT POST {'<<<'*6} \n\n")
access_token = cognito_auth.get_access_token({
"username": os.environ["TEST_USER_EMAIL"],
"password": os.environ["TEST_USER_PASSWORD"]
})
data = {
"client_code": "999999.9.9",
"name": "AUTOMATED TEST CLIENT",
"short_name": "AUTOMATED TEST CLIENT",
"br_cnpj": "123809128312",
"br_im": "213798238974324",
"br_ie": "7893248932794324",
"address_id": 7665,
"is_inserted": False,
"skin_id": 1,
"plan_id": 1,
"organization": "CFR-100000",
"is_api_connected": False
}
response = client.post('http://localhost:5000/dev/api/client', json=data, headers={
"Authorization": f"Bearer {access_token}"
})
print("THE RESPONSE")
print(response.json)
According to this doc, everything should be fine, but instead, I get the following postgres error:
{'error': {'code': 500, 'type': '/errors/internal-server-error', 'message': '(psycopg2.errors.InvalidTextRepresentation) invalid input syntax for type integer: ""\nLINE 1: ... plan_id, organization, is_api_connected) VALUES (\'\', \'99999...\n ^\n\n[SQL: INSERT INTO tb_client (client_id, client_code, name, short_name, br_cnpj, br_im, br_ie, address_id, is_inserted, skin_id, plan_id, organization, is_api_connected) VALUES (%(client_id)s, %(client_code)s, %(name)s, %(short_name)s, %(br_cnpj)s, %(br_im)s, %(br_ie)s, %(address_id)s, %(is_inserted)s, %(skin_id)s, %(plan_id)s, %(organization)s, %(is_api_connected)s) ON CONFLICT ON CONSTRAINT tb_client_client_code_key DO NOTHING]\n[parameters: {\'client_id\': \'\', \'client_code\': \'999999.9.9\', \'name\': \'AUTOMATED TEST CLIENT\', \'short_name\': \'AUTOMATED TEST CLIENT\', \'br_cnpj\': \'123809128312\', \'br_im\': \'213798238974324\', \'br_ie\': \'7893248932794324\', \'address_id\': 7665, \'is_inserted\': False, \'skin_id\': 1, \'plan_id\': 1, \'organization\': \'CFR-100000\', \'is_api_connected\': False}]\n(Background on this error at: http://sqlalche.me/e/13/9h9h)'}}
Is the client post function seriously only expecting strings for json? It seems that the problem goes away when I use only strings, but I'm not expecting that on the API.
Even if I include "'Content-Type': 'application/json'" on the headers, I get the same error. What could be happening?

Having trouble getting AWS Lambda to send prompt message to Lex

I am trying to make a template to write new functions for AWS Lambda to validate input from Lex. I am not having any trouble with fulfilling correct inputs, only in getting Lex to relay the prompt messages when the slots are incorrectly filled.
def hello_world(intent_request):
object_of_greeting = get_slots(intent_request)["HelloObject"]
type_of_greeting = get_slots(intent_request)["GreetingType"]
# add more of these calls if more slots are involved
source = intent_request['invocationSource']
if source == 'DialogCodeHook':
# Perform basic validation on the supplied input slot(s).
# Use the elicitSlot dialog action to re-prompt for the first violation detected.
slots = get_slots(intent_request)
validation_result = validate_greeting(object_of_greeting, type_of_greeting)
if not validation_result['isValid']:
slots[validation_result['violatedSlot']] = None
return elicit_slot(intent_request['sessionAttributes'],
intent_request['currentIntent']['name'],
slots,
validation_result['violatedSlot'],
validation_result['message'])
Etc... Then, the validation function:
def validate_greeting(object_of_greeting, type_of_greeting):
objects = ['world', 'kitty', 'my dear']
# these should match the listed examples of their corresponding slot, which is 'HelloObject' here
if object_of_greeting is not None and object_of_greeting.lower() not in objects:
return build_validation_result(False,
'HelloObject',
f'You cannot greet {object_of_greeting},
would you like to greet someone else?')
greetings = ['hello', "gday", "whassup"]
if type_of_greeting is not None and type_of_greeting.lower() not in greetings:
return build_validation_result(False,
'GreetingType',
f'You cannot say {type_of_greeting} to anyone.
How about gday or whassup?')
return build_validation_result(True, None, None)
I am able to return a correct JSON with a test script for the build_validation_result function, like the following:
{
'isValid': False,
'violatedSlot': 'HelloObject',
'message': {
'contentType': 'PlainText',
'content': 'You cannot greet foobar, would you like to greet someone else?
}
}
I have also written a Lambda test for false inputs, which gives the following response:
{
"sessionAttributes": {},
"dialogAction": {
"type": "ElicitSlot",
"intentName": "HelloWorld",
"slots": {
"HelloObject": null,
"GreetingType": "howdy"
},
"slotToElicit": "HelloObject",
"message": {
"contentType": "PlainText",
"content": "You cannot greet foobar, would you like to greet someone else?"
}
}
}
Somehow, the validation result isn't making its way back to Lex, and so it's only returning the console prompt, 'Who would you like to greet?' over and over before giving up after n-attempts.
My testing suggests that the correct JSONs are being output from the Lambda function, so I don't know why I can never get their prompt messages to appear in the chat bot. Any help much appreciated. Will post more details if required.

Adding session attributes in Python for Alexa skills

I have 3 slots (account, dollar_value, recipient_first) within my intent schema for an Alexa skill and I want to save whatever slots are provided by the speaker in the session Attributes.
I am using the following methods to set session attributes:
def create_dollar_value_attribute(dollar_value):
return {"dollar_value": dollar_value}
def create_account_attribute(account):
return {"account": account}
def create_recipient_first_attribute(recipient_first):
return {"recipient_first": recipient_first}
However, as you may guess, if I want to save more than one slot as data in sessionAttributes, the sessionAttributes is overwritten as in the following case:
session_attributes = {}
if session.get('attributes', {}) and "recipient_first" not in session.get('attributes', {}):
recipient_first = intent['slots']['recipient_first']['value']
session_attributes = create_recipient_first_attribute(recipient_first)
if session.get('attributes', {}) and "dollar_value" not in session.get('attributes', {}):
dollar_value = intent['slots']['dollar_value']['value']
session_attributes = create_dollar_value_attribute(dollar_value)
The JSON response from my lambda function for a speech input in which two slots (dollar_value and recipient_first) were provided is as follows (my guess is that the create_dollar_value_attribute method in the second if statement is overwriting the first):
{
"version": "1.0",
"response": {
"outputSpeech": {
"type": "PlainText",
"text": "Some text output"
},
"card": {
"content": "SessionSpeechlet - Some text output",
"title": "SessionSpeechlet - Send Money",
"type": "Simple"
},
"reprompt": {
"outputSpeech": {
"type": "PlainText"
}
},
"shouldEndSession": false
},
"sessionAttributes": {
"dollar_value": "30"
}
}
The correct response for sessionAttributes should be:
"sessionAttributes": {
"dollar_value": "30",
"recipient_first": "Some Name"
},
How do I create this response? Is there a better way to add values to sessionAttributes in the JSON response?

The easiest way to add sessionAttributes with Python in my opinion seems to be by using a dictionary. For example, if you want to store some of the slots for future in the session attributes:
session['attributes']['slotKey'] = intent['slots']['slotKey']['value']
Next, you can just pass it on to the build response method:
buildResponse(session['attributes'], buildSpeechletResponse(title, output, reprompt, should_end_session))
The implementation in this case:
def buildSpeechletResponse(title, output, reprompt_text, should_end_session):
return {
'outputSpeech': {
'type': 'PlainText',
'text': output
},
'card': {
'type': 'Simple',
'title': "SessionSpeechlet - " + title,
'content': "SessionSpeechlet - " + output
},
'reprompt': {
'outputSpeech': {
'type': 'PlainText',
'text': reprompt_text
}
},
'shouldEndSession': should_end_session
}
def buildResponse(session_attributes, speechlet_response):
return {
'version': '1.0',
'sessionAttributes': session_attributes,
'response': speechlet_response
}
This creates the sessionAttributes in the recommended way in the Lambda response JSON.
Also just adding a new sessionAttribute doesn't overwrite the last one if it doesn't exist. It will just create a new key-value pair.
Do note, that this may work well in the service simulator but may return a key attribute error when testing on an actual Amazon Echo. According to this post,
On Service Simulator, sessions starts with Session:{ ... Attributes:{}, ... }
When sessions start on the Echo, Session does not have an Attributes key at all.
The way I worked around this was to just manually create it in the lambda handler whenever a new session is created:
if event['session']['new']:
event['session']['attributes'] = {}
onSessionStarted( {'requestId': event['request']['requestId'] }, event['session'])
if event['request']['type'] == 'IntentRequest':
return onIntent(event['request'], event['session'])

First, you have to define the session_attributes.
session_attributes = {}
Then instead of using
session_attributes = create_recipient_first_attribute(recipient_first)
You should use
session_attributes.update(create_recipient_first_attribute(recipient_first)).
The problem you are facing is because you are reassigning the session_attributes. Instead of this, you should just update the session_attributes.
So your final code will become:
session_attributes = {}
if session.get('attributes', {}) and "recipient_first" not in session.get('attributes', {}):
recipient_first = intent['slots']['recipient_first']['value']
session_attributes.update(create_recipient_first_attribute(recipient_first))
if session.get('attributes', {}) and "dollar_value" not in session.get('attributes', {}):
dollar_value = intent['slots']['dollar_value']['value']
session_attributes.update(create_dollar_value_attribute(dollar_value))

The ASK SDK for Python provides an attribute manager, to manage request/session/persistence level attributes in the skill. You can look at the color picker sample, to see how to use these attributes in skill development.

Take a look at the below:
account = intent['slots']['account']['value']
dollar_value = intent['slots']['dollar_value']['value']
recipient_first = intent['slots']['recipient_first']['value']
# put your data in a dictionary
attributes = {
'account':account,
'dollar_value':dollar_value,
'recipient_first':recipient_first
}
Put the attributes dictionary in 'sessionAttributes' in your response. You should get it back in 'sessionAttributes' once Alexa replies to you.
Hope this helps.

The following code snippet will also prevent overwriting the session attributes:
session_attributes = session.get('attributes', {})
if "recipient_first" not in session_attributes:
session_attributes['recipient_first'] = intent['slots']['recipient_first']['value']
if "dollar_value" not in session_attributes:
session_attributes['dollar_value'] = = intent['slots']['dollar_value']['value']

Google Analytics API(Client library for Python) - Error handling

I am using Batching requests in Google Analytics API(Python). Link to Batching : https://developers.google.com/api-client-library/python/guide/batch
Batching works fine when all the records via .add() are correct(valid). When one or more values are invalid, then the batching fails for all the records.
I added a call back function to handle the error and I saw that BAtching request is failing for all the records in the batch ( as opposed to only the invalid record). Is there a way to handle the error and skip the row/record which is invalid and continue with the rest of the records in the batch?
Below is the sample code I used and the error message :
def add_user_callback(request_id, response, exception):
if exception:
print "error :",exception
else:
print "successful"
def main():
## code to set the account, property and other variables
batch.add(service.management().webpropertyUserLinks().insert(
accountId=account_id,
webPropertyId=property_at,
body={
'permissions': {
'local': [
'READ_AND_ANALYZE'
]
},
'userRef': {
'email': 'valid_address#domain.com'
}
}))
batch.add(service.management().webpropertyUserLinks().insert(
accountId=account_id,
webPropertyId=property_at,
body={
'permissions': {
'local': [
'READ_AND_ANALYZE'
]
},
'userRef': {
'email': 'invalid_address#ddomain.com' ## i used a dummy id : pppp#domain.com
}
}))
batch.execute()
#Error :
#error : <HttpError 400 when requesting https://www.googleapis.com/analytics/v3/management/accounts/62974313/webproperties/UA-62974313-35/entityUserLinks?alt=json returned "Value for field user.email = ppppp#domain.com is not valid.">
#error : <HttpError 400 when requesting https://www.googleapis.com/analytics/v3/management/accounts/62974313/webproperties/UA-62974313-11/entityUserLinks?alt=json returned "Value for field user.email = ppppp#domain.com is not valid.">
Please let me know if you need more info.

Let's assume you have a list of users you want to add to profiles stored in a list users.
You can remove the bad emails with the following callback function:
def call_back(request_id, response, exception):
if exception is not None:
if isinstance(exception, HttpError):
message = json.loads(exception.content)['error']['message']
bad = 'Value for field user.email = (\S*) is not valid.'
match = re.match(bad, message)
if match:
bad_user = match.group(1)
if bad_user in users:
users.remove(bad_user)
else:
print response
After all the failed calls return you can re-attempt the batch call again by looping through the users and constructing a new batch request:
batch = BatchHttpRequest(callback=call_back)
for user in users:
request = analytics.management().profileUserLinks().insert(
accountId=ACCOUNT_ID,
webPropertyId=PROFILE_ID,
profileId=profile,
body={
'permissions': {'local': ['READ_AND_ANALYZE']},
'userRef': {'email': user}
}
)
batch.add(request, request_id=PROFILE_ID + user)
batch.execute()

Why is code not executing after response received

I'm running the code below and it takes the user to PayPal to make a payment and then returns them to the return_url as expected. However the code doesn't execute any further and it doesn't execute the payment.
I have based my code on https://github.com/paypal/rest-api-sdk-python:
class PayPalHandler(tornado.web.RequestHandler):
def get(self):
logging.basicConfig(level=logging.INFO)
paypal.configure({
"mode": PAYPAL_MODE,
"client_id": PAYPAL_CLIENT_ID,
"client_secret": PAYPAL_CLIENT_SECRET})
payment = paypal.Payment({
"intent": "sale",
"payer": {
"payment_method": "paypal" },
"redirect_urls": {
"return_url": "http://127.0.0.1:8000/ty",
"cancel_url": "http://127.0.0.1:8000/" },
"transactions": [ {
"item_list": {
"items": [{
"name": "membership",
"price": "2.00",
"currency": "GBP",
"quantity": 1 }]},
"amount": {
"total": "2.00",
"currency": "GBP" },
"description": "One of membership fee." } ] } )
redirect_url = ""
if payment.create():
print("Payment[%s] created successfully"%(payment.id))
for link in payment.links:
if link.method == "REDIRECT":
redirect_url = link.href
print("Redirect for approval: %s"%(redirect_url))
return self.redirect(redirect_url)
else:
print("Error while creating payment.")
print(payment.error)
response = payment.to_dict()
print response
payment = paypal.Payment.find(payment.id)
if payment.execute({"payer_id": response['payer_id']}):
print ("Payment executed successfully")
else:
print(payment.error) # Error Hash
print payment.to_dict()
print userData
So in the example at https://devtools-paypal.com/guide/pay_paypal/python?success=true&token=EC-8JL96732FP068791F&PayerID=QQGSRNHDACTLJ. Step 5 is not happening and no response is sent from PayPal?

This is Avi from PayPal here. I am not super familiar with Tornado, but after the line return self.redirect(redirect_url) happens in your code, and returns the user to the return_url, in payment.execute({"payer_id": response['payer_id']}) are you getting the payer_id correctly? Payer_id is returned appended to the return_url as one of the parameters in the format http://<return_url>?token=EC-60U79048BN7719609&PayerID=7E7MGXCWTTKK2. Also, what is the status of the payment after you execute payment = paypal.Payment.find(payment.id). The other suggestion I would have is to see if print payment.error prints a useful debug message and a debug_id which paypal merchant technical services can use to look at the issue.

You need other url where papypal redirects when the payment had been successful, where you will receive the token and the PayerID. In that GET method, you can put this part of the code (pseudocode):
payerid_param = request.get('PayerID')
payment = paypal.Payment.find(db_payment.id)
if payment.execute({"payer_id": payerid_param}):
print ("Payment executed successfully")
else:
print(payment.error) # Error Hash
You will need to save the payment_id between calls.

Why you use
return self.redirect(redirect_url)
I think you can use just
self.redirect(redirect_url)
I've never seen return statement in Tornado handlers.