We Keep Coding

Django: byte indices must be integers or slices, not str error

I am trying to get some data from a payload using a webhook, in the response that I sent along side I added the meta data that should return with the payload and this is how I did it (code below), if you take a close look at the keys, there I one called meta, now I am trying to get some of the data that is in the meta from my webhook view like this:
views.py webhook
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = request.body
order_id = payload['meta']['order_id']
print(order_id)
return HttpResponse("testing...")
Sending the reponse to the payment gateway
#csrf_exempt
def process_payment(request, name, email, amount, order_id):
auth_token = "FLWSECK_TEST-9efb9ee17d8afe40c6c890294a1163de-X"
hed = {'Authorization': 'Bearer ' + auth_token}
data = {
"tx_ref":''+str(math.floor(1000000 + random.random()*9000000)),
"amount":amount,
"currency":"USD",
"order_id":order_id,
"redirect_url":f"http://localhost:3000/service-detail/booking/confirmation/{order_id}",
"payment_options":"card",
"meta":{
"order_id":order_id,
"consumer_id":23,
"consumer_mac":"92a3-912ba-1192a"
},
"customer":{
"email":email,
"name":name,
"order_id":order_id
},
"customizations":{
"title":"ForecastFaceoff",
"description":"Leading Political Betting Platform",
"logo":"https://i.im.ge/2022/08/03m/FELzix.stridearn-high-quality-logo-circle.jpg"
}
}
url = ' https://api.flutterwave.com/v3/payments'
response = requests.post(url, json=data, headers=hed)
response=response.json()
link=response['data']['link']
return redirect(link)
payload
{
"event": "charge.completed",
"data": {
"id": 4136234873,
"tx_ref": "6473247093",
"flw_ref": "FLW-MOCK-b33e86ab2342316fec664110e8eb842a3c2f956",
"device_fingerprint": "df38c8854324598c54e16feacc65348a5e446",
"amount": 152,
"currency": "USD",
"charged_amount": 152,
"app_fee": 5.78,
"merchant_fee": 0,
"processor_response": "Approved. Successful",
"auth_model": "VBVSECUREertrCODE",
"ip": "52.209.154.143",
"narration": "CARD Transaction ",
"status": "successful",
"payment_type": "card",
"created_at": "2023-02-06T11:19:45.000Z",
"account_id": 685622,
"customer": {
"id": 1970338,
"name": "destiny ",
"phone_number": null,
"email": "******#gmail.com",
"created_at": "2023-02-06T11:19:45.000Z"
},
"card": {
"first_6digits": "653188",
"last_4digits": "2340",
"issuer": "MASTERCARD CREDIT",
"country": "NG",
"type": "MASTERCARD",
"expiry": "49/32"
}
},
"event.type": "CARD_TRANSACTION"
}
The metadata is not even showing in the payload, must it show up there in the payload before I can grab the data that is sent with it?

request.body is byte not a dict, so you need to load the string as JSON.
payload = json.loads(request.body)

The meta data is not in the payload dictionary. At first try to extract the data key from the payload like this:
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = json.loads(request.body.decode("utf-8"))
data = payload.get("data")
if data:
order_id = data.get("meta", {}).get("order_id")
print(order_id)
return HttpResponse("just testing")
Edit:
The error message
as you described above in comment JSONDecodeError: Expecting value: line 1 column 1 (char 0) indicates that json.loads() is expecting a JSON string as input.
It may be possible that request body is empty. You can check that by printing the request.body before calling json.loads().
Try this:
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = json.loads(request.body.decode("utf-8"))
data = payload.get("data")
if data and "meta" in data:
order_id = data.get("meta").get("order_id")
print(order_id)
else:
print("Meta key not found in data or data not found in payload")
print(payload)
return HttpResponse("just testing")
It would tell you whether the "meta" is present in data dictionary or not and also prints payload dict at the end.

Compare expected vs actual json response for a get request in Python

You will get a small json response back when you go to this site https://reqres.in/api/users/2
I am saving the response in a variable(actual). I have also put the response in another variable(expected).Both responses are same. I am changing the values to test failed cases. The ultimate goal is to compare 2 and make sure they match.
I have 2 functions, 1 compares keys and value of both dictionaries and the other function sorts the dictionaries. Code below:
import json
import requests
response = requests.get('https://reqres.in/api/users/2')
#actual_response saves the json as we get it from url above
actual_response= json.loads(response.text)
#expected response is saved after using pretty json that will be used to testing/comparing actual vs expected
expected_response={
"data": {
"id": 2,
"first_name": "Janet",
"last_name": "Weaver",
"avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/josephstein/128.jpg"
}
}
# sort the key values before comparing
def dict_sort(dictA,dictB):
dictA, dictB = json.dumps(dictA, sort_keys=True), json.dumps(dictB, sort_keys=True)
dictA == dictB
#if there are any failure due to mismatch in key value the function below will show that
def key_diff(dictA,dictB):
for key,value in dictA.items():
for keyB,valueB in dictB.items():
for k,v in value.items():
for k2,v2 in valueB.items():
if(key!= keyB):
print('Expected',key,' but got',keyB)
if(k!=k2):
print('Expected', k, ' but got', k2)
if(v!=v2):
print('Expected', v, ' but got', v2)
else:
print()
dict_sort(actual_response,expected_response)
if(actual_response==expected_response):
print('Passed')
else:
print('Failed')
key_diff(actual_response,expected_response)
Problem: The test passes when there is no difference.However if there is any difference the order goes crazy. Here is an example where I changed data to dat inside expected response:
Expected data but got dat
Expected id but got last_name
Expected 2 but got Weaver
Should the sort function be more specific rather than using sort_keys=True?By the way thought about **args but I don't think that is a good choice in this scenario.
Thank You for your expert comment and time.

I advise to use unittest and avoid using so much nested for loops
from unittest import TestCase
import pandas as pd
import requests
def mocked_server_response():
expected = {"data": {"id": 2, "first_name": "Janet", "last_name": "Weaver",
"avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/josephstein/128.jpg"}}
data = expected['data']
df = pd.DataFrame(my_dict['data'], index=[0])
return [expected, data, df]
At this point, the mocked_server_response()will get you this :
Out[27]:
id first_name last_name avatar
0 2 Janet Weaver https://s3.amazonaws.com/uifaces/faces/twitter...
Now, you can easily make test in a class.
class TestServerResponse(TestCase):
real_response = requests.get('https://reqres.in/api/users/2')
def setUp(self):
self.actual_response = real_response
def response(self):
self.assertEqual(self.actual_response, mocked_server_response()[0])
def test_data_in_response(self):
self.assertEqual(self.actual_response['data'], mocked_server_response()[1])
def test_dataframe(self):
self.assertEqual(pd.DataFrame(self.actual_response['data'], index=[0]), mocked_server_response()[2])

Key order is not guaranteed in Python versions under 3.7; you should use collections.OrderedDict when you need to create an object that remembers key order.
In Python 3.7 the insertion ordered is preserved, so your keys will always match.

import requests
import json
# Here I am converting the expected payload in dictionary
expected_payload = json.loads("""[
{
"key": "data-center",
"value": "All",
"values": [
"1",
"2"
]
},
{
"key": "router",
"value": "All",
"values": [
"cisco",
"juniper"
]
},
{
"key": "virtual-machine",
"value": "All",
"values": [
"dell",
"hp",
"None"
]
},
]""")
def test_get_all_system_attributes():
url = "http://" + str(ipaddr) + ":" + str(port) + "/Service/" + "system_attribute/"
payload = {}
headers = {
'Content-Type': 'application/json'
}
actual_response = requests.request("GET", url, headers=headers, data=payload)
assert json.loads(actual_response.text) == expected_payload
# json.loads(actual_response.text) will convert the response in dictionary
# using assert I am comparing the actual_response with exepcted_response
if __name__ == '__main__':
test_get_all_system_attributes()

Adding session attributes in Python for Alexa skills

I have 3 slots (account, dollar_value, recipient_first) within my intent schema for an Alexa skill and I want to save whatever slots are provided by the speaker in the session Attributes.
I am using the following methods to set session attributes:
def create_dollar_value_attribute(dollar_value):
return {"dollar_value": dollar_value}
def create_account_attribute(account):
return {"account": account}
def create_recipient_first_attribute(recipient_first):
return {"recipient_first": recipient_first}
However, as you may guess, if I want to save more than one slot as data in sessionAttributes, the sessionAttributes is overwritten as in the following case:
session_attributes = {}
if session.get('attributes', {}) and "recipient_first" not in session.get('attributes', {}):
recipient_first = intent['slots']['recipient_first']['value']
session_attributes = create_recipient_first_attribute(recipient_first)
if session.get('attributes', {}) and "dollar_value" not in session.get('attributes', {}):
dollar_value = intent['slots']['dollar_value']['value']
session_attributes = create_dollar_value_attribute(dollar_value)
The JSON response from my lambda function for a speech input in which two slots (dollar_value and recipient_first) were provided is as follows (my guess is that the create_dollar_value_attribute method in the second if statement is overwriting the first):
{
"version": "1.0",
"response": {
"outputSpeech": {
"type": "PlainText",
"text": "Some text output"
},
"card": {
"content": "SessionSpeechlet - Some text output",
"title": "SessionSpeechlet - Send Money",
"type": "Simple"
},
"reprompt": {
"outputSpeech": {
"type": "PlainText"
}
},
"shouldEndSession": false
},
"sessionAttributes": {
"dollar_value": "30"
}
}
The correct response for sessionAttributes should be:
"sessionAttributes": {
"dollar_value": "30",
"recipient_first": "Some Name"
},
How do I create this response? Is there a better way to add values to sessionAttributes in the JSON response?

The easiest way to add sessionAttributes with Python in my opinion seems to be by using a dictionary. For example, if you want to store some of the slots for future in the session attributes:
session['attributes']['slotKey'] = intent['slots']['slotKey']['value']
Next, you can just pass it on to the build response method:
buildResponse(session['attributes'], buildSpeechletResponse(title, output, reprompt, should_end_session))
The implementation in this case:
def buildSpeechletResponse(title, output, reprompt_text, should_end_session):
return {
'outputSpeech': {
'type': 'PlainText',
'text': output
},
'card': {
'type': 'Simple',
'title': "SessionSpeechlet - " + title,
'content': "SessionSpeechlet - " + output
},
'reprompt': {
'outputSpeech': {
'type': 'PlainText',
'text': reprompt_text
}
},
'shouldEndSession': should_end_session
}
def buildResponse(session_attributes, speechlet_response):
return {
'version': '1.0',
'sessionAttributes': session_attributes,
'response': speechlet_response
}
This creates the sessionAttributes in the recommended way in the Lambda response JSON.
Also just adding a new sessionAttribute doesn't overwrite the last one if it doesn't exist. It will just create a new key-value pair.
Do note, that this may work well in the service simulator but may return a key attribute error when testing on an actual Amazon Echo. According to this post,
On Service Simulator, sessions starts with Session:{ ... Attributes:{}, ... }
When sessions start on the Echo, Session does not have an Attributes key at all.
The way I worked around this was to just manually create it in the lambda handler whenever a new session is created:
if event['session']['new']:
event['session']['attributes'] = {}
onSessionStarted( {'requestId': event['request']['requestId'] }, event['session'])
if event['request']['type'] == 'IntentRequest':
return onIntent(event['request'], event['session'])

First, you have to define the session_attributes.
session_attributes = {}
Then instead of using
session_attributes = create_recipient_first_attribute(recipient_first)
You should use
session_attributes.update(create_recipient_first_attribute(recipient_first)).
The problem you are facing is because you are reassigning the session_attributes. Instead of this, you should just update the session_attributes.
So your final code will become:
session_attributes = {}
if session.get('attributes', {}) and "recipient_first" not in session.get('attributes', {}):
recipient_first = intent['slots']['recipient_first']['value']
session_attributes.update(create_recipient_first_attribute(recipient_first))
if session.get('attributes', {}) and "dollar_value" not in session.get('attributes', {}):
dollar_value = intent['slots']['dollar_value']['value']
session_attributes.update(create_dollar_value_attribute(dollar_value))

The ASK SDK for Python provides an attribute manager, to manage request/session/persistence level attributes in the skill. You can look at the color picker sample, to see how to use these attributes in skill development.

Take a look at the below:
account = intent['slots']['account']['value']
dollar_value = intent['slots']['dollar_value']['value']
recipient_first = intent['slots']['recipient_first']['value']
# put your data in a dictionary
attributes = {
'account':account,
'dollar_value':dollar_value,
'recipient_first':recipient_first
}
Put the attributes dictionary in 'sessionAttributes' in your response. You should get it back in 'sessionAttributes' once Alexa replies to you.
Hope this helps.

The following code snippet will also prevent overwriting the session attributes:
session_attributes = session.get('attributes', {})
if "recipient_first" not in session_attributes:
session_attributes['recipient_first'] = intent['slots']['recipient_first']['value']
if "dollar_value" not in session_attributes:
session_attributes['dollar_value'] = = intent['slots']['dollar_value']['value']

How do I parse this data with fabric?

I use fabric perform scrapyd task, the server returns the id of the task being performed, but I want to get this code below and put all id in list, but when I use r.status , there occur error: '_AttributeString' object has no attribute 'status' error, how do I get all id?the code blow:
#task
def stop_slave_machine(slave_ip = None):
jobs_id = []
with cd("/spider/distributed/wzws"):
if not None:
r = local("curl http://%s:%s/listjobs.json?project=WzwsSpider" % (slave_ip, scrapyd_port))
print(r.status)
and the server return data:
{"status": "ok", "running": [{"start_time": "2016-03-28 18:21:21.951943", "id": "d10eae6cf4ce11e5a6646cae8b23c5da", "spider": "wzws"}, {"start_time": "2016-03-28 18:21:26.945244", "id": "d11a47f4f4ce11e5a6646cae8b23c5da", "spider": "wzws"}, {"start_time": "2016-03-28 18:21:31.941162", "id": "d12320ccf4ce11e5a6646cae8b23c5da", "spider": "wzws"}, {"start_time": "2016-03-28 18:21:36.941122", "id": "d12975b2f4ce11e5a6646cae8b23c5da", "spider": "wzws"}, {"start_time": "2016-03-28 18:21:41.941010", "id": "d131096cf4ce11e5a6646cae8b23c5da", "spider": "wzws"}], "finished": [], "pending": [], "node_name": "XXXXXXX"}

That's a JSON body being returned. You can use python's json library to turn the response into a python object. From there you can iterate over the list of "running" to extract out the id for each.
Something like this:
from json import loads
# turn r into a python object as long as r is a string (hence loads not load)
returned = loads(r)
# Make a list ids from a list comprehension where we pull out the value
# id from each item in the list 'running' from the object returned
ids = [ r["id"] for r in returned["running"] ]

KeyError: 'Bytes_Written' python

I do not understand why I get this error Bytes_Written is in the dataset but why can't python find it? I am getting this information(see dataset below) from a VM, I want to select Bytes_Written and Bytes_Read and then subtract the previous values from current value and print a json object like this
{'Bytes_Written': previousValue-currentValue, 'Bytes_Read': previousValue-currentValue}
here is what the data looks like:
{
"Number of Devices": 2,
"Block Devices": {
"bdev0": {
"Backend_Device_Path": "/dev/disk/by-path/ip-192.168.26.1:3260-iscsi-iqn.2010-10.org.openstack:volume-d1c8e7c6-8c77-444c-9a93-8b56fa1e37f2-lun-010.0.0.142",
"Capacity": "2147483648",
"Guest_Device_Name": "vdb",
"IO_Operations": "97069",
"Bytes_Written": "34410496",
"Bytes_Read": "363172864"
},
"bdev1": {
"Backend_Device_Path": "/dev/disk/by-path/ip-192.168.26.1:3260-iscsi-iqn.2010-10.org.openstack:volume-b27110f9-41ba-4bc6-b97c-b5dde23af1f9-lun-010.0.0.146",
"Capacity": "2147483648",
"Guest_Device_Name": "vdb",
"IO_Operations": "93",
"Bytes_Written": "0",
"Bytes_Read": "380928"
}
}
}
This is the complete code that I am running.
FIELDS = ("Bytes_Written", "Bytes_Read", "IO_Operation")
def counterVolume_one(state):
url = 'http://url'
r = requests.get(url)
data = r.json()
for field in FIELDS:
state[field] += data[field]
return state
state = {"Bytes_Written": 0, "Bytes_Read": 0, "IO_Operation": 0}
while True:
counterVolume_one(state)
time.sleep(1)
for field in FIELDS:
print("{field:s}: {count:d}".format(field=field, count=state[field]))
counterVolume_one(state)

Your returned JSON structure does not have any of these FIELDS = ("Bytes_Written", "Bytes_Read", "IO_Operation") keys directly.
You'll need to modify your code slightly.
data = r.json()
for block_device in data['Block Devices'].iterkeys():
for field in FIELDS:
state[field] += int(data['Block Devices'][block_device][field])