We Keep Coding

Django: byte indices must be integers or slices, not str error

I am trying to get some data from a payload using a webhook, in the response that I sent along side I added the meta data that should return with the payload and this is how I did it (code below), if you take a close look at the keys, there I one called meta, now I am trying to get some of the data that is in the meta from my webhook view like this:
views.py webhook
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = request.body
order_id = payload['meta']['order_id']
print(order_id)
return HttpResponse("testing...")
Sending the reponse to the payment gateway
#csrf_exempt
def process_payment(request, name, email, amount, order_id):
auth_token = "FLWSECK_TEST-9efb9ee17d8afe40c6c890294a1163de-X"
hed = {'Authorization': 'Bearer ' + auth_token}
data = {
"tx_ref":''+str(math.floor(1000000 + random.random()*9000000)),
"amount":amount,
"currency":"USD",
"order_id":order_id,
"redirect_url":f"http://localhost:3000/service-detail/booking/confirmation/{order_id}",
"payment_options":"card",
"meta":{
"order_id":order_id,
"consumer_id":23,
"consumer_mac":"92a3-912ba-1192a"
},
"customer":{
"email":email,
"name":name,
"order_id":order_id
},
"customizations":{
"title":"ForecastFaceoff",
"description":"Leading Political Betting Platform",
"logo":"https://i.im.ge/2022/08/03m/FELzix.stridearn-high-quality-logo-circle.jpg"
}
}
url = ' https://api.flutterwave.com/v3/payments'
response = requests.post(url, json=data, headers=hed)
response=response.json()
link=response['data']['link']
return redirect(link)
payload
{
"event": "charge.completed",
"data": {
"id": 4136234873,
"tx_ref": "6473247093",
"flw_ref": "FLW-MOCK-b33e86ab2342316fec664110e8eb842a3c2f956",
"device_fingerprint": "df38c8854324598c54e16feacc65348a5e446",
"amount": 152,
"currency": "USD",
"charged_amount": 152,
"app_fee": 5.78,
"merchant_fee": 0,
"processor_response": "Approved. Successful",
"auth_model": "VBVSECUREertrCODE",
"ip": "52.209.154.143",
"narration": "CARD Transaction ",
"status": "successful",
"payment_type": "card",
"created_at": "2023-02-06T11:19:45.000Z",
"account_id": 685622,
"customer": {
"id": 1970338,
"name": "destiny ",
"phone_number": null,
"email": "******#gmail.com",
"created_at": "2023-02-06T11:19:45.000Z"
},
"card": {
"first_6digits": "653188",
"last_4digits": "2340",
"issuer": "MASTERCARD CREDIT",
"country": "NG",
"type": "MASTERCARD",
"expiry": "49/32"
}
},
"event.type": "CARD_TRANSACTION"
}
The metadata is not even showing in the payload, must it show up there in the payload before I can grab the data that is sent with it?

request.body is byte not a dict, so you need to load the string as JSON.
payload = json.loads(request.body)

The meta data is not in the payload dictionary. At first try to extract the data key from the payload like this:
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = json.loads(request.body.decode("utf-8"))
data = payload.get("data")
if data:
order_id = data.get("meta", {}).get("order_id")
print(order_id)
return HttpResponse("just testing")
Edit:
The error message
as you described above in comment JSONDecodeError: Expecting value: line 1 column 1 (char 0) indicates that json.loads() is expecting a JSON string as input.
It may be possible that request body is empty. You can check that by printing the request.body before calling json.loads().
Try this:
#csrf_exempt
#require_http_methods(['POST', 'GET'])
def webhook(request):
payload = json.loads(request.body.decode("utf-8"))
data = payload.get("data")
if data and "meta" in data:
order_id = data.get("meta").get("order_id")
print(order_id)
else:
print("Meta key not found in data or data not found in payload")
print(payload)
return HttpResponse("just testing")
It would tell you whether the "meta" is present in data dictionary or not and also prints payload dict at the end.

Flask is caching the returned response

i have an app that return a json response (no HTML).
the problem is the response get cached like the following.
from flask import jsonify, make_response
# base api template
baseApi = {'status':None, 'message':None, 'data':{}}
#app.route('/getdata1')
def getData1():
result = baseApi.copy()
result['status'] = 'success'
result['data'] = {'data1':'some data'}
return make_response(jsonify(result), 200)
#app.route('/getdata2')
def getData2():
result = baseApi.copy()
result['status'] = 'success'
result['data'] = {'data2':'some other data'}
return make_response(jsonify(result), 200)
if i go to the link /getdata1 then the rsponse will be as i expected
{
"data": {
"data1": "some data"
},
"message": null,
"status": "success"
}
but if after that i run the link /getdata2 i got the two results back
{
"data": {
"data1": "some data",
"data2": "some other data"
},
"message": null,
"status": "success"
}
NOTE : the cache not happen in the browser , i have a mobile app , if i go to /getdata1 in the mobile and then try /getdata2 in the PC i got the same thing.
i tried this but it not seem to work !
#api.after_request
def after_request(response):
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate, public, max-age=0"
response.headers["Expires"] = 0
response.headers["Pragma"] = "no-cache"
return response
thanks a lot!!

i guess you should try to use a deepcopy instead of shallow copy of your dictionnary.
the problem is not the cash but rather the dictionary has the same reference :
if we read about the definition of shallow copy
A shallow copy creates a new object which stores the reference of the original elements.
To get a fully independent copy of an object you can use the copy.deepcopy() which is the case of this question.

Compare expected vs actual json response for a get request in Python

You will get a small json response back when you go to this site https://reqres.in/api/users/2
I am saving the response in a variable(actual). I have also put the response in another variable(expected).Both responses are same. I am changing the values to test failed cases. The ultimate goal is to compare 2 and make sure they match.
I have 2 functions, 1 compares keys and value of both dictionaries and the other function sorts the dictionaries. Code below:
import json
import requests
response = requests.get('https://reqres.in/api/users/2')
#actual_response saves the json as we get it from url above
actual_response= json.loads(response.text)
#expected response is saved after using pretty json that will be used to testing/comparing actual vs expected
expected_response={
"data": {
"id": 2,
"first_name": "Janet",
"last_name": "Weaver",
"avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/josephstein/128.jpg"
}
}
# sort the key values before comparing
def dict_sort(dictA,dictB):
dictA, dictB = json.dumps(dictA, sort_keys=True), json.dumps(dictB, sort_keys=True)
dictA == dictB
#if there are any failure due to mismatch in key value the function below will show that
def key_diff(dictA,dictB):
for key,value in dictA.items():
for keyB,valueB in dictB.items():
for k,v in value.items():
for k2,v2 in valueB.items():
if(key!= keyB):
print('Expected',key,' but got',keyB)
if(k!=k2):
print('Expected', k, ' but got', k2)
if(v!=v2):
print('Expected', v, ' but got', v2)
else:
print()
dict_sort(actual_response,expected_response)
if(actual_response==expected_response):
print('Passed')
else:
print('Failed')
key_diff(actual_response,expected_response)
Problem: The test passes when there is no difference.However if there is any difference the order goes crazy. Here is an example where I changed data to dat inside expected response:
Expected data but got dat
Expected id but got last_name
Expected 2 but got Weaver
Should the sort function be more specific rather than using sort_keys=True?By the way thought about **args but I don't think that is a good choice in this scenario.
Thank You for your expert comment and time.

I advise to use unittest and avoid using so much nested for loops
from unittest import TestCase
import pandas as pd
import requests
def mocked_server_response():
expected = {"data": {"id": 2, "first_name": "Janet", "last_name": "Weaver",
"avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/josephstein/128.jpg"}}
data = expected['data']
df = pd.DataFrame(my_dict['data'], index=[0])
return [expected, data, df]
At this point, the mocked_server_response()will get you this :
Out[27]:
id first_name last_name avatar
0 2 Janet Weaver https://s3.amazonaws.com/uifaces/faces/twitter...
Now, you can easily make test in a class.
class TestServerResponse(TestCase):
real_response = requests.get('https://reqres.in/api/users/2')
def setUp(self):
self.actual_response = real_response
def response(self):
self.assertEqual(self.actual_response, mocked_server_response()[0])
def test_data_in_response(self):
self.assertEqual(self.actual_response['data'], mocked_server_response()[1])
def test_dataframe(self):
self.assertEqual(pd.DataFrame(self.actual_response['data'], index=[0]), mocked_server_response()[2])

Key order is not guaranteed in Python versions under 3.7; you should use collections.OrderedDict when you need to create an object that remembers key order.
In Python 3.7 the insertion ordered is preserved, so your keys will always match.

import requests
import json
# Here I am converting the expected payload in dictionary
expected_payload = json.loads("""[
{
"key": "data-center",
"value": "All",
"values": [
"1",
"2"
]
},
{
"key": "router",
"value": "All",
"values": [
"cisco",
"juniper"
]
},
{
"key": "virtual-machine",
"value": "All",
"values": [
"dell",
"hp",
"None"
]
},
]""")
def test_get_all_system_attributes():
url = "http://" + str(ipaddr) + ":" + str(port) + "/Service/" + "system_attribute/"
payload = {}
headers = {
'Content-Type': 'application/json'
}
actual_response = requests.request("GET", url, headers=headers, data=payload)
assert json.loads(actual_response.text) == expected_payload
# json.loads(actual_response.text) will convert the response in dictionary
# using assert I am comparing the actual_response with exepcted_response
if __name__ == '__main__':
test_get_all_system_attributes()

Python - JSON Module how to query nested objects

I am trying to build a database-backed system using Python as an engine and JSON as the DB file. Currently, there are three files:
# functions.py
import json
from pprint import pprint
with open('db.json') as data_file:
data = json.load(data_file)
def getStudentByUsername(username, record):
detail = json.load(open('db.json'))["students"][username][record]
pprint(detail)
# main.py
import functions
functions.getStudentByUsername('youngh', 'age')
{ // db.json
"students": {
"youngh": {
"name": "Hayden Young",
"age": 13,
"email": "user#school.county.sch.uk"
}
}
}
But, I can't find a way to change my students object like this and still query by username:
{ //db.json "students": [
{
"username": "youngh",
"name": "Hayden Young"
}]
}
Any ideas? I'm new to this type of use of Python 3.

In your second example, you have the user attributes in a dictionary inside a list.
You will need to iterate over your list and get the dictionary with the username you seek.
with open('db.json') as dbfile:
for student in json.load(dbfile)["students"]:
if student["username"] == username:
return(student)
Keep in mind that because the usernames are contained in their own separate dictionaries, you can now have duplicate usernames, and the for loop method will only return the first match.

I think you can use lambda:
#let this var be your file output after parsing to json
users = {
"students":
[
{
"username": "youngh",
"name": "Hayden Young"
}
]
}
def getStudentByUsername(username, record):
detail= filter(lambda user: username == user['username'], users["students"])
print (next(detail)[record])
For duplicate usernames, you can do something like this:
def getStudentByUsername(username, record):
detail= filter(lambda user: username.lower() == user['username'].lower(), users["students"])
while True:
try:
print (next(detail)[record])
except StopIteration:
return

How to assert deeply in JSON with python

I am trigerring an API call, and the server response is in JSON format.
the response looks like this:
{
"status": 0,
"not_passed": 1,
"why": [
{
"code": 229,
"reason": "some reason",
}
]
}
I need to assert two thing.
Status and reason
fro status I am using:
r = requests.get_simple(url=Server.MY_SERVER, params=p)
data = json.loads(r.content)
assert data["status"] == 0
but it doesn't work for the 'reason', maybe because the 'reason' is deeper in the nested structure. How can I fix this one?

assert data['why'][0]['reason'] == 'something'
Of course this assumes that data['why'] exists, is a list, and contains a dict as its first element.