Parsing a list of dictionaries passed as a POST parameter - python

I have a list of python dictionaries that look like this:
sandwiches = [
{'bread':'wheat', 'topping':'tomatoes', 'meat':'bacon'},
{'bread':'white', 'topping':'peanut butter', 'meat':'bacon'},
{'bread':'sourdough', 'topping':'cheese', 'meat':'bacon'}
]
I want to pass this as a POST parameter to another Django app. What does the client app need to do to iterate through the list?
I want to do something like:
for sandwich in request.POST['sandwiches']:
print "%s on %s with %s is yummy!" % (sandwich['meat'], sandwich['bread'], sandwich['topping'])
But I don't seem to have a list of dicts when my data arrives at my client app.


You don't say how you're POSTing to the app. My advice would be to serialize the dictionaries as JSON, then simply POST that.
import json, urllib, urllib2
data = json.dumps(sandwiches)
urllib2.urlopen(myurl, urllib.urlencode({'data': data}))
... and on the receiver:
data = request.POST['data']
sandwiches = json.loads(data)


I would use the JSON libraries to serialize your array of dicts into a single string. Send this string as a POST param, and parse the string back into the python datatypes in the Django app using the same library.
http://docs.python.org/library/json.html


firstly make string of the list
import ast, requests
# request send code
dev_list = [{"1": 2}, {"2": 4}]
str_list = str(dev_list) # '[{"1": 2}, {"2": 4}]'
data = {"a" : "mynema",
'str_list': str_list
}
requests.post(url, data )
#request receive side
data = request.data
dev_list = data.get('str_list')
dev_list = ast.literal_eval(dev_list)
#dev_list --> [{"1": 2}, {"2": 4}]
Now you receive list of dictionary.

Related

How to filter out data from a print out message

I have this one problem, where I print out a message response from a website(JSON response), and the response I get is this.
Here is my model with fake data:
{"token": "MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl", "user_settings": {"locale": "en-US", "theme": "dark"}, "user_id": "101900638614857883"}
And, if I only want the value of "token" data which are this (MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl) and I want to store it into a txt file, is there any good way to do it?
Thank you, guys!
I tried print(r.text('token')) but it did not work, since it only works on printing the category of the data's (like : Category : {"token" : 'daefafa', "user-id" : 'er121231231', more})

In python, JSON is treated as a dictionary.
To filter it use dictionary comprehension
tokenData = {key: val for key,val in data_json.items() if key == 'token'}
Full Code Snippet :
from urllib.request import urlopen
import json
url = "enter-your-url"
response = urlopen(url)
data_json = json.loads(response.read())
print(type(data_json)) # <class 'dict'>
#use dict comprehension
jsonToken = {key: val for key,val in data_json.items() if key == 'result'}
strToken = json.dumps(jsonToken)
# Only string json can be written to files
with open('data.txt','w') as file:
file.write(strToken)
file.close()

You need to parse the JSON into a dictionary using json.loads(). Like this:
import json
# ...
# request-getting code
# ...
data = json.loads(r.text)
print(data['token'])

Reading JSon return values from an API

I'm getting below output from an API and I want to read all purchaseOrder data. Not really sure how to loop on this data. Also it comes with b' at the front.
b'[
{"purchaseOrder":
[
{
"id":"d01f0f6d-398f-4220-8a9a-44f47beedf04",
"installationNumber":null,
"peerId":"308866ba-90cb-47a7-8c73-589c0f355eb7",
"validFrom":"2019-06-07T12:51:15.000+0000",
"validTo":"2019-06-07T13:51:15.000+0000",
"originalQuantity":5,
"quantity":5,
"price":5,
"periodInitial":"2019-06-07T13:00:00.000+0000",
"periodFinal":"2019-06-07T14:00:00.000+0000"
}
],
"salesOrder":null,
"agreement":null,
"status":""
}
]'
Have tried things like loaded_json = json.load(r.content) and it didn't work.
This is code I use to get the response:
r = requests.post(url=api_endpoint, data=json.dumps(json_post), headers=headers)

To get the json of a response use data = response.json().
After that you can step through it like normal lists and dicts:
import json
data = r.json()
print(json.dumps(data , indent=2)) # If you want to see the data from the response
for dic in data :
if 'purchaseOrder' in dic:
for item in dic['purchaseOrder']:
# item here is the `dict` for each purchaseOrder (PO).
print(json.dumps(item, indent=2)) # This will print each item in PO.

Thanks all for support. The next code works for me:
data = r.json()
print(json.dumps(data, indent=2))
for dic in data:
if 'purchaseOrder' in dic:
for itemdata in dic['purchaseOrder']:
for key in itemdata:
if key == 'id':
print("Id:")
print(itemdata['id'])
print("Price:")
print(itemdata['price'])

Remove backslashes from Json String in Django Rest Framework

dct_data = json_tour_data.__dict__
tour_data = json.dumps(dct_data)
How to remove these backslashes from json? Here is my output:
"{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\",
\"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\":
\"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\",
\"strFileOpenAccountCodeAjxKey\": \"CARTZS\",
\"strFileOpenProfileCodeAjxKey\": \"CARTZS\",
\"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1,
\"strTransactionCurrencyJsKey\": \"Shillings\",
\"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\",
\"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}"

The answer is you have to just play with json.dumps() and json.loads().
The following is my code which worked:-
import json
json_data = {'key1': 'first'}
json_data = json.dumps(json_data)
return {
'statusCode': 200,
'schools': json.loads(json_data)
}
The output of above code is as follows:
Response:
{
"schools": {
"key1": "first"
},
"statusCode": 200
}

I would recommend checking your Response object where your parse your tour_data variable/dictionary in your views. I originally had the same issue as you but here's what I changed.
Original implementation: Response(json.dumps(a_dictionary), status=status.HTTP_200_OK)
to
New implementation: Response(a_dictionary, status=status.HTTP_200_OK, content_type='json')
The key things here are:
1. Getting rid of the json.dumps conversion method and just pass through a plain python dictionary e.g. see a_dictionary.
2. Setting content_type='json' on the Response object.

you can use replace("\'", '"') for that.
json = '''{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\",
\"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\":
\"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\",
\"strFileOpenAccountCodeAjxKey\": \"CARTZS\",
\"strFileOpenProfileCodeAjxKey\": \"CARTZS\",
\"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1,
\"strTransactionCurrencyJsKey\": \"Shillings\",
\"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\",
\"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}'''
newString = json.replace("\'", '"')
print(newString)
check from here
that is the output when pressed the run in my side.

To anyone else who might be facing this issue.
simply put:
the original question seems to be taking JSON data (I emphasize, it's already JSON) and rendering it into JSON again.
dct_data = json_tour_data.dict
tour_data = json.dumps(dct_data)

How to turn a list into a paginated JSON response for REST?

I'm new to Django REST Framework and I faced a problem.
I'm building a backend for a social app. The task is to return a paginated JSON response to client. In the docs I only found how to do that for model instances, but what I have is a list:
[368625, 507694, 687854, 765213, 778491, 1004752, 1024781, 1303354, 1311339, 1407238, 1506842, 1530012, 1797981, 2113318, 2179297, 2312363, 2361973, 2610241, 3005224, 3252169, 3291575, 3333882, 3486264, 3860625, 3964299, 3968863, 4299124, 4907284, 4941503, 5120504, 5210060, 5292840, 5460981, 5622576, 5746708, 5757967, 5968243, 6025451, 6040799, 6267952, 6282564, 6603517, 7271663, 7288106, 7486229, 7600623, 7981711, 8106982, 8460028, 10471602]
Is there some nice way to do it? Do I have to serialize it in some special way?
What client is waiting for is:
{"users": [{"id": "368625"}, {"id": "507694"}, ...]}
The question is: How to paginate such response?
Any input is highly appreciated!
Best regards,
Alexey.

TLDR:
import json
data=[368625, 507694, 687854, 765213, 778491, 1004752, 1024781, 1303354, 1311339, 1407238, 1506842, 1530012, 1797981, 2113318, 2179297, 2312363, 2361973, 2610241, 3005224, 3252169, 3291575, 3333882, 3486264, 3860625, 3964299, 3968863, 4299124, 4907284, 4941503, 5120504, 5210060, 5292840, 5460981, 5622576, 5746708, 5757967, 5968243, 6025451, 6040799, 6267952, 6282564, 6603517, 7271663, 7288106, 7486229, 7600623, 7981711, 8106982, 8460028, 10471602]
print(json.dumps({"users":[{"id":value} for value in data]}))
import json imports the json package, which is a JSON serialization/deserialization library
json.dumps(obj) takes obj, a python object, and serializes it to a JSON string
[{"id":value} for value in data] is just a list comprehension which creates a list of python dictionaries with "id" mapped to each value in the data array
EDIT: Pagination
I'm not sure if there's some standard on pagination, but a simple model would be:
"data": {
"prevPage": "id",
"nextPage": "id",
"data": [
...
]
}
Honestly, implementing that in python wouldn't be that hard:
data=[ ... ]
currentPage={"pageID":0,"data":[]}
prevPage={"pageID":-1}
pageSize=5
for value in data:
currentPage["data"].append({"id":value})
if len(currentPage)==pageSize:
currentPage["prevPage"]=prevPage["pageID"]
prevPage["nextPage"]=currentPage["pageID"]
# add currentPage to some database of pages
prevPage=currentPage
currentPage={"pageID":"generate new page id","data":[]}
Obviously, this isn't very polished, but shows the basic concept.
EDIT: Pagination without storing pages
You could of course recreate the page every time it is requested:
def getPage(pageNum)
#calculate pageStart and pageEnd based on your own requiremnets
pageStart = (pageNum // 5) * 5
pageEnd = (pageNum // 5)*5+5
return [{"id":data[idx] for idx in range(pageStart, pageEnd)}]

Python request.post Not Accepting Dictionary

here is the data that I want to post,
this is the printed version of the dictionary answers:
{'Emails': set([u'vernon.fadel#denesik.com', u'helyn67#cruickshankmckenzie.com', u'tryan#fishercremin.com', u'cecilia.kerluke#gmail.com', u'sonia.ruecker#schmidt.info', u'reichel.tamatha#gmail.com', u'johntyree69#greendale.edu', u'gail34#reichel.com', u'birdman#quitzon.net ', u'onolan#gmail.com', u'lindgren.merry#howe.com', u'wilderman.nyree#heidenreich.com', u'yschneider#jacobson.biz', u'nakia.larkin#gmail.com', u'vblanda#gmail.com', u'pkoch#yahoo.com', u'ygoyette#yahoo.com', u'wiegand.lynnette#beer.com', u'ubradtke#hotmail.com', u'troy67#yahoo.com', u'eileen.gusikowski#oreilly.info', u'glubowitz#yahoo.com', u'inolan#hotmail.com', u'lfranecki#hotmail.com', u'miller.hana#gmail.com', u'morissette.lalla#kuhicmuller.com', u'harber.gisele#gmail.com', u'odalys.dubuque#ebertrunte.net', u'jones.kyree#auer.net', u'adams77#yahoo.com', u'parisian.linda#hotmail.com', u'channing.cremin#toy.org', u'chad.armstrong#yahoo.com', u'isaac.nolan#hotmail.com', u'kozey.luciana#collins.biz', u'cstehr#keebler.com', u'jquitzon#gmail.com', u'hhirthe#yahoo.com', u'emilia16#block.com', u'arice#greenholtconsidine.com', u'tobin.schuppe#gmail.com', u'wyman.junious#gmail.com', u'mrath#ruecker.com ', u' johns.sonji#lakinwalker.info', u'thiel.cassie#hodkiewicz.info', u'fjaskolski#hotmail.com', u'mosciski.destany#hotmail.com', u'vrippin#gmail.com', u'zschuppe#hotmail.com', u'clotilda26#goodwinlind.info', u'lizette.walker#stanton.com', u'hettinger.kamari#hotmail.com', u'emelia54#yahoo.com', u'terry.ezekiel#hotmail.com', u'douglyass.dickinson#hotmail.com', u'zulauf.olympia#gmail.com', u'maxxx.waters#sauer.biz', u'abshire.charmaine#schusterlabadie.com', u'kuvalis.dewitt#wolfschaefer.com', u"elberta.o'connell#yahoo.com", u'henery76#yahoo.com', u'britny60#gmail.com', u'ed.hilpert#hansen.com', u'burnice.feest#yahoo.com', u' ybartoletti#littel.biz', u'sharde.murazik#hotmail.com', u'angelica.klocko#ankunding', u'brandi.klein#hotmail.com', u'dollie49#hotmail.com', u'mrath#ruecker.com', u'hayley27#koelpin.com', u'georgiann.mertz#moen.net', u'floretta39#volkmanrosenbaum.net', u'vicy.predovic#gmail.com', u'raven53#yahoo.com', u'fmann#yahoo.com', u'sdaugherty#gmail.com', u'gerlach.villa#koelpinfeeney.net', u'alysha.gibson#hotmail.com', u'hoy.kozey#gmail.com', u'cheyenne84#lehneroconner.com', u'uoberbrunner#hotmail.com', u'hhaag#swaniawskiterry.info', u'infant80#yahoo.com', u'manilla.west#hotmail.com', u'hconsidine#bernhardhills.info', u'dean94#hotmail.com', u'bahringer.karl#yahoo.com', u'crona.meredith#gmail.com', u'letta.larkin#quitzon.net', u'owindler#strosin.info', u'shanahan.anders#yahoo.com', u'mclaughlin.desi#gmail.com', u'erdman.icy#hotmail.com', u'salvatore21#hintz.com', u'jkoch#hotmail.com', u'red.beatty#yahoo.com', u'helma91#hotmail.com', u'arland.koepp#hotmail.com', u'ymoore#cristhalvorson.org', u'woodie.crooks#kozey.com', None, u'jchamplin#hotmail.com ', u'walter.elwyn#yahoo.com']), '# of Emails per Domain': {u'#hotmail.com': 22, u'#yahoo.com': 18, u'#gmail.com': 18}, 'April Users': 70}
when I run:
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data = answers)
I get this error:
{"message": "Could not parse request body into json: Unrecognized token \'Emails\': was expecting (\'true\', \'false\' or \'null\')\n at [Source: [B#311a7237; line: 1, column: 8]"}
Not sure what this means at all, I've tried to parse the data into json, why doesn't it like the 'Email' string?
If I add the json.dump code I get:
TypeError
TypeError: set([u'vernon.fadel#denesik.com', u'helyn67#cruickshankmckenzie.com', u'tryan#fishercremin.com', u'cecilia.kerluke#gmail.com', u'sonia.ruecker#schmidt.info', u'reichel.tamatha#gmail.com', u'johntyree69#greendale.edu', u'gail34#reichel.com', u'birdman#quitzon.net ', u'onolan#gmail.com', u'lindgren.merry#howe.com', u'wilderman.nyree#heidenreich.com', u'yschneider#jacobson.biz', u'nakia.larkin#gmail.com', u'vblanda#gmail.com', u'pkoch#yahoo.com', u'ygoyette#yahoo.com', u'wiegand.lynnette#beer.com', u'ubradtke#hotmail.com', u'troy67#yahoo.com', u'eileen.gusikowski#oreilly.info', u'glubowitz#yahoo.com', u'inolan#hotmail.com', u'lfranecki#hotmail.com', u'miller.hana#gmail.com', u'morissette.lalla#kuhicmuller.com', u'harber.gisele#gmail.com', u'odalys.dubuque#ebertrunte.net', u'jones.kyree#auer.net', u'adams77#yahoo.com', u'parisian.linda#hotmail.com', u'channing.cremin#toy.org', u'chad.armstrong#yahoo.com', u'isaac.nolan#hotmail.com', u'kozey.luciana#collins.biz', u'cstehr#keebler.com', u'jquitzon#gmail.com', u'hhirthe#yahoo.com', u'emilia16#block.com', u'arice#greenholtconsidine.com', u'tobin.schuppe#gmail.com', u'wyman.junious#gmail.com', u'mrath#ruecker.com ', u' johns.sonji#lakinwalker.info', u'thiel.cassie#hodkiewicz.info', u'fjaskolski#hotmail.com', u'mosciski.destany#hotmail.com', u'vrippin#gmail.com', u'zschuppe#hotmail.com', u'clotilda26#goodwinlind.info', u'lizette.walker#stanton.com', u'hettinger.kamari#hotmail.com', u'emelia54#yahoo.com', u'terry.ezekiel#hotmail.com', u'douglyass.dickinson#hotmail.com', u'zulauf.olympia#gmail.com', u'maxxx.waters#sauer.biz', u'abshire.charmaine#schusterlabadie.com', u'kuvalis.dewitt#wolfschaefer.com', u"elberta.o'connell#yahoo.com", u'henery76#yahoo.com', u'britny60#gmail.com', u'ed.hilpert#hansen.com', u'burnice.feest#yahoo.com', u' ybartoletti#littel.biz', u'sharde.murazik#hotmail.com', u'angelica.klocko#ankunding', u'brandi.klein#hotmail.com', u'dollie49#hotmail.com', u'mrath#ruecker.com', u'hayley27#koelpin.com', u'georgiann.mertz#moen.net', u'floretta39#volkmanrosenbaum.net', u'vicy.predovic#gmail.com', u'raven53#yahoo.com', u'fmann#yahoo.com', u'sdaugherty#gmail.com', u'gerlach.villa#koelpinfeeney.net', u'alysha.gibson#hotmail.com', u'hoy.kozey#gmail.com', u'cheyenne84#lehneroconner.com', u'uoberbrunner#hotmail.com', u'hhaag#swaniawskiterry.info', u'infant80#yahoo.com', u'manilla.west#hotmail.com', u'hconsidine#bernhardhills.info', u'dean94#hotmail.com', u'bahringer.karl#yahoo.com', u'crona.meredith#gmail.com', u'letta.larkin#quitzon.net', u'owindler#strosin.info', u'shanahan.anders#yahoo.com', u'mclaughlin.desi#gmail.com', u'erdman.icy#hotmail.com', u'salvatore21#hintz.com', u'jkoch#hotmail.com', u'red.beatty#yahoo.com', u'helma91#hotmail.com', u'arland.koepp#hotmail.com', u'ymoore#cristhalvorson.org', u'woodie.crooks#kozey.com', None, u'jchamplin#hotmail.com ', u'walter.elwyn#yahoo.com']) is not JSON serializable

This works:
import requests
import json
answers = {} #...
url = "..."
answers['Emails'] = list(answers['Emails'])
r = requests.post(url, data=json.dumps(answers))
The reason the other way doesn't work is that by default requests sends data with the Content-Type application/x-www-form-urlencoded instead of JSON.
To change that, you can also use json instead of the data parameter:
r = requests.post(url, json=answers)
To debug code using the requests library, you can look at a prepared request before sending it.

TypeError: set([ ... ]) is not JSON serializable
Sets are not JSON serializable. You can either convert all of the sets to lists in the dictionary:
answers = {key: list(value) if isinstance(value, set) else value
for key, value in answers.items()}
(Note that this would not work for nested dictionaries as is).
Or, use a custom JSON encoder that would handle the set serialization.
Dumping and loading it back with json usually helps in debugging:
import json
answers = json.loads(json.dumps(answers))
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data=answers)

Categories