Python request.post Not Accepting Dictionary - python
here is the data that I want to post,
this is the printed version of the dictionary answers:
{'Emails': set([u'vernon.fadel#denesik.com', u'helyn67#cruickshankmckenzie.com', u'tryan#fishercremin.com', u'cecilia.kerluke#gmail.com', u'sonia.ruecker#schmidt.info', u'reichel.tamatha#gmail.com', u'johntyree69#greendale.edu', u'gail34#reichel.com', u'birdman#quitzon.net ', u'onolan#gmail.com', u'lindgren.merry#howe.com', u'wilderman.nyree#heidenreich.com', u'yschneider#jacobson.biz', u'nakia.larkin#gmail.com', u'vblanda#gmail.com', u'pkoch#yahoo.com', u'ygoyette#yahoo.com', u'wiegand.lynnette#beer.com', u'ubradtke#hotmail.com', u'troy67#yahoo.com', u'eileen.gusikowski#oreilly.info', u'glubowitz#yahoo.com', u'inolan#hotmail.com', u'lfranecki#hotmail.com', u'miller.hana#gmail.com', u'morissette.lalla#kuhicmuller.com', u'harber.gisele#gmail.com', u'odalys.dubuque#ebertrunte.net', u'jones.kyree#auer.net', u'adams77#yahoo.com', u'parisian.linda#hotmail.com', u'channing.cremin#toy.org', u'chad.armstrong#yahoo.com', u'isaac.nolan#hotmail.com', u'kozey.luciana#collins.biz', u'cstehr#keebler.com', u'jquitzon#gmail.com', u'hhirthe#yahoo.com', u'emilia16#block.com', u'arice#greenholtconsidine.com', u'tobin.schuppe#gmail.com', u'wyman.junious#gmail.com', u'mrath#ruecker.com ', u' johns.sonji#lakinwalker.info', u'thiel.cassie#hodkiewicz.info', u'fjaskolski#hotmail.com', u'mosciski.destany#hotmail.com', u'vrippin#gmail.com', u'zschuppe#hotmail.com', u'clotilda26#goodwinlind.info', u'lizette.walker#stanton.com', u'hettinger.kamari#hotmail.com', u'emelia54#yahoo.com', u'terry.ezekiel#hotmail.com', u'douglyass.dickinson#hotmail.com', u'zulauf.olympia#gmail.com', u'maxxx.waters#sauer.biz', u'abshire.charmaine#schusterlabadie.com', u'kuvalis.dewitt#wolfschaefer.com', u"elberta.o'connell#yahoo.com", u'henery76#yahoo.com', u'britny60#gmail.com', u'ed.hilpert#hansen.com', u'burnice.feest#yahoo.com', u' ybartoletti#littel.biz', u'sharde.murazik#hotmail.com', u'angelica.klocko#ankunding', u'brandi.klein#hotmail.com', u'dollie49#hotmail.com', u'mrath#ruecker.com', u'hayley27#koelpin.com', u'georgiann.mertz#moen.net', u'floretta39#volkmanrosenbaum.net', u'vicy.predovic#gmail.com', u'raven53#yahoo.com', u'fmann#yahoo.com', u'sdaugherty#gmail.com', u'gerlach.villa#koelpinfeeney.net', u'alysha.gibson#hotmail.com', u'hoy.kozey#gmail.com', u'cheyenne84#lehneroconner.com', u'uoberbrunner#hotmail.com', u'hhaag#swaniawskiterry.info', u'infant80#yahoo.com', u'manilla.west#hotmail.com', u'hconsidine#bernhardhills.info', u'dean94#hotmail.com', u'bahringer.karl#yahoo.com', u'crona.meredith#gmail.com', u'letta.larkin#quitzon.net', u'owindler#strosin.info', u'shanahan.anders#yahoo.com', u'mclaughlin.desi#gmail.com', u'erdman.icy#hotmail.com', u'salvatore21#hintz.com', u'jkoch#hotmail.com', u'red.beatty#yahoo.com', u'helma91#hotmail.com', u'arland.koepp#hotmail.com', u'ymoore#cristhalvorson.org', u'woodie.crooks#kozey.com', None, u'jchamplin#hotmail.com ', u'walter.elwyn#yahoo.com']), '# of Emails per Domain': {u'#hotmail.com': 22, u'#yahoo.com': 18, u'#gmail.com': 18}, 'April Users': 70}
when I run:
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data = answers)
I get this error:
{"message": "Could not parse request body into json: Unrecognized token \'Emails\': was expecting (\'true\', \'false\' or \'null\')\n at [Source: [B#311a7237; line: 1, column: 8]"}
Not sure what this means at all, I've tried to parse the data into json, why doesn't it like the 'Email' string?
If I add the json.dump code I get:
TypeError
TypeError: set([u'vernon.fadel#denesik.com', u'helyn67#cruickshankmckenzie.com', u'tryan#fishercremin.com', u'cecilia.kerluke#gmail.com', u'sonia.ruecker#schmidt.info', u'reichel.tamatha#gmail.com', u'johntyree69#greendale.edu', u'gail34#reichel.com', u'birdman#quitzon.net ', u'onolan#gmail.com', u'lindgren.merry#howe.com', u'wilderman.nyree#heidenreich.com', u'yschneider#jacobson.biz', u'nakia.larkin#gmail.com', u'vblanda#gmail.com', u'pkoch#yahoo.com', u'ygoyette#yahoo.com', u'wiegand.lynnette#beer.com', u'ubradtke#hotmail.com', u'troy67#yahoo.com', u'eileen.gusikowski#oreilly.info', u'glubowitz#yahoo.com', u'inolan#hotmail.com', u'lfranecki#hotmail.com', u'miller.hana#gmail.com', u'morissette.lalla#kuhicmuller.com', u'harber.gisele#gmail.com', u'odalys.dubuque#ebertrunte.net', u'jones.kyree#auer.net', u'adams77#yahoo.com', u'parisian.linda#hotmail.com', u'channing.cremin#toy.org', u'chad.armstrong#yahoo.com', u'isaac.nolan#hotmail.com', u'kozey.luciana#collins.biz', u'cstehr#keebler.com', u'jquitzon#gmail.com', u'hhirthe#yahoo.com', u'emilia16#block.com', u'arice#greenholtconsidine.com', u'tobin.schuppe#gmail.com', u'wyman.junious#gmail.com', u'mrath#ruecker.com ', u' johns.sonji#lakinwalker.info', u'thiel.cassie#hodkiewicz.info', u'fjaskolski#hotmail.com', u'mosciski.destany#hotmail.com', u'vrippin#gmail.com', u'zschuppe#hotmail.com', u'clotilda26#goodwinlind.info', u'lizette.walker#stanton.com', u'hettinger.kamari#hotmail.com', u'emelia54#yahoo.com', u'terry.ezekiel#hotmail.com', u'douglyass.dickinson#hotmail.com', u'zulauf.olympia#gmail.com', u'maxxx.waters#sauer.biz', u'abshire.charmaine#schusterlabadie.com', u'kuvalis.dewitt#wolfschaefer.com', u"elberta.o'connell#yahoo.com", u'henery76#yahoo.com', u'britny60#gmail.com', u'ed.hilpert#hansen.com', u'burnice.feest#yahoo.com', u' ybartoletti#littel.biz', u'sharde.murazik#hotmail.com', u'angelica.klocko#ankunding', u'brandi.klein#hotmail.com', u'dollie49#hotmail.com', u'mrath#ruecker.com', u'hayley27#koelpin.com', u'georgiann.mertz#moen.net', u'floretta39#volkmanrosenbaum.net', u'vicy.predovic#gmail.com', u'raven53#yahoo.com', u'fmann#yahoo.com', u'sdaugherty#gmail.com', u'gerlach.villa#koelpinfeeney.net', u'alysha.gibson#hotmail.com', u'hoy.kozey#gmail.com', u'cheyenne84#lehneroconner.com', u'uoberbrunner#hotmail.com', u'hhaag#swaniawskiterry.info', u'infant80#yahoo.com', u'manilla.west#hotmail.com', u'hconsidine#bernhardhills.info', u'dean94#hotmail.com', u'bahringer.karl#yahoo.com', u'crona.meredith#gmail.com', u'letta.larkin#quitzon.net', u'owindler#strosin.info', u'shanahan.anders#yahoo.com', u'mclaughlin.desi#gmail.com', u'erdman.icy#hotmail.com', u'salvatore21#hintz.com', u'jkoch#hotmail.com', u'red.beatty#yahoo.com', u'helma91#hotmail.com', u'arland.koepp#hotmail.com', u'ymoore#cristhalvorson.org', u'woodie.crooks#kozey.com', None, u'jchamplin#hotmail.com ', u'walter.elwyn#yahoo.com']) is not JSON serializable
This works:
import requests
import json
answers = {} #...
url = "..."
answers['Emails'] = list(answers['Emails'])
r = requests.post(url, data=json.dumps(answers))
The reason the other way doesn't work is that by default requests sends data with the Content-Type application/x-www-form-urlencoded instead of JSON.
To change that, you can also use json instead of the data parameter:
r = requests.post(url, json=answers)
To debug code using the requests library, you can look at a prepared request before sending it.
TypeError: set([ ... ]) is not JSON serializable
Sets are not JSON serializable. You can either convert all of the sets to lists in the dictionary:
answers = {key: list(value) if isinstance(value, set) else value
for key, value in answers.items()}
(Note that this would not work for nested dictionaries as is).
Or, use a custom JSON encoder that would handle the set serialization.
Dumping and loading it back with json usually helps in debugging:
import json
answers = json.loads(json.dumps(answers))
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data=answers)
Related
How to filter out data from a print out message
I have this one problem, where I print out a message response from a website(JSON response), and the response I get is this. Here is my model with fake data: {"token": "MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl", "user_settings": {"locale": "en-US", "theme": "dark"}, "user_id": "101900638614857883"} And, if I only want the value of "token" data which are this (MTAxOTAwNjM4NjEyMzg0OTkwMQ.8hkyLV.n0ir2UA4qFE5pXen9YnPtFzgn4xP8tHmVmmkrl) and I want to store it into a txt file, is there any good way to do it? Thank you, guys! I tried print(r.text('token')) but it did not work, since it only works on printing the category of the data's (like : Category : {"token" : 'daefafa', "user-id" : 'er121231231', more})
In python, JSON is treated as a dictionary. To filter it use dictionary comprehension tokenData = {key: val for key,val in data_json.items() if key == 'token'} Full Code Snippet : from urllib.request import urlopen import json url = "enter-your-url" response = urlopen(url) data_json = json.loads(response.read()) print(type(data_json)) # <class 'dict'> #use dict comprehension jsonToken = {key: val for key,val in data_json.items() if key == 'result'} strToken = json.dumps(jsonToken) # Only string json can be written to files with open('data.txt','w') as file: file.write(strToken) file.close()
You need to parse the JSON into a dictionary using json.loads(). Like this: import json # ... # request-getting code # ... data = json.loads(r.text) print(data['token'])
Python format json output
I have output from json as below from curl: {'TableViewConfig': {'displayLength': 50, 'Resources': ...}} it needs to be passed in another curl get command with data but in below format: {\"TableViewConfig\":{\"displayLength\":50,\"Resources\":...}} Can you please help in resolving the str replace in python dict? Tried below, but it didnt work: request = 'url' response = requests.get(request) data = json.loads(response) print(data) orig = list(map(lambda item: dict((k.replace(''', ''\'), v.replace(''', ''\')) for k, v in item.items()), data))
How to convert a complex Python Dict into JSON serializable payload for an API?
So, I have the following DICT in python: {'documentType': {'documentTypeId': 'XXXXXXXXXXXXXX', 'documentTypeName': 'XXXXXXXXXXXXXXXXXXXX', 'documentTypeVersion': 1}, 'tags': [{'boundingBoxCoOrdinates': {'x1': 440.66135, 'y1': 894.0904, 'x2': 468.74966, 'y2': 917.39105}, 'digitalPdfTagDetails': {'startPage': 0, 'endPage': 0, 'startCharIndex': 0, 'endCharIndex': 0}, 'editedDate': 'SOMEDATET12:47:36Z', 'contains': True, 'tagId': 0, 'tagName': 'SOMETAG', 'type': 'SOMETYPE', 'color': '#SOMECOLOR', 'extractedData': '', 'editedData': '', 'editedBy': 'SOMEPERSON'}], 'uploadedFileId': 'SOMEID', 'imageHeight': 55, 'imageWidth': 55, 'Engine': 'SOMEPROCESSINGENGINE', 'fileName': 'SOMEFILENAME.jpg'} When I pass it for a POST api using Requests library in the JSON argument, I am greeted with TypeError: Object of type 'float32' is not JSON serializable I suspect, this is because of the values inside the 'boundingBoxCoOrdinates' which are numpy.float32 type. If I convert them to numpy.float64 using astype(numpy.float64), then it seems to be working fine. How do I convert all such values in this DICT so as to JSON serializable error does not persist? EDIT: There can be multiple such dictionaries inside 'tags' LIST
You need to use a custom encoder for all data types that don't fit in the standard range of data types. Custom encoders are created by subclassing the json.JSONEncoder class: import json import numpy as np class NumpyEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, np.float32): return "...an appropriate representation of the item that the recipient understands..." # add more cases for NumPy data types as you need them # all other cases get default encoding return json.JSONEncoder.default(self, obj) json_str = json.dumps(your_data_structure, cls=NumpyEncoder)
As stated into the requests documentation here, you can use json parameter to let requests manage the encoding of the dict as this: url = 'https://api.github.com/some/endpoint' payload = {'some': 'data'} r = requests.post(url, json=payload) But, since you're getting that error, you can try to encode the dict yourself and send it via the data parameter like this: import json url = 'https://api.github.com/some/endpoint' payload = {'some': 'data'} r = requests.post(url, data=json.dumps(payload))
You can use the default argument in the json call to handle conversion of numpy arrays: import numpy as np import json d = {"floats": np.ones(10) / np.arange(1, 11)} def handler(x): if isinstance(x, np.ndarray): return [y.item() for y in x] else: return x json_str = json.dumps(d, default=handler)
Remove backslashes from Json String in Django Rest Framework
dct_data = json_tour_data.__dict__ tour_data = json.dumps(dct_data) How to remove these backslashes from json? Here is my output: "{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\", \"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\": \"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\", \"strFileOpenAccountCodeAjxKey\": \"CARTZS\", \"strFileOpenProfileCodeAjxKey\": \"CARTZS\", \"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1, \"strTransactionCurrencyJsKey\": \"Shillings\", \"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\", \"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}"
The answer is you have to just play with json.dumps() and json.loads(). The following is my code which worked:- import json json_data = {'key1': 'first'} json_data = json.dumps(json_data) return { 'statusCode': 200, 'schools': json.loads(json_data) } The output of above code is as follows: Response: { "schools": { "key1": "first" }, "statusCode": 200 }
I would recommend checking your Response object where your parse your tour_data variable/dictionary in your views. I originally had the same issue as you but here's what I changed. Original implementation: Response(json.dumps(a_dictionary), status=status.HTTP_200_OK) to New implementation: Response(a_dictionary, status=status.HTTP_200_OK, content_type='json') The key things here are: 1. Getting rid of the json.dumps conversion method and just pass through a plain python dictionary e.g. see a_dictionary. 2. Setting content_type='json' on the Response object.
you can use replace("\'", '"') for that. json = '''{\"strFileOpenDateAjxKey\": \"2018-01-16 12:40:22.526417\", \"strFilePassengerAjxKey\": \"Zahra Walji\", \"strFileOpenMobileAjxKey\": \"46464664\", \"strFileOpenDepartmentAjxKey\": \"Finance department\", \"strFileOpenAccountCodeAjxKey\": \"CARTZS\", \"strFileOpenProfileCodeAjxKey\": \"CARTZS\", \"strFileopenOriginalCountryIdAjxKey\": 61, \"blnBoundAjxKey\": 1, \"strTransactionCurrencyJsKey\": \"Shillings\", \"intCurrencyPrecisionJsKey\": 3, \"strPackageTypeJsKey\": \"PKG\", \"strUserNameAjxKey\": \"admin\", \"strPasswordAjxKey\": \"1234\"}''' newString = json.replace("\'", '"') print(newString) check from here that is the output when pressed the run in my side.
To anyone else who might be facing this issue. simply put: the original question seems to be taking JSON data (I emphasize, it's already JSON) and rendering it into JSON again. dct_data = json_tour_data.dict tour_data = json.dumps(dct_data)
Parsing a list of dictionaries passed as a POST parameter
I have a list of python dictionaries that look like this: sandwiches = [ {'bread':'wheat', 'topping':'tomatoes', 'meat':'bacon'}, {'bread':'white', 'topping':'peanut butter', 'meat':'bacon'}, {'bread':'sourdough', 'topping':'cheese', 'meat':'bacon'} ] I want to pass this as a POST parameter to another Django app. What does the client app need to do to iterate through the list? I want to do something like: for sandwich in request.POST['sandwiches']: print "%s on %s with %s is yummy!" % (sandwich['meat'], sandwich['bread'], sandwich['topping']) But I don't seem to have a list of dicts when my data arrives at my client app.
You don't say how you're POSTing to the app. My advice would be to serialize the dictionaries as JSON, then simply POST that. import json, urllib, urllib2 data = json.dumps(sandwiches) urllib2.urlopen(myurl, urllib.urlencode({'data': data})) ... and on the receiver: data = request.POST['data'] sandwiches = json.loads(data)
I would use the JSON libraries to serialize your array of dicts into a single string. Send this string as a POST param, and parse the string back into the python datatypes in the Django app using the same library. http://docs.python.org/library/json.html
firstly make string of the list import ast, requests # request send code dev_list = [{"1": 2}, {"2": 4}] str_list = str(dev_list) # '[{"1": 2}, {"2": 4}]' data = {"a" : "mynema", 'str_list': str_list } requests.post(url, data ) #request receive side data = request.data dev_list = data.get('str_list') dev_list = ast.literal_eval(dev_list) #dev_list --> [{"1": 2}, {"2": 4}] Now you receive list of dictionary.