This question already has answers here:
Get the data received in a Flask request
(23 answers)
How to get POSTed JSON in Flask?
(13 answers)
Closed 4 years ago.
I'm finding it difficult to turn a json like string into a json/dict object, I've got the data like this
{"key1":"value1","key2":[{"key2.1":"value2.1"},{"key2.2":"value2.2"}]}
When running type on the variable it tells me it is of class str i.e <class 'str'>. I need to get the string into a json or dict like format so I can extract all of the values named value...
I've tried json.loads, json.load, json.dumps, ast.literal_eval but nothing seems to be working. I've tried adding [' and '] either side of the string but still no luck. Any ideas? Thanks
EDIT:
I'm using a nodeJS backend sending a request to a flask server, the nodeJS axios request is this
get_values: (key_one, key_two) =>
axios.post('http://localhost:5000/example', {
key_one: key_one,
key_two: key_two
}).then(res => res.data),
on the flask side I'm doing this
#app.route('/example', methods=["POST","OPTIONS"])
def example():
convert_data = request.get_data()
string_data = convert_data.decode('utf8').replace("'", '"')
new_string = "'''" + string_data + "'''"
print(json.loads(new_string))
Then I get the an error
I modified your function a bit:
#app.route('/example', methods=["POST","OPTIONS"])
def example():
convert_data = request.get_data()
string_data = convert_data.decode('utf8')
print(json.loads(string_data))
It seems string_data is already a perfectly formatted json string that you can pass into loads. I removed the replacement of ' with " which seems unnecessary, and definitely the addition of ''', which will make this string become Python multiline string literal syntax but definitely break json compatibility, which is what your error tells you too:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This should be a lot closer to what you wanted. Otherwise let me know what exactly is inside string_data when you run this.
Not sure if I understand the question correctly. But, let's say you have a string as follows:
var a = {
"key1": "value1",
"key2": [{
"key2.1": "value2.1"
}, {
"key2.2": "value2.2"
}]
}
Then you can do something like this:
try {
const dictionary = JSON.parse(a)
console.log(dictionary.key1) // This will print value1
Object.keys(dictionary).map((key) => {
console.log(dictionary[key]);
});
} catch(e){
// do something with error here
}
Related
I have a JSON text grabbed from an API of a website:
{"result":"true","product":{"made":{"Taiwan":"Taipei","HongKong":"KongStore","Area":"Asia"}}}
I want to capture "Taiwan" and "Taipei" but always fail.
Here is my code:
import json
weather = urllib2.urlopen('url')
wjson = weather.read()
wjdata = json.loads(wjson)
print wjdata['product']['made'][0]['Taiwan']
I always get the following error:
Keyword 0 error
Whats the correct way to parse that json?
You are indexing an array where there are none.
The JSON is the following:
{
"result":"true",
"product": {
"made": {
"Taiwan":"Taipei",
"HongKong":"KongStore",
"Area":"Asia"
}
}
}
And the above contains no arrays.
You are assuming the JSON structure to be something like this:
{
"result":"true",
"product": {
"made": [
{"Taiwan":"Taipei"},
{"HongKong":"KongStore"},
{"Area":"Asia"}
]
}
}
From a brief look at the doc pages for the json package, I found this conversion table: Conversion table using json.loads
It tells us that a JSON object translates to a dict. And a dict has a method called keys, which returns a list of the keys.
I suggest you try something like this:
#... omitted code
objectKeys = wjdata['product']['made'].keys()
# You should now have a list of the keys stored in objectKeys.
for key in objectKeys:
print key
if key == 'Taiwan':
print 'Eureka'
I haven't tested the above code, but I think you get the gist here :)
wjdata['product']['made']['Taiwan'] works
I am passing a dictionary to json.dumps and it is still throwing an error:
TypeError: Undefined is not json serializable
I even check the type before calling the function to make sure it is a dictionary type.I'm using the Flask microframework and trying to return a Response object containing really simple json to an ajax request:
$(document).ready(function() {
$.getJSON($SCRIPT_ROOT + '/getDictionary', function(data) {
console.log(data);
});
});
#app.route('/getDictionary')
def getDictionary():
empty_dic = {'empty' : 'dict'}
if type(empty_dic) is dict:
return Response(json.dumps(empty_dic), mimetype = "application/json")
Here's an example that works:
import json
emptydic = {'simple' : 'dict'}
if isinstance(emptydic, dict): # Note capitalization
print(json.dumps(emptydic)) # {"simple": "dict"}
The type checking condition has changed slightly, and I stripped the Response/mimetype stuff, because it seems orthogonal to your issue.
Your capitalization is off, note you define emptydic but try to serialize emptyDic. Try this:
empty_dict = {'simple' : 'dict'}
if type(empty_dict) is dict:
return Response(json.dumps(empty_dict).encoded('utf8'), mimetype = "application/json")
Note it's also against PEP8 to use camel case for variables. Snake case is recommended
Your dictionary has name "emptydic"
In the condition, you use "emptyDic"
Try:
emptyDic = {"simple": "dict"}
if type(emptyDic) is dict:
return Response(json.dumps(emptyDic).encoded('utf8'), mimetype = "application/json")
Are you trying this by any chance
json.dumps(emptydic, encoding='utf-8')
I am trying this on Python 2.7 and the string object will not have an attribute called encoded.
First, reference links to other questions I read through. They might be clues to what I am experiencing, although I am not understanding enough yet to see the solution to my problem.
How can I use Python to transform MongoDB's bsondump into JSON?
Unable to deserialize PyMongo ObjectId from JSON
I've got a Flask Restful API I'm working on. An excerpt of my code is as follows:
class DeviceAPI(Resource):
def get(self, deviceID):
# do the query
deviceCollection = db['device']
device = deviceCollection.find_one({'deviceID': deviceID})
print device #1
print ''
print json_util.dumps(device) #2
print ''
s = json_util.dumps(device)
print s #3
print ''
results = {}
results['device'] = s
print results #4
# respond
return results #5
At Print #1, I get the following, and I understand and expect this.
{u'deviceID': u'ABC123', u'_id': ObjectId('....')}
At Print #2 and #3 are identical outputs as expected, and again I understand and expect this (I think).
{"deviceID": "ABC123", "_id": {"$oid": "...."}}
Print #4 has an added key in the dictionary. However, it looks like the value of the key:value is a string where it should be the dictionary as in #2 and #3.
{'device': '{"deviceID": "ABC123", "_id": {"$oid": "...."}}'}
The returned result, #5, according to CURL is along the lines of the following. There are the added / in the there. I suspect because of #4 value looking like a string and that continues in #5 as well.
{"device": "{\"deviceID\": \"ABC123\", \"_id\": {\"$oid\": \"....\"}}"}
I'm trying to get a pure JSON output, not a string representation of the device document. #2 and #3 looked like JSON, but in #4 became a string. Why? And how to do this correctly?
I believe it's because json_utils.dumps is converting your device variable into a string when you should be just returning a complete json object. You essentially end up returning something that resembles this:
return {"device": "a string that resembles json"}
Instead, modify your code to look like this:
class DeviceAPI(Resource):
def get(self, deviceID):
# do the query
deviceCollection = db['device']
device = deviceCollection.find_one({'deviceID': deviceID})
results = {'device': device}
return results
Now, we're returning json that looks more like this:
return {"device": {"deviceID": "ABC123", "_id": {"$oid": "...."}}}
However, it looks like the recommended way to return json in flask is to actually use the flask.jsonify method so that Flask will return a proper response object:
from flask import jsonify
class DeviceAPI(Resource):
def get(self, deviceID):
# do the query
deviceCollection = db['device']
device = deviceCollection.find_one({'deviceID': deviceID})
return jsonify(device=device)
Michael0x2a helped to clear some fog in my mind, and after more experimenting and thinking this through, the following is working for me.
results['device'] = json.loads(json_util.dumps(device))
In my experimenting I was using json_util.dumps and json_util.loads. But I didn't recognize that while json_util.dumps was converting the BSON like item to JSON string, that json_util.loads was converting it directly back. Therefore, another function was needed to take the string like JSON output of json_util.dumps and make it into a dictionary JSON like object. Thus a combo was needed. This is using "import json" and "from bson import json_util" (of Pymongo).
Thank you for your reply
I had the same problem as you, and it was fixed just because you post back the solution, thanks million! Follow below my code
from flask_restplus import Namespace, Resource, fields
from .. import mongo
import json
from bson import json_util
api = Namespace('inventory', description='Store management related operations')
#api.route('/items')
class Inventory(Resource):
def get(self):
inventory_collection = mongo.db.inventory
resp = inventory_collection.find({})
a = json.loads(json_util.dumps(resp))
return {'result': a }
I've done some coding in RoR, and in Rails, when I return a JSON object via an API call, it returns as
{ "id" : "1", "name" : "Dan" }.
However in Python (with Flask and Flask-SQLAlchemy), when I return a JSON object via json.dumps or jsonpickle.encode it is returned as
"{ \"id\" : \"1\", \"name\": \"Dan\" }" which seems very unwieldily as it can't easily be parsed on the other end (by an iOS app in this case - Obj-C).
What am I missing here, and what should I do to return it as a JSON literal, rather than a JSON string?
This is what my code looks like:
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
json_object = jsonpickle.encode(friends.first().as_dict(), unpicklable=False, keys=True)
print(json_object) # this prints here, i.e. { "id" : "1", "name" : "Dan" }
return json_object # this returns "{ \"id\" : \"1\", \"name\": \"Dan\" }" to the browser
What is missing in your understanding here is that when you use the JSON modules in Python, you're not working with a JSON object. JSON is by definition just a string that matches a certain standard.
Lets say you have the string:
friends = '{"name": "Fred", "id": 1}'
If you want to work with this data in python, you will want to load it into a python object:
import json
friends_obj = json.loads(friends)
At this point friends_obj is a python dictionary.
If you want to convert it (or any other python dictionary or list) then this is where json.dumps comes in handy:
friends_str = json.dumps(friends_obj)
print friends_str
'{"name": "Fred", "id": 1}'
However if we attempt to "dump" the original friends string you'll see you get a different result:
dumped_str = json.dumps(friends)
print dumped_str
'"{\\"name\\": \\"Fred\\", \\"id\\": 1}"'
This is because you're basically attempting to encode an ordinary string as JSON and it is escaping the characters. I hope this helps make sense of things!
Cheers
Looks like you are using Django here, in which case do something like
from django.utils import simplejson as json
...
return HttpResponse(json.dumps(friends.first().as_dict()))
This is almost always a sign that you're double-encoding your data somewhere. For example:
>>> obj = { "id" : "1", "name" : "Dan" }
>>> j = json.dumps(obj)
>>> jj = json.dumps(j)
>>> print(obj)
{'id': '1', 'name': 'Dan'}
>>> print(j)
{"id": "1", "name": "Dan"}
>>> print(jj)
"{\"id\": \"1\", \"name\": \"Dan\"}"
Here, jj is a perfectly valid JSON string representation—but it's not a representation of obj, it's a representation of the string j, which is useless.
Normally you don't do this directly; instead, either you started with a JSON string rather than an object in the first place (e.g., you got it from a client request or from a text file), or you called some function in a library like requests or jsonpickle that implicitly calls json.dumps with an already-encoded string. But either way, it's the same problem, with the same solution: Just don't double-encode.
You should be using flask.jsonify, which will not only encode correctly, but also set the content-type headers accordingly.
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
return jsonify(friends.first().as_dict())
I have a python as CGI and the POST from jquery will transform json object to array, so when I see the POST from jquery, I actually see:
login_user[username]=dfdsfdsf&login_user[password]=dsfsdf
(the [ and ] already escaped)
My question is how I can convert this string back to JSON in python? Or, how can I convert this string to python array/dict structure so that I can process it easier?
[edit]
My jquery is posting:
{'login_user': {'username':username, 'password':password}}
If what you want to accomplish is to send structured data from the browser and then unpack it in your Python backend and keep the same structure, I suggest the following:
Create JavaScript objects in the browser to hold your data:
var d = {}
d['login_user'] = { 'username': 'foo', 'password': 'bar' }
Serialize to JSON, with https://github.com/douglascrockford/JSON-js
POST to your backend doing something like this:
$.post(url, {'data': encoded_json_data}, ...)
In your Python code, parse the JSON, POST in my example is where you get your POST data in your CGI script:
data = json.loads(POST['data'])
data['login_user']
import re
thestring = "login_user[username]=dfdsfdsf&login_user[password]=dafef"
pattern = re.compile(r'^login_user\[username\]=(.*)&login_user\[password\]=(.*)')
match = pattern.search(thestring)
print match.groups()
Output:
>>> ('dfdsfdsf', 'dafef')
Thus,
lp = match.groups()
print "{'login_user': {'username':"+lp[0]+", 'password':"+lp[1]+"}}"
shall bear: >>> {'login_user': {'username':dfdsfdsf, 'password':dafef}}
>>> import json
>>> data = {'login_user':{'username':'dfdsfdsf', 'password':'dsfsdf'}}
>>> json.dumps(data)
'{"login_user": {"username": "dfdsfdsf", "password": "dsfsdf"}}'
I suspect that data would already be contained in a GET var if that's coming from the URL...