I have a json which looks like the below. When i validate json, i get an error "Invalid Json" since i have just a single quote and also, there is u which i can't tell why it is there.
{u'domain': u'127.0.0.1', u'user_id': u'example.com', u'sender': u'shop_1'}
The Json above is invalid. How can i make the json appear with double quotes and also remove the u from the response to get a valid json.
PS: Beginner with Python
You can use json.dumps() for that:
>>> import json
>>> my_json = {u'domain': u'127.0.0.1', u'user_id': u'example.com', u'sender': u'shop_1'}
>>> print(json.dumps(my_json))
{"domain": "127.0.0.1", "user_id": "example.com", "sender": "shop_1"}
The 'u that you see at the start of each string is an indication that this is a unicode string:
What's the u prefix in a Python string?
https://docs.python.org/2/tutorial/introduction.html#unicode-strings
Related
so I am trying to read a json file like this:
{
"Name": "hello",
"Source": " import json \n x= 10, .... "
}
the way I am trying to read it by using the json library in python
thus my code looks something like this:
import json
input =''' {
"Name": "python code",
"Source": " import json \n x= 10, .... "
}'''
output = json.load(input)
print(output)
the problem that source has the invalid character "\n". I don't want to replace \n with \n as this is code will be later run in another program.
I know that json.JSONDecoder is able to handle \n but I am not sure how to use.
You need to escape the backslash in the input string, so that it will be taken literally.
import json
input =''' {
"Name": "python code",
"Source": " import json \\n x= 10, .... "
}'''
output = json.loads(input)
print output
Also, you should be using json.loads to parse JSON in a string, json.load is for getting it from a file.
Note that if you're actually getting the JSON from a file or URL, you don't need to worry about this. Backslash only has special meaning to Python when it's in a string literal in the program, not when it's read from somewhere else.
Alternatively, you can define the string in raw format using r:
import json
# note the r
input = r''' {
"Name": "python code",
"Source": " import json \n x= 10, .... "
}'''
# loads (not load) is used to parse a string
output = json.loads(input)
print(output)
I'm running into some issues getting a cookie into a dictionary with python. It seems to be all escaped somehow even after running the command provided by requests.
resp = requests.get(geturl, cookies=cookies)
cookies = requests.utils.dict_from_cookiejar(resp.cookies)
and this is what cookies looks like
{'P-fa9d887b1fe1a997d543493080644610': '"\\050dp1\\012S\'variant\'\\012p2\\012S\'corrected\'\\012p3\\012sS\'pid\'\\012p4\\012VNTA2NjU0OTU4MDc5MTgwOA\\075\\075\\012p5\\012sS\'format\'\\012p6\\012S\'m3u8\'\\012p7\\012sS\'mode\'\\012p8\\012Vlive\\012p9\\012sS\'type\'\\012p10\\012S\'video/mp2t\'\\012p11\\012s."'}
Is there any way to make the characters unescaped in the value section of P-fa9d887b1fe1a997d543493080644610 become escaped and part of the dict itself?
Edit:
I would like the dictionary to look something like:
{'format': 'm3u8', 'variant': 'corrected', 'mode': u'live', 'pid': u'NTA2NjU0OTU4MDc5MTgwOA==', 'type': 'video/mp2t'}
You are dealing with the Python Pickle format for data serialisation. Once you have evaluated the expression, so escaped characters are unescaped, you need to load the pickle from a string using the pickle.loads function.
>>> import pickle
>>> import ast
>>> pickle.loads(ast.literal_eval("'''" + cookies.values()[0] + "'''")[1:-1])
{'pid': u'NTA2NjU0OTU4MDc5MTgwOA==', 'type': 'video/mp2t', 'variant': 'corrected', 'mode': u'live', 'format': 'm3u8'}
I've got a script which makes a JSON request that may return text in any script, then outputs the text (I dont have any control over the text being returned).
It works fine with latin characters, but other scripts output as a mojibake, and I'm not sure what's going wrong.
In the response, the problematic characters are encoded using \u syntax. In particular, I have a string containing \u00d0\u00b8\u00d1\u0081\u00d0\u00bf\u00d1\u008b\u00d1\u0082\u00d0\u00b0\u00d0\u00bd\u00d0\u00b8\u00d0\u00b5 which should output as испытание but instead outputs as иÑпÑÑание.
Obviously this is something to do with how python deals with unicode and UTF, but I despite all I've read I don't understand what's going on well enough to know how to solve it.
I've tried to extract the salient points from the code below:
response = requests.get(url, params=params, cookies=self.cookies, auth=self.auth)
text = response.text
print text
status = json.loads(text)
print status
for folder in status['folders']
print folder['name']
Output:
{ "folders": [ { "name": "\u00d0\u00b8\u00d1\u0081\u00d0\u00bf\u00d1\u008b\u00d1\u0082\u00d0\u00b0\u00d0\u00bd\u00d0\u00b8\u00d0\u00b5" } ] }
{u'folders': [{ u'name': u'\xd0\xb8\xd1\x81\xd0\xbf\xd1\x8b\xd1\x82\xd0\xb0\xd0\xbd\xd0\xb8\xd0\xb5' }]}
иÑпÑÑание
I've also tried
status = response.json();
for folder in status['folders']:
print folder['name']
With the same result.
Note, I'm really passing the string to a GTKMenuItem to be displayed, but the output from printing the string is the same as from showing it in the menu.
As #Ricardo Cárdenes said in the comment the server sends incorrect response. The response that you've got is double encoded:
>>>> u = u'\xd0\xb8\xd1\x81\xd0\xbf\xd1\x8b\xd1\x82\xd0\xb0\xd0\xbd\xd0\xb8\xd0\xb5'
>>>> print u.encode('latin-1').decode('utf-8')
испытание
The correct string would look like:
>>>> s = {u"name": u"испытание"}
>>>> import json
>>>> print json.dumps(s)
{"name": "\u0438\u0441\u043f\u044b\u0442\u0430\u043d\u0438\u0435"}
>>>> print s['name']
испытание
>>>> print s['name'].encode('unicode-escape')
\u0438\u0441\u043f\u044b\u0442\u0430\u043d\u0438\u0435
>>>> print s['name'].encode('utf-8')
испытание
>>>> s['name'].encode('utf-8')
'\xd0\xb8\xd1\x81\xd0\xbf\xd1\x8b\xd1\x82\xd0\xb0\xd0\xbd\xd0\xb8\xd0\xb5'
I've done some coding in RoR, and in Rails, when I return a JSON object via an API call, it returns as
{ "id" : "1", "name" : "Dan" }.
However in Python (with Flask and Flask-SQLAlchemy), when I return a JSON object via json.dumps or jsonpickle.encode it is returned as
"{ \"id\" : \"1\", \"name\": \"Dan\" }" which seems very unwieldily as it can't easily be parsed on the other end (by an iOS app in this case - Obj-C).
What am I missing here, and what should I do to return it as a JSON literal, rather than a JSON string?
This is what my code looks like:
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
json_object = jsonpickle.encode(friends.first().as_dict(), unpicklable=False, keys=True)
print(json_object) # this prints here, i.e. { "id" : "1", "name" : "Dan" }
return json_object # this returns "{ \"id\" : \"1\", \"name\": \"Dan\" }" to the browser
What is missing in your understanding here is that when you use the JSON modules in Python, you're not working with a JSON object. JSON is by definition just a string that matches a certain standard.
Lets say you have the string:
friends = '{"name": "Fred", "id": 1}'
If you want to work with this data in python, you will want to load it into a python object:
import json
friends_obj = json.loads(friends)
At this point friends_obj is a python dictionary.
If you want to convert it (or any other python dictionary or list) then this is where json.dumps comes in handy:
friends_str = json.dumps(friends_obj)
print friends_str
'{"name": "Fred", "id": 1}'
However if we attempt to "dump" the original friends string you'll see you get a different result:
dumped_str = json.dumps(friends)
print dumped_str
'"{\\"name\\": \\"Fred\\", \\"id\\": 1}"'
This is because you're basically attempting to encode an ordinary string as JSON and it is escaping the characters. I hope this helps make sense of things!
Cheers
Looks like you are using Django here, in which case do something like
from django.utils import simplejson as json
...
return HttpResponse(json.dumps(friends.first().as_dict()))
This is almost always a sign that you're double-encoding your data somewhere. For example:
>>> obj = { "id" : "1", "name" : "Dan" }
>>> j = json.dumps(obj)
>>> jj = json.dumps(j)
>>> print(obj)
{'id': '1', 'name': 'Dan'}
>>> print(j)
{"id": "1", "name": "Dan"}
>>> print(jj)
"{\"id\": \"1\", \"name\": \"Dan\"}"
Here, jj is a perfectly valid JSON string representation—but it's not a representation of obj, it's a representation of the string j, which is useless.
Normally you don't do this directly; instead, either you started with a JSON string rather than an object in the first place (e.g., you got it from a client request or from a text file), or you called some function in a library like requests or jsonpickle that implicitly calls json.dumps with an already-encoded string. But either way, it's the same problem, with the same solution: Just don't double-encode.
You should be using flask.jsonify, which will not only encode correctly, but also set the content-type headers accordingly.
people = models.UserRelationships.query.filter_by(user_id=user_id, active=ACTIVE_RECORD)
friends = people.filter_by(friends=YES)
return jsonify(friends.first().as_dict())
I have a python as CGI and the POST from jquery will transform json object to array, so when I see the POST from jquery, I actually see:
login_user[username]=dfdsfdsf&login_user[password]=dsfsdf
(the [ and ] already escaped)
My question is how I can convert this string back to JSON in python? Or, how can I convert this string to python array/dict structure so that I can process it easier?
[edit]
My jquery is posting:
{'login_user': {'username':username, 'password':password}}
If what you want to accomplish is to send structured data from the browser and then unpack it in your Python backend and keep the same structure, I suggest the following:
Create JavaScript objects in the browser to hold your data:
var d = {}
d['login_user'] = { 'username': 'foo', 'password': 'bar' }
Serialize to JSON, with https://github.com/douglascrockford/JSON-js
POST to your backend doing something like this:
$.post(url, {'data': encoded_json_data}, ...)
In your Python code, parse the JSON, POST in my example is where you get your POST data in your CGI script:
data = json.loads(POST['data'])
data['login_user']
import re
thestring = "login_user[username]=dfdsfdsf&login_user[password]=dafef"
pattern = re.compile(r'^login_user\[username\]=(.*)&login_user\[password\]=(.*)')
match = pattern.search(thestring)
print match.groups()
Output:
>>> ('dfdsfdsf', 'dafef')
Thus,
lp = match.groups()
print "{'login_user': {'username':"+lp[0]+", 'password':"+lp[1]+"}}"
shall bear: >>> {'login_user': {'username':dfdsfdsf, 'password':dafef}}
>>> import json
>>> data = {'login_user':{'username':'dfdsfdsf', 'password':'dsfsdf'}}
>>> json.dumps(data)
'{"login_user": {"username": "dfdsfdsf", "password": "dsfsdf"}}'
I suspect that data would already be contained in a GET var if that's coming from the URL...