When I call the JSON file for Vincent van Gogh's List of Works wikipedia page, using this url,
it obviously returns a huge blob of text which I believe is some sort of dictionary of lists.
Now, someone has already shown me Python's import wikipedia feature, so skip that. How can I decode this JSON? I feel like I have tried everything in Python 3's library, and always get an error, like I get if I try this code for example:
data = urllib.request.urlopen(long_json_url)
stuff = json.load(data) #or json.loads(data)
print(stuff)
it returns
TypeError: the JSON object must be str, not 'bytes'
Or if I try this code:
data = urllib.request.urlopen(longurl)
json_string = data.read().decode('utf-8')
json_data = json.loads(json_string)
print(json_data)
It doesn't return an error, but just what looks like nothing
>>>
>>>
But if I highlight that empty space and paste it, it pastes the same blob of text.
{'warnings': {'main': {'*': "Unrecognized parameter: 'Page'"}}, 'query': {'normalized': [{'from': 'list of works by Vincent van Gogh',... etc
If I try a for loop:
for entry in json_data:
print(entry)
It returns
>>>
query
warnings
>>>
And that's it. So it's not returning an error there, but not really much else, just two values? How would you make the JSON data into a workable Python dict or list? Or at the very least, into a more vertical format that I could actually read?
How would you make the JSON data into a workable Python dict or list?
You're already doing that with
json_data = json.loads(json_string)
This however:
for entry in json_data:
print(entry)
will only print the keys of your dictionaries. If you want to print the values, you need to use:
for entry in json_data:
print(json_data[entry])
if you inspect the data, you'll see that there are two keys for the main dictionary. The ones you already got by iterating over the dict:
{u'query': {...}, u'warnings': {...}}
Related
I am creating a python script to extract values, Name from the JSON Key Details from the JSON result. Python error mentioned KeyError 'details[name]'. The JSON example is found below. The JSON is incomplete. JSON has other data which I am not going to put it here as it is confidential.
details: {'id': 5555, 'name': 'Timothy', 'Gender': 'Male'}
My Python script is shown below
print(json_data['details[name]'])
Error Message
print(json_data['details[name]'])
KeyError: 'details[name]'
I want to print the result
Timothy
What am I missing?
Assuming json_data is the name you've chosen for the entirety of the JSON object, you need provide a proper index for json_data in your print statement. Your current index is a string because you've captured your brackets inside of the apostrophes. The way you're indexing the JSON object is also incorrect.
JSON objects are essentially multidimensional dictionaries. The way you print the value of a value of a key is like this: print(dict["key"]["value"]).
Assuming the details key is also a string, the correct way to print the value of name from the key "details" is: print(json_data["details"]["name"]
do it one key at a time rather than trying to fit both the keys into one set of quotes as you originally had.
print(json_data['details']['name'])
I have a data.json file, which looks like this:
["{\"Day\":\"Today\",\"Event\":\"1\", \"Date\":\"2019-03-20\"}"]
I am trying to get "Event" from this file using python and miserably failing at this.
with open('data.json', 'r') as json_file:
data = json.load(json_file)
print (data['Event'])
I get the following error:
TypeError: list indices must be integers or slices, not str
And even when I try
print (data[0]['Event'])
then I get this error:
TypeError: string indices must be integers
One more thing:
print(type(data))
gives me "list"
I have searched all over and have not found a solution to this. I would really appreciate your suggestions.
You could use the ast module for this:
import ast
mydata = ["{\"Day\":\"Today\",\"Event\":\"1\", \"Date\":\"2019-03-20\"}"]
data = ast.literal_eval(mydata[0])
data
{'Day': 'Today', 'Event': '1', 'Date': '2019-03-20'}
data['Event']
'1'
Edit
Your original code does load the data into a list structure, but only contains a single string entry inside that list, despite proper json syntax. ast, like json, will parse that string entry into a python data structure, dict.
As it sits, when you try to index that list, it's not the same as calling a key in a dict, hence the slices cannot be str:
alist = [{'a':1, 'b':2, 'c':3}]
alist['a']
TypeError
# need to grab the dict entry from the list
adict = alist[0]
adict['a']
1
You need to convert the elements in data to dict using json module.
Ex:
import json
with open(filename) as infile:
data = json.load(infile)
for d in data:
print(json.loads(d)['Event'])
Or:
data = list(map(json.loads, data))
print(data[0]["Event"])
Output:
1
Your problem is that you are parsing it as a list that consists of a single element that is a string.
["{\"Day\":\"Today\",\"Event\":\"1\", \"Date\":\"2019-03-20\"}"]
See how the entire content of the list is surrounded by " on either side and every other " is preceded by a \? The slash generally means to ignore the special meaning the following character might have, but interpret it as purely a string.
If you have control over the file's contents, the easiest solution would be to adjust it. You will want it to be in a format like this:
[{"Day":"Today", "Event": "1", "Date": "2019-03-20"}]
Edit: As others have suggested, you can also parse it in its current state. Granted, cleaning the data is tedious, but oftentimes worth the effort. Though this may not be one of those cases. I'm leaving this answer up anyway because it may help with explaining why OPs initial attempt did not work, and why he received the error messages he got.
I have a simple file (username.json) as shown below:
{"lastname": "doe", "firstname": "john"}
I use the following code to read the file:
with open(filename) as file_obj:
dictionary = json.load(file_obj)
print(dictionary['firstname'])
But when I print the dictionary value for the key "firstname" it prints nothing.
When I print the dictionary I get the following:
{u'lastname': u'doe', u'firstname': u'john'}
I know that "u" stands for unicode but for some reason I am not able to use the firstname and lastname keys.
UPDATE:
For some reason it works now!
json.loads converts a json object to the python equivalent.
This means it uses lists and dicts instead of arrays and objects. You are seeing the representation of the former.
doctionary["firstname"] will get you the value in first name (ie, "doe") while it's still a python object.
If you want to see json again, you'll need to pass it through json.dumps - but of course you won't be able to manipulate it as above when in that format.
I don't have much experience in Python and I've ran into problem converting sql query data which is technically a list containing a JSON string into a Python dictionary. I'm querying the sqlite3 database which is returning a piece of data like this:
def get_answer(self, id):
self.__cur.execute('select answer from some_table where id= %s;' % id)
res_data = self.__cur.fetchall()
return res_data
The data is a single JSON format element which its simplified version looks like this:
[
{"ind": [ {"v": 101}, {"v": 102}, {"v": 65 }]},
{"ind": [ {"v": 33}, {"v": 102}, {"v": 65}]}
]
But when I try to convert the res_data to JSON, with code like this:
temp_json = simplejson.dumps(get_answer(trace_id))
it returns a string and when I get the len(temp_json) it returns the number of characters in res_data instead of the number of objects in res_data. However, if I use Visual Studio's JSON visualizer on what get_answer(trace_id) returns, it properly shows each of the objects res_data.
I also tried to convert the res_data to a dictionary with code like this:
dict_data = ast.literal_eval(Json_data)
or
dict_data = ast.literal_eval(Json_data[0])
and in both cases it throws a "malformed string" exception. I tried to write it to a file and read it back as a JSON but it didn't worked.
Before doing that I had the copy pasted the res_data manually and used:
with open(file_name) as json_file:
Json_data = simplejson.load(json_file)
and it worked like a charm. I've been experimenting different ways stated in SO and elsewhere but although the problem seems very straight forward, I still haven't found a solution so your help is highly appreciated.
OK, I finally found the solution:
states = json.loads(temp_json[0][0])
one confusing issue was that states = json.loads(temp_json[0]) was throwing the "Expected string or buffer" exception and temp_json was a list containing only one element, so I didn't think I will get anything from temp_json[0][0].
I hope it helps others too!
I think you are confusing the data formats. What you supposedly get back from your database wrapper seems to be a list of dictionaries (it is not SQL either - your question title is misleading). But I actually think that sqlite3 would give you a list of tuples.
JSON is a text format or more precisely the serialization of an object to a string. That's why json.dumps (= dump to string) results in a string and json.loads(= load string) would result in a python object, a dictionary or a list of dictionaries).
json.dumps does NOT mean "dump JSON to string". It is rather the .dumps method of the json module which takes a Python object (dict, list) and serializes it to JSON.
I will extend the answer if I understand what exactly you want to achieve but you get JSON with json.dumps(), JSON is a string format. In contrast simplejson.load() gives you a Python list or dict.
Did you try json.loads() just in case the return from your database is actually a string (which you could easily test).
I'm making a python bot and the response is coming back as
JSON.
Here is a quick show of what it brings back:
[["Message","User string from here."]]
So first what I've done is, loaded the json from the python module json.
json.loads(resp)
and it brings back:
[[u"Message",u"user string from here"]]
How do I print out the Message which will return the value user string from here?
All you need to do is:
from __future__ import print_function
import json
raw_response = '[["Message","User string from here."]]'
data = json.loads(raw_response)
print(data[0][1])
json.loads converts json into the native datatypes of python. So, when you did json.loads(resp), you get back a list of lists.
You can simply iterate over the list to access whatever elements you want.
In your case, print json.loads(resp)[0][1] should suffice.