How to decode JSON with Python [duplicate] - python

This question already has answers here:
How can I parse (read) and use JSON?
(5 answers)
Closed 3 years ago.
I'm getting my JSON from reddit.com, essentially something like this. I have done quite a bit of reading, but I don't really understand how I can grab the information I want from this JSON (I want a list of the story links). I understand that I can "decode" the JSON into a dictionary, but do I need to recur throughout the JSON to get what I need?
Thanks in advance.


If you're using Python 2.6 or later, use the built-in json library. Otherwise, use simplejson which has exactly the same interface.
You can do this adaptively without having to check the Python version yourself, using code such as the following:
try:
import json
except ImportError:
import simplejson as json
Then, use json.loads() or whatever as appropriate.


import urllib2
import json
u = urllib2.urlopen('http://www.reddit.com/.json')
print json.load(u)
u.close()


There are two ways you can "decode" json with Python, after you've parsed it into dicts and lists with the json library.
First, accessing it by indexes, like this:
url_list = [t['entries'][0]['url'] for t in data['windows'][0]['tabs']]
Or, you can iterate over its tree structure. The example function below isn't general purpose, it just illustrates that you need to consider JSON's three different kinds of "nodes" differently when parsing the tree. A key's "value" might be data, a list of child nodes with no keys, or a dict that's basically a new JSON object. You can't just run through checking every node for its name, data, and children like you would with a regular tree.
def depthFirstSearch(self, jsonobj, target, parentKey=None):
if isinstance(jsonobj, dict):
for key, value in jsonobj.items():
if isinstance(value, (dict, list)):
self.depthFirstSearch(value, target, key)
else: # "data" node
if key == target and parentKey not in self.parentsToExclude:
self.results.append(value)
self.parents[parentKey] += 1
if isinstance(jsonobj, list):
for value in jsonobj:
#lists don't have keys, pass along key from last dict
self.depthFirstSearch(value, target, parentKey)

Related

Is there a way to search for a string and copy text in front until it reaches a comma?

I am new to python and wanted to store the recentAveragePrice inside a variable (from a string like this one)
{"assetStock":null,"sales":250694,"numberRemaining":null,"recentAveragePrice":731,"originalPrice":null,"priceDataPoints":[{"value":661,"date":"2022-08-11T05:00:00Z"},{"value":592,"date":"2022-08-10T05:00:00Z"},{"value":443,"date":"2022-08-09T05:00:00Z"}],"volumeDataPoints":[{"value":155,"date":"2022-08-11T05:00:00Z"},{"value":4595,"date":"2022-08-10T05:00:00Z"},{"value":12675,"date":"2022-08-09T05:00:00Z"},{"value":22179,"date":"2022-08-08T05:00:00Z"},{"value":15181,"date":"2022-08-07T05:00:00Z"},{"value":14541,"date":"2022-08-06T05:00:00Z"},{"value":15310,"date":"2022-08-05T05:00:00Z"},{"value":14146,"date":"2022-08-04T05:00:00Z"},{"value":13083,"date":"2022-08-03T05:00:00Z"},{"value":14460,"date":"2022-08-02T05:00:00Z"},{"value":16809,"date":"2022-08-01T05:00:00Z"},{"value":17571,"date":"2022-07-31T05:00:00Z"},{"value":23907,"date":"2022-07-30T05:00:00Z"},{"value":39007,"date":"2022-07-29T05:00:00Z"},{"value":38823,"date":"2022-07-28T05:00:00Z"}]}
My current solution is this:
var = sampleStr[78] + sampleStr[79] + sampleStr[80]
It works for the current string but if the recentAveragePrice was above 999 it would stop working and i was wondering if instead of getting a fixed number i could search for it inside the string.

Your replit code shows that you're acquiring JSON data from some website. Here's an example based on the URL that you're using. It shows how you check the response status, acquire the JSON data as a Python dictionary then print a value associated with a particular key. If the key is missing, it will print None:
import requests
(r := requests.get('https://economy.roblox.com/v1/assets/10159617728/resale-data')).raise_for_status()
jdata = r.json()
print(jdata.get('recentAveragePrice'))
Output:
640

Since this is json you should just be able to parse it and access recentAveragePrice:
import json
sample_string = '''{"assetStock":null,"sales":250694,"numberRemaining":null,"recentAveragePrice":731,"originalPrice":null,"priceDataPoints":[{"value":661,"date":"2022-08-11T05:00:00Z"},{"value":592,"date":"2022-08-10T05:00:00Z"},{"value":443,"date":"2022-08-09T05:00:00Z"}],"volumeDataPoints":[{"value":155,"date":"2022-08-11T05:00:00Z"},{"value":4595,"date":"2022-08-10T05:00:00Z"},{"value":12675,"date":"2022-08-09T05:00:00Z"},{"value":22179,"date":"2022-08-08T05:00:00Z"},{"value":15181,"date":"2022-08-07T05:00:00Z"},{"value":14541,"date":"2022-08-06T05:00:00Z"},{"value":15310,"date":"2022-08-05T05:00:00Z"},{"value":14146,"date":"2022-08-04T05:00:00Z"},{"value":13083,"date":"2022-08-03T05:00:00Z"},{"value":14460,"date":"2022-08-02T05:00:00Z"},{"value":16809,"date":"2022-08-01T05:00:00Z"},{"value":17571,"date":"2022-07-31T05:00:00Z"},{"value":23907,"date":"2022-07-30T05:00:00Z"},{"value":39007,"date":"2022-07-29T05:00:00Z"},{"value":38823,"date":"2022-07-28T05:00:00Z"}]}'''
data = json.loads(sample_string)
recent_price = data['recentAveragePrice']
print(recent_price)
outputs:
731

Your data is in a popular format called JSON (JavaScript Object Notation). It's commonly used to exchange data between different systems like a server and a client, or a Python program and JavaScript program.
Now Python doesn't use JSON per-se, but it has a data type called a dictionary that behaves very similarly to JSON. You can access elements of a dictionary as simply as:
print(my_dictionary["recentAveragePrice"])
Python has a built-in library meant specifically to handle JSON data, and it includes a function called loads() that can convert a string into a Python dictionary. We'll use that.
Finally, putting all that together, here is a more robust program to help parse your string and pick out the data you need. Dictionaries can do a lot more cool stuff, so make sure you take a look at the links above.
# import the JSON library
# specifically, we import the `loads()` function, which will convert a JSON string into a Python object
from json import loads
# let's store your string in a variable
original_string = """
{"assetStock":null,"sales":250694,"numberRemaining":null,"recentAveragePrice":731,"originalPrice":null,"priceDataPoints":[{"value":661,"date":"2022-08-11T05:00:00Z"},{"value":592,"date":"2022-08-10T05:00:00Z"},{"value":443,"date":"2022-08-09T05:00:00Z"}],"volumeDataPoints":[{"value":155,"date":"2022-08-11T05:00:00Z"},{"value":4595,"date":"2022-08-10T05:00:00Z"},{"value":12675,"date":"2022-08-09T05:00:00Z"},{"value":22179,"date":"2022-08-08T05:00:00Z"},{"value":15181,"date":"2022-08-07T05:00:00Z"},{"value":14541,"date":"2022-08-06T05:00:00Z"},{"value":15310,"date":"2022-08-05T05:00:00Z"},{"value":14146,"date":"2022-08-04T05:00:00Z"},{"value":13083,"date":"2022-08-03T05:00:00Z"},{"value":14460,"date":"2022-08-02T05:00:00Z"},{"value":16809,"date":"2022-08-01T05:00:00Z"},{"value":17571,"date":"2022-07-31T05:00:00Z"},{"value":23907,"date":"2022-07-30T05:00:00Z"},{"value":39007,"date":"2022-07-29T05:00:00Z"},{"value":38823,"date":"2022-07-28T05:00:00Z"}]}
"""
# convert the string into a dictionary object
dictionary_object = loads(original_string)
# access the element you need
print(dictionary_object["recentAveragePrice"])
Output upon running this program:
$ python exp.py
731

How to extract only wanted property from JSON object

When I run the code:
import requests
import json
def get_fact():
catFact = requests.get("https://catfact.ninja/fact?max_length=140")
json_data = json.loads(catFact.text)
return json_data
print(get_fact())
The output is like
{'fact': "Cats are the world's most popular pets, outnumbering dogs by as many as three to one", 'length': 84}
However I just want the fact.
How do I get rid of the 'fact:' at the front and 'length:' at the back?

What you want is to access the key in the python dict you made with the json.loads call. We actually don't need the json library as requests can read and deserialize JSON itself.
This code also checks if the response was OK and fails with informative error message. It follows PEP 20 – The Zen of Python.
import requests
def get_fact():
# Get the facts dictionary in a JSON serialized form.
cat_fact_response = requests.get("https://catfact.ninja/fact?max_length=140")
# Let the response raise the exception if something bad happened to the cat facts server connection.
cat_fact_response.raise_for_status()
# Deserialize the json (make a Python dict from the text we got). requests can do that on it's own:
cat_fact_dict = cat_fact_response.json()
# Access the fact from the json from the dictionary
return cat_fact_dict['fact']
print(get_fact())
When called you get following output as wanted:
# python3 script.py
The cat's tail is used to maintain balance.

Short answer:
you need to use either get_fact()['fact'] or get_fact().get('fact'). The former will throw an exception if fact doesn't exist whereas the latter will return None.
Why:
In your code sample you fetch some json data, and then print out the entire bit of json. When you parse json, the output is a key/value map called a dictionary (or map or object in other languages). The dictionary in this case contains two keys: fact and length. If you only one want of the values, then you need to tell python that you want only a single value -- fact in this case.
Remember though: this wouldn't apply to every json object you read. Not every one is going to have a fact key.

What you are returning in get_fact is a complete JSON object which you are then printing.
To get just its property fact (without the length) use a reference to that key or property like:
return json_data["fact"]
Below is also a link to a tutorial on using JSON in Python:
w3schools: Python JSON

To extract fact field from the response, use:
import requests
import json
def get_fact():
catFact = requests.get("https://catfact.ninja/fact?max_length=140")
json_data = json.loads(catFact.text)
return json_data['fact'] # <- HERE
print(get_fact())
Output:
Cats have "nine lives" thanks to a flexible spine and powerful leg and back muscles
Note: you don't need json module here, use json() method of Response instance returned by requests:
import requests
def get_fact():
catFact = requests.get("https://catfact.ninja/fact?max_length=140").json()
return catFact['fact']
print(get_fact())

Extract a dictionnary value in Python

I have an API call result in Python which is returning the following:
b'[{"type":"deposit","currency":"bch","amount":"0.00000001","available":"0.00000001"}]'
I tried to extract the value 0.00000001 but without any success.
I know how to extract values from lists and dictionaries in Python,but as there is the b' value before the results I am not figuring out how to get it.
Any ideas?

I think what you have here is actually a bytes string, rather than a Python dictionary. Try this to convert it to a dictionary (actually a list containing a dictionary given the square brackets):
import json
data = json.loads(b'[{"type":"deposit","currency":"bch","amount":"0.00000001","available":"0.00000001"}]')
value = data[0]['amount']

The API is probably returning json data, you should parse it this way:
import json
data = json.loads(json_data)
print data[0]['amount']
json_data is what the API returns

Parse JSON in python to a dictionary

A bit lost after much research. My code below parses the JSON to a dictionary I have thought using json load
response = json.load(MSW) # -->Will take a JSON String & Turn it into a python dict
Using the iteration below I return a series like this which is fine
{u'swell': {u'components': {u'primary': {u'direction': 222.5}}}}
{u'swell': {u'components': {u'primary': {u'direction': 221.94}}}}
ourResult = response
for rs in ourResult:
print rs
But how oh how do I access the 222.5 value. The above appears to just be one long string eg response[1] and not a dictionary structure at all.
In short all I need is the numerical value (which I assume is a part of that sting) so I can test conditions in the rest of my code. Is is a dictionary? With thanks as new and lost

You have to use python syntax as follows:
>>> print response['swell']['components']['primary']['direction']
222.5

Just access the nested dictionaries, unwrapping each layer with an additional key:
for rs in ourResult:
print rs['components']['primary']['direction']

Parsing JSON string/object in Python

I've recently started working with JSON in python. Now I'm passing a JSON string to Python(Django) through a post request. Now I want to parse/iterate of that data. But I can't find a elegant way to parse this data, which somehow I'm pretty sure exists.
data = request.POST['postformdata']
print data
{"c1r1":"{\"Choice\":\"i1\"}","c2r1":"{\"Bool\":\"i2\"}","c1r2":"{\"Chars\":\"i3\"}"}
jdata = json.loads(data)
print jdata
{u'c1r2': u'{"Chars":"i3"}', u'c1r1': u'{"Choice":"i1"}', u'c2r1': u'{"Bool":"i2"}'}
This is what was expected. But now when I want to get the values, I start running into problems. I have to do something like
mydecoder = json.JSONDecoder()
for part in mydecoder.decode(data):
print part
# c1r2 c1r1 c2r1 ,//Was expecting values as well
I was hoping to get the value + key, instead of just the key. Now, I have to use the keys to get values using something like
print jdata[key]
How do I iterate over this data in a simpler fashion, so that I can iterate over key, values?

To iterate key and value, you can write
for key, value in jdata.iteritems():
print key, value
You can read the document here: dict.iteritems

Just for others to help. In Python3 dict.iteritems() has been renamed to dict.iter()

Categories