This question already has answers here:
Items in JSON object are out of order using "json.dumps"?
(8 answers)
Closed 4 years ago.
I need to show an output like below as a result of a webservice call
{
"predictions": [
{
"id": 18009,
"cuisine": "italian",
"probability": 0.17846838753494407
},
{
"id": 28583,
"cuisine": "italian",
"probability": 0.1918703125538735
}
]
}
I have the below code to create the object:
json_data = []
for i in range (0, len(predicted_labels)):
data = {}
data['id'] = int(test['id'][i])
data['cuisine'] = categoricalTransformer[predicted_labels[i]]
item = predicted_probability[i]
data['probability'] = item[predicted_labels[i]]
json_data.append(data)
json_return = {"predictions":json_data}
return jsonify(json_return)
which reorders the key alphabetically as shown below.
{
"predictions": [
{
"cuisine": "italian",
"id": 18009,
"probability": 0.17846838753494407
},
{
"cuisine": "italian",
"id": 28583,
"probability": 0.1918703125538735
}
]
}
What should I do?
Rebuild your new dictionary with a collections.defaultdict(), and append ordered dictionaries with a collections.OrderedDict():
from collections import OrderedDict
from collections import defaultdict
from json import dumps
data = {
"predictions": [
{"id": 18009, "cuisine": "italian", "probability": 0.17846838753494407},
{"id": 28583, "cuisine": "italian", "probability": 0.1918703125538735},
]
}
key_order = ["id", "cuisine", "probability"]
result = defaultdict(list)
for dic in data["predictions"]:
ordered = OrderedDict((key, dic.get(key)) for key in key_order)
result["predictions"].append(ordered)
print(dumps(result))
# {"predictions": [{"id": 18009, "cuisine": "italian", "probability": 0.17846838753494407}, {"id": 28583, "cuisine": "italian", "probability": 0.1918703125538735}]}
json.dumps() here serializes the dictionary into a JSON formatted string.
Note: If you are using Python3.6+, you can use normal dictionaries instead of OrderedDict(), since insertion order of keys is remembered. However, in 3.6 it is an implementation feature, whereas in 3.7 it is a language feature. You read more about this at Are dictionaries ordered in Python 3.6+?.
You can probably rely on this feature for applications that have a minimum requirement of Python3.6. Furthermore, its probably safer to user OrderedDict() anyways, so your code can be backwards compatible for all python versions.
You can look into Ordered Dictionary. It remembers the insertion order, and you would be able to organise your json via this collection this way.
Example:
datadict = {'a':'a', 'c':'c', 'b':'b'}
import collections
print (collections.OrderedDict(datadict))
#OrderedDict([('a':'a'), ('c':'c'), ('b':'b')])
x = collections.OrderedDict(datadict)
print (dict(x))
#{'a':'a', 'c':'c', 'b':'b'}
print (x == datadict)
#True
print (collections.OrderedDict(sorted(datadict.items(), key=lambda t:t[0])))
#OrderedDict([('a':'a'), ('b':'b'), ('c':'c')])
z = collections.OrderedDict(sorted(datadict.items(), key=lambda t:t[0]))
print (dict(z))
#{'a':'a', 'b':'b', 'c':'c'}
print (z == datadict)
#True
Using this, you can convert your json_return dict object into an OrderedDict with your desired insertion order and then return that OrderedDict object or convert it back into a dict and return that dict
Related
I have functions witch generate data witch I add t dict the think is there I want my json file to look like this 1.to have multiple data not only one key value pair like in my code:
{"data":[
{"key":"Shyam", "value":10.4},
{"key":"Bob", "value":12.5},
{"key":"Jai", "value":24.2}
]}
This is how is look like the moment only one key value pair is added:
{
"key": "Amadm",
"value": 14.5
}
This is my code to assign to dict before json dumps.
:
def gen_dict(key, value, ts):
data = {
"name": key,
"value": value,
"ts": ts
}
return data
json_object = json.dumps(gen_dict(gen_key(), gen_value()), indent = 4)
So my question is how to assign more than one key value pair in the dict and later to transform to json obj like in the example I show in the example.
You are creating a single dictionary, what you want is a list of dictionaries:
Assuming that you each call to gen_key() and gen_value() generates a single instance of the data, you can use:
# Some random key
def gen_key():
return ''.join((random.choice(string.ascii_lowercase) for x in range(5)))
# Some random value
def gen_value():
return random.choice(range(1000))
s = json.dumps({"data": [ {
"name": gen_key(), "value": gen_value()} for i in range(3)] }, indent = 4)
output:
{
"data": [
{
"name": "rrqct",
"value": 162
},
{
"name": "vbuyq",
"value": 422
},
{
"name": "kfyqt",
"value": 7
}
]
}
I am using a python and getting the data from an API the data formatted as listed in the example I have a problem getting out Cust_id and name put of the API
Below is one of the things I tried and one of the things answered by SimonR. I am sure I am doing something really dumb right now but I get the error
typeError: the JSON object must be str, bytes or bytearray, not dict. Thank you everyone in advance for your answers
import json
a = {
"count": 5,
"Customers": {
"32759": {
"cust_id": "1234",
"name": "Mickey Mouse"
},
"11053": {
"cust_id": "1235",
"name": "Mini Mouse"
},
"21483": {
"cust_id": "1236",
"name": "Goofy"
},
"12441": {
"cust_id": "1237",
"name": "Pluto"
},
"16640": {
"cust_id": "1238",
"name": "Donald Duck"
}
}
}
d = json.loads(a)
customers = {v["cust_id"]: v["name"] for v in d["Customers"].values()}
Is this what you're trying to do ?
import json
d = json.loads(a)
customers = {v["cust_id"]: v["name"] for v in d["Customers"].values()}
outputs :
{'1234': 'Mickey Mouse',
'1235': 'Mini Mouse',
'1236': 'Goofy',
'1237': 'Pluto',
'1238': 'Donald Duck'}
Well if I understood correctly you can do this:
# d is the API response in your post
# This will give you the list of customers
customers = d['Customers']
Then you can iterate over the customers dictionary and save them to any data structure you want:
# This will print out the name and cust_id
for k, v in customers.items():
print(v['cust_id'], v['name'])
Hope it helps!
import json
# convert json to python dict
response = json.loads(json_string)
# loop through all customers
for key, customer in response['Customers'].items():
# get customer id
customer['cust_id']
# get customer name
custoemr['name']
I have some JSON I'm looping through in the following format. I need to create an object for each unique primary key found in the source data and append to an array. I'm not sure how I would create the object on first encounter of the key and append to it on the next encounter. My initial attempt just creates a new object for each object in the source. Wasn't able to find an example in python only js.
Source data format:
[
...
{
"Id": "NOT NEEDED DATA",
"Client": {
"Id": "KEY",
"Name": "NOT NEEDED DATA"
},
"Name": "DESIRED DATAPOINT"
},
...
]
Desired format:
[
...
{
"client_id": "KEY",
"locations": ["DATA", "DATA"]
}
...
]
pseudocode
for i in sourcedata:
client_id = i['Client']['Id']
location_name = i['Name']
obj = {
"client_id": client_id,
"locations": [location_name]
}
new_array.append(obj)
You can first iterate and build a dictionary for then creating a list of dictionaries as specified in your output format.
from collections import defaultdict
# create and populate the dictionary
d = defaultdict(list)
for i in sourcedata:
client_id = i['Client']['Id']
location_name = i['Name']
d[client_id].append(location_name)
# build the result
res = [{"client_id": k, "locations": v} for k,v in d.items()]
I have a nested JSON data like this of about 5000 records.
{
"data": {
"attributes": [
{
"alert_type": "download",
"severity_level": "med",
"user": "10.1.1.16"
},
{
"alert_type": "download",
"severity_level": "low",
"user": "10.2.1.18"
}
]
}
}
Now , I need to parse this JSON and get only certain fields in a CSV format. Let's we would need alert_type & user in a CSV format.
I tried to parse this JSON dictionary:
>>> import json
>>> resp = '{"data":{"attributes":[{"alert_type":"download","severity_level":"med","user":"10.1.1.16"},{"alert_type":"download","severity_level":"low","user":"10.2.1.18"}]}}'
>>> user_dict = json.loads(resp)
>>> event_cnt = user_dict['data']['attributes']
>>> print event_cnt[0]['alert_type']
download
>>> print event_cnt[0]['user']
10.1.1.16
>>> print event_cnt[0]['alert_type'] + "," + event_cnt[0]['user']
download,10.1.1.16
>>>
How to get all the elements/values of a particular keys in a CSV format and in a single iteration ?
Output:
download,10.1.1.16
download,10.2.1.18
Simple list comprehension:
>>> jdict=json.loads(resp)
>>> ["{},{}".format(d["alert_type"],d["user"]) for d in jdict["data"]["attributes"]]
['download,10.1.1.16', 'download,10.2.1.18']
Which you can join for your desired output:
>>> li=["{},{}".format(d["alert_type"],d["user"]) for d in jdict["data"]["attributes"]]
>>> print '\n'.join(li)
download,10.1.1.16
download,10.2.1.18
Since {"data":{"attributes": is a list, you can loop over it and print the values for desired keys (d is the user dict):
for item in d['data']['attributes']:
print(item['alert_type'],',',item['user'], sep='')
You could make it somewhat data-driven like this:
import json
DESIRED_KEYS = 'alert_type', 'user'
resp = '''{ "data": {
"attributes": [
{
"alert_type": "download",
"severity_level": "med",
"user": "10.1.1.16"
},
{
"alert_type": "download",
"severity_level": "low",
"user": "10.2.1.18"
}
]
}
}
'''
user_dict = json.loads(resp)
for attribute in user_dict['data']['attributes']:
print(','.join(attribute[key] for key in DESIRED_KEYS))
To handle attributes that don't have all the keys, you could instead use this as the last line which will assign missing values a default value (such as a blank string as shown) instead of it causing an exception.
print(','.join(attribute.get(key, '') for key in DESIRED_KEYS))
Using jq, a one-line solution is straightforward:
$ jq -r '.data.attributes[] | [.alert_type, .user] | #csv' input.json
"download","10.1.1.16"
"download","10.2.1.18"
If you don't want the strings to be quoted, use join(",") instead of #csv
I'm new to JSON and Python, and I'm attempting to load a json file that I created from disk to manipulate and output to an xml file. I've gotten most of it figured out, except, I want to 'sort' the JSON file after I load it by a certain value.
Example of json file:
{
"0" : {
"name": "John Doe",
"finished": "4",
"logo": "http://someurl.com/icon.png"
},
"1" : {
"name": "Jane Doe",
"finished": "10",
"logo": "http://anotherurl.com/icon.png"
},
"2" : {
"name": "Jacob Smith",
"finished": "3",
"logo": "http://example.com/icon.png"
}
}
What I want to do is sort 'tree' by the 'finished' key.
JSONFILE = "file.json"
with open(CHANS) as json_file:
tree = json.load(json_file)
Depends on how do you "consume" the tree dictionary. are you using tree.keys(), tree.values() or tree.items()?
tree.keys()
ordered_keys = sorted(tree.keys(), key=lambda k: int(tree[k]['finished']))
tree.values()
ordered_keys = sorted(tree.values(), key=lambda v: int(v['finished']))
tree.items()
ordered_keys = sorted(tree.items(), key=lambda t: int(t[1]['finished']))
You only keep in mind that JSON is what's inside the actual file, the result of json.load() is just a Python value/object, so just work with them.
If you are walking over the sorted dictionary once, the above snippets will work just fine. However if you need to access it multiple times, then I would follow ~Jean-François suggestion and use OrderedDict, with something like:
from collections import OrderedDict
tree = OrderedDict(sorted(tree.items(), key=lambda t: int(t[1]['finished'])))
That way the sorting operation (arguably the most expensive) is done just once.