I have a JSON data with structure like this:
{ "a":"1",
"b":[{ "a":"4",
"b":[{}],
"c":"6"}]
"c":"3"
}
Here the key a is always unique even if nested.
I want to separate my JSON data so that it should look like this:
{"a":"1"
"b":[]
"c":"3"
},
{"a":"4",
"b":[],
"c":"6"
}
JSON data can be nested up to many times.
How to do that?
I'd use an input and output stack:
x = {
"a":1,
"b":[
{
"a":2,
"b":[ { "a":3, }, { "a":4, } ]
}
]
}
input_stack = [x]
output_stack = []
while input_stack:
# for the first element in the input stack
front = input_stack.pop(0)
b = front.get('b')
# put all nested elements onto the input stack:
if b:
input_stack.extend(b)
# then put the element onto the output stack:
output_stack.append(front)
output_stack ==
[{'a': 1, 'b': [{'a': 2, 'b': [{'a': 3}, {'a': 4}]}]},
{'a': 2, 'b': [{'a': 3}, {'a': 4}]},
{'a': 3},
{'a': 4}]
output_stack can be a dict of cause. Then replace
output_stack.append(front)
with
output_dict[front['a']] = front
Not sure about a Python implementation, but in JavaScript this could be done using recursion:
function flatten(objIn) {
var out = [];
function unwrap(obj) {
var arrayItem = {};
for(var idx in obj) {
if(!obj.hasOwnProperty(idx)) {continue;}
if(typeof obj[idx] === 'object') {
if(isNaN(parseInt(idx)) === true) {
arrayItem[idx] = [];
}
unwrap(obj[idx]);
continue;
}
arrayItem[idx] = obj[idx];
}
if(JSON.stringify(arrayItem) !== '{}') {
out.unshift(arrayItem);
}
}
unwrap(objIn);
return out;
}
This will only work as expected if the object key names are not numbers.
See JSFiddle.
Related
I believe there must be a way to point specific key from nested dict, not in the traditional ways.
imagine dictionary like this.
dict1 = { 'level1': "value",
'unexpectable': { 'maybe': { 'onemotime': {'name': 'John'} } } }
dict2 = { 'level1': "value", 'name': 'Steve'}
dict3 = { 'find': { 'what': { 'you': { 'want': { 'in': { 'this': { 'maze': { 'I': { 'made': { 'for': { 'you': { 'which': { 'is in': { 'fact that': { 'was just': { 'bully your': { 'searching': { 'for': { 'the name': { 'even tho': { 'in fact': { 'actually': { 'there': { 'is': { 'in reality': { 'only': { 'one': { 'key': { 'named': { 'name': 'Michael' } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }
in this case, if we want to point 'name' key to get 'John' and 'Steve' and the 'Michael', you should code differently against dict1 and dict2 and dict3
and the traditional way to point the key buried in nested dictionary that I know is this.
print(dict1['unexpectable']['maybe']['onemotime']['name'])
and if you don't want your code to break because of empty value of dict, you may want to use get() function.
and I'm curious that if I want to get 'name' of dict1 safely with get(), should I code like this?
print(dict1.get('unexpectable', '').get('maybe', '').get('onemotime', '').get('name', ''))
in fact, i've got error when run those get().get().get().get() thing.
And please consider if you have to print() 'name' from that horrible dict3 even it has actually only one key.
and, imagine the case you extract 'name' from unknown dict4 which you cannot imagine what nesting structure the dict4 would have.
I believe that python must have a way to deal with this.
I searched on the internet about this problem, but the solutions seems really really difficult.
I just wanted simple solution.
the solution without pointing every keys on the every level.
like just pointing that last level key, the most important key.
like, print(dict1.lastlevel('name')) --> 'John'
like, no matter what structure of nesting levels they have, no matter how many duplicates they have, even if they omitted nested key in the middle of nested dict so that dict17 has one less level of dict16, you could get what you want, the last level value of the last level key.
So Conclusion.
I want to know if there is a simple solution like
print(dict.lastlevel('name'))
without creating custom function.
I want to know if there is solution like above from the default python methods, syntax, function, logic or concept.
The solution like above can be applied to dict1, dict2, dict3, to whatever dict would come.
There is no built-in method to accomplish what you are asking for. However, you can use a recursive function to dig through a nested dictionary. The function checks if the desired key is in the dictionary and returns the value if it is. Otherwise it iterates over the dict's values for other dictionaries and scans their keys as well, repeating until it reaches the bottom.
dict1 = { 'level1': "value",
'unexpectable': { 'maybe': { 'onemotime': {'name': 'John'} } } }
dict2 = { 'level1': "value", 'name': 'Steve'}
dict3 = { 'find': { 'what': { 'you': { 'want': { 'in': { 'this': { 'maze': { 'I': {
'made': { 'for': { 'you': { 'which': { 'is in': { 'fact that': {
'was just': { 'bully your': { 'searching': { 'for': { 'the name': {
'even tho': { 'in fact': { 'actually': { 'there': { 'is': { 'in reality': {
'only': { 'one': { 'key': { 'named': { 'name': 'Michael'
} } } } } } } } } } } } } } } } } } } } } } } } } } } } } }
def get_nested_dict_key(d, key):
if key in d:
return d[key]
else:
for item in d.values():
if not isinstance(item, dict):
continue
return get_nested_dict_key(item, key)
print(get_nested_dict_key(dict1, 'name'))
print(get_nested_dict_key(dict2, 'name'))
print(get_nested_dict_key(dict3, 'name'))
# prints:
# John
# Steve
# Michael
You can make simple recursive generator function which yields value of every particular key:
def get_nested_key(source, key):
if isinstance(source, dict):
key_value = source.get(key)
if key_value:
yield key_value
for value in source.values():
yield from get_nested_key(value, key)
elif isinstance(source, (list, tuple)):
for value in source:
yield from get_nested_key(value, key)
Usage:
dictionaries = [
{'level1': 'value', 'unexpectable': {'maybe': {'onemotime': {'name': 'John'}}}},
{'level1': 'value', 'name': 'Steve'},
{'find': {'what': {'you': {'want': {'in': {'this': {'maze': {'I': {'made': {'for': {'you': {'which': {'is in': {'fact that': {'was just': {'bully your': {'searching': {'for': {'the name': {'even tho': {'in fact': {'actually': {'there': {'is': {'in reality': {'only': {'one': {'key': {'named': {'name': 'Michael'}}}}}}}}}}}}}}}}}}}}}}}}}}}}}},
{'level1': 'value', 'unexpectable': {'name': 'Alex', 'maybe': {'onemotime': {'name': 'John'}}}},
{}
]
for d in dictionaries:
print(*get_nested_key(d, 'name'), sep=', ')
Output:
John
Steve
Michael
Alex, John
I have a dictionary like this
d = {
'Benefits': {
1: {
'BEN1': {
'D': [{'description': 'D1'}],
'C': [{'description': 'C1'}]
}
},
2: {
'BEN2': {
'D': [{'description': 'D2'}],
'C': [{'description': 'C2'}]
}
}
}
}
I am trying to sort dictionary based on KEY OF LAST VALUES(LIST).
FOR EXAMPLE
I am looking for get dictionary value like 'C' IN first and 'D' in second
I'm trying to get correct order. Here is code:
d1 = collections.OrderedDict(sorted(d.items()))
Unfortunately didn't get correct result
This is my expected output
{'Benefits':
{1:
{'BEN1':
{'C':[{'description': 'C1'}], 'D': [{'description': 'D1'}]
}
},
2:
{'BEN2':
{'C': [{'description': 'C2'}], 'D': [{'description': 'D2'}]
}
}
}
}
I am using python 3.5 . I am trying to get order like this
{'C':[{'description': 'C1'}], 'D': [{'description': 'D1'}]}
The following code will sort any dictionary by its key and recursively sort any dictionary value that is also a dictionary by its key and makes no assumption as to the content of the dictionary being sorted. It uses an OrderedDict but if you can be sure it will always run on Python 3.6 or greater, a simple change can be made to use a dict.
from collections import OrderedDict
d = {
'Benefits': {
1: {
'BEN1': {
'D': [{'description': 'D1'}],
'C': [{'description': 'C1'}]
}
},
2: {
'BEN2': {
'D': [{'description': 'D2'}],
'C': [{'description': 'C2'}]
}
}
}
}
def sort_dict(d):
items = [[k, v] for k, v in sorted(d.items(), key=lambda x: x[0])]
for item in items:
if isinstance(item[1], dict):
item[1] = sort_dict(item[1])
return OrderedDict(items)
#return dict(items)
print(sort_dict(d))
See demo
d1 = collections.OrderedDict(sorted(d.items()))
This is not working because it is sorting only on the Benefits item. Here you want to sort inner items, so we have to reach the inner items and sort them.
d1 = {'Benefits': {}}
for a_benefit in d['Benefits']:
d1['Benefits'][a_benefit] = {}
for a_ben in d['Benefits'][a_benefit]:
d1['Benefits'][a_benefit][a_ben] = dict(collections.OrderedDict(sorted(d['Benefits'][a_benefit][a_ben].items())))
I'll just go straight to example:
Here we have a dictionary with a test name, and another dict which contains the level categorization.
EDIT
Input:
test_values={
{
"name":"test1",
"level_map":{
"system":1,
"system_apps":2,
"app_test":3
}
},
{
"name":"test2",
"level_map":{
"system":1,
"system_apps":2,
"app_test":3
}
},
{
"name":"test3",
"level_map":{
"system":1,
"memory":2,
"memory_test":3
}
}
}
Output:
What I want is this:
dict_obj:
{
"system":{
"system_apps":{
"app_test":{
test1 object,
test2 object
},
"memory":{
"memory_test":{
test3 object
}
}
}
}
I just can't wrap my head around the logic and I'm struggling to even come up with an approach. If someone could guide me, that would be great.
Let's start with level_map. You can sort keys on values to get the ordered levels:
>>> level_map = { "system": 1, "system_apps": 2, "app_test": 3}
>>> L = sorted(level_map.keys(), key=lambda k: level_map[k])
>>> L
['system', 'system_apps', 'app_test']
Use these elements to build a tree:
>>> root = {}
>>> temp = root
>>> for k in L[:-1]:
... temp = temp.setdefault(k, {}) # create new inner dict if necessary
...
>>> temp.setdefault(L[-1], []).append("test") # and add a name
>>> root
{'system': {'system_apps': {'app_test': ['test']}}}
I split the list before the last element, because the last element will be associated to a list, not a dict (leaves of the tree are lists in your example).
Now, the it's easy to repeat this with the list of dicts:
ds = [{ "name": "test1",
"level_map": { "system": 1, "system_apps": 2, "app_test": 3}
}, { "name": "test2",
"level_map": { "system": 1, "system_apps": 2, "app_test": 3}
}, { "name": "test3",
"level_map": { "system": 1, "memory": 2, "memory_test": 3}
}]
root = {}
for d in ds:
name = d["name"]
level_map = d["level_map"]
L = sorted(level_map.keys(), key=lambda k: level_map[k])
temp = root
for k in L[:-1]:
temp = temp.setdefault(k, {})
temp.setdefault(L[-1], []).append(name)
# root is: {'system': {'system_apps': {'app_test': ['test1', 'test2']}, 'memory': {'memory_test': ['test3']}}}
I have a very large json file with several nested keys. From whaat I've read so far, if you do:
x = json.loads(data)
Python will interpret it as a dictionary (correct me if I'm wrong). The fourth level of nesting in the json file contains several elements named by an ID number and all of them contain an element called children, something like this:
{"level1":
{"level2":
{"level3":
{"ID1":
{"children": [1,2,3,4,5]}
}
{"ID2":
{"children": []}
}
{"ID3":
{"children": [6,7,8,9,10]}
}
}
}
}
What I need to do is to replace all items in all the "children" elements with nothing, meaning "children": [] if the ID number is in a list called new_ids and then convert it back to json. I've been reading on the subject for a few hours now but I haven't found anything similar to this to try to help myself.
I'm running Python 3.3.3. Any ideas are greatly appreciated!!
Thanks!!
EDIT
List:
new_ids=["ID1","ID3"]
Expected result:
{"level1":
{"level2":
{"level3":
{"ID1":
{"children": []}
}
{"ID2":
{"children": []}
}
{"ID3":
{"children": []}
}
}
}
}
First of all, your JSON is invalid. I assume you want this:
{"level1":
{"level2":
{"level3":
{
"ID1":{"children": [1,2,3,4,5]},
"ID2":{"children": []},
"ID3":{"children": [6,7,8,9,10]}
}
}
}
}
Now, load your data as a dictionary:
>>> with open('file', 'r') as f:
... x = json.load(f)
...
>>> x
{u'level1': {u'level2': {u'level3': {u'ID2': {u'children': []}, u'ID3': {u'children': [6, 7, 8, 9, 10]}, u'ID1': {u'children': [1, 2, 3, 4, 5]}}}}}
Now you can loop over the keys in x['level1']['level2']['level3'] and check whether they are in your new_ids.
>>> new_ids=["ID1","ID3"]
>>> for key in x['level1']['level2']['level3']:
... if key in new_ids:
... x['level1']['level2']['level3'][key]['children'] = []
...
>>> x
{u'level1': {u'level2': {u'level3': {u'ID2': {u'children': []}, u'ID3': {u'children': []}, u'ID1': {u'children': []}}}}}
You can now write x back to a file like this:
with open('myfile', 'w') as f:
f.write(json.dumps(x))
If your new_ids list is large, consider making it a set.
If you have simple dictionary like this
data_dict = {
"level1": {
"level2":{
"level3":{
"ID1":{"children": [1,2,3,4,5]},
"ID2":{"children": [] },
"ID3":{"children": [6,7,8,9,10]},
}
}
}
}
than you need only this:
data_dict = {
"level1": {
"level2":{
"level3":{
"ID1":{"children": [1,2,3,4,5]},
"ID2":{"children": [] },
"ID3":{"children": [6,7,8,9,10]},
}
}
}
}
new_ids=["ID1","ID3"]
for idx in new_ids:
if idx in data_dict['level1']["level2"]["level3"]:
data_dict['level1']["level2"]["level3"][idx]['children'] = []
print data_dict
'''
{
'level1': {
'level2': {
'level3': {
'ID2': {'children': []},
'ID3': {'children': []},
'ID1': {'children': []}
}
}
}
}
'''
but if you have more complicated dictionary
data_dict = {
"level1a": {
"level2a":{
"level3a":{
"ID2":{"children": [] },
"ID3":{"children": [6,7,8,9,10]},
}
}
},
"level1b": {
"level2b":{
"level3b":{
"ID1":{"children": [1,2,3,4,5]},
}
}
}
}
new_ids =["ID1","ID3"]
for level1 in data_dict.values():
for level2 in level1.values():
for level3 in level2.values():
for idx in new_ids:
if idx in level3:
level3[idx]['children'] = []
print data_dict
'''
{
'level1a': {
'level2a': {
'level3a': {
'ID2': {'children': []},
'ID3': {'children': []}
}
}
},
'level1b': {
'level2b': {
'level3b': {
'ID1': {'children': []}
}
}
}
}
'''
I have two dictionary objects which are very complex and created by converting large xml files into python dictionaries.
I don't know the depth of the dictionaries and just want to compare and want the following output...
e.g. My dictionaries are like this
d1 = {"great grand father":
{"name":"John",
"grand father":
{"name":"Tom",
"father":
{"name":"Andy",
"Me":
{"name":"Mike",
"son":
{"name":"Tom"}
}
}
}
}
}
d2 is also a similar but could be possible any one of the field is missing or changed as below
d2 = {"great grand father":
{"name":"John",
"grand father":
{"name":"Tom",
"father":
{"name":"Andy",
"Me":
{"name":"Tonny",
"son":
{"name":"Tom"}
}
}
}
}
}
The dictionary comparison should give me results like this -
Expected Key/Val : Me->name/"Mike"
Actual Key/Val : Me->name/"Tonny"
If the key "name" does not exists in "Me" in d2, it should give me following output
Expected Key/Val : Me->name/"Mike"
Actual Key/Val : Me->name/NOT_FOUND
I repeat the dictionary depth could be variable or dynamically generated. The two dictionaries here are given as examples...
All the dictionary comparison questions and their answers which I have seen in SO are related fixed depth Dictionaries.....
You're in luck, I did this as part of a project where I worked.
You need a recursive function something like:
def checkDifferences(dict_a,dict_b,differences=[])
You can first check for keys that don't exist in one or the other.
e.g
Expected Name/Tom Actual None
Then you compare the types of the values i.e check if the value is a dict or a list etc.
If it is then you can recursively call the function using the value as dict_a/b. When calling recursively pass the differences array.
If the type of the value is a list and the list may have dictionaries within it then you need to covert the list to a dict and call the function on the converted dictionary.
I'm sorry I can't help more but I no longer have access to the source code. Hopefully this is enough to get you started.
Here I found a way to compare any two dictionaries -
I have tried with various dictionaries of any depths and worked for me. The code is not so modular but just for the reference -
import pprint
pp = pprint.PrettyPrinter(indent=4)
dict1 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20},
'Rafa' : {'age' : 25}
}
},
'Female' : { 'Girls' : {'Serena' : {'age' : 23},
'Maria' : {'age' : 15}
}
}
},
'Animal' : { 'Huge' : {'Elephant' : {'color' : 'black' }
}
}
}
'''
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20}
}
},
'Female' : { 'Girls' : {'Serena' : {'age' : 23},
'Maria' : {'age' : 1}
}
}
}
}
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20},
'Rafa' : {'age' : 2}
}
}
}
}
'''
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 2}}},
'Female' : 'Serena'}
}
key_list = []
err_list = {}
def comp(exp,act):
for key in exp:
key_list.append(key)
exp_val = exp[key]
try:
act_val = act[key]
is_dict_exp = isinstance(exp_val,__builtins__.dict)
is_dict_act = isinstance(act_val,__builtins__.dict)
if is_dict_exp == is_dict_act == True:
comp(exp_val,act_val)
elif is_dict_exp == is_dict_act == False:
if not exp_val == act_val:
temp = {"Exp" : exp_val,"Act" : act_val}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
else:
temp = {"Exp" : exp_val, "Act" : act_val}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
except KeyError:
temp = {"Exp" : exp_val,"Act" : "NOT_FOUND"}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
key_list.pop()
comp(dict1,dict2)
pp.pprint(err_list)
Here is the output of my code -
{ 'Animal': { 'Act': 'NOT_FOUND',
'Exp': { 'Huge': { 'Elephant': { 'color': 'black'}}}},
'Person-->Female': { 'Act': 'Serena',
'Exp': { 'Girls': { 'Maria': { 'age': 15},
'Serena': { 'age': 23}}}},
'Person-->Male-->Boys-->Rafa': { 'Act': 'NOT_FOUND', 'Exp': { 'age': 25}},
'Person-->Male-->Boys-->Roger-->age': { 'Act': 2, 'Exp': 20}
}
One can also try with other dictionaries given in commented code..
One more thing - The keys are checked in expected dictionary and the matched with an actual. If we pass dictionaries in alternate order the other way matching is also possible...
comp(dict2,dict1)