Consider this input dictionary:
my_dict = {
'group1':{
'type1': {'val1' : 45, 'val2' : 12, 'val3' : 65},
'type2': {'val5' : 65, 'val6' : 132, 'val7' : 656},
},
'group2':{
'type3': {'val11' : 45, 'val12' : 123, 'val13' : 3},
'type4': {'val51' : 1, 'val61' : 2, 'val71' : 3, },
},
}
I would like to remove the last 'level' (the one that only has numbers), and get something like:
new_dict = {
'group1':{
'type1': ['val1', 'val2', 'val3'],
'type2': ['val5', 'val6', 'val7'],
},
'group2':{
'type3': ['val11', 'val12', 'val13'],
'type4': ['val51', 'val61', 'val71'],
},
}
I am currently doing it by manually looping and so on, but I wonder if there is a way to do it more efficiently.
If your dictionary doesn't have a fixed number of nesting levels, you could write a recursive function to do it:
def stripBottom(d):
if not isinstance(d,dict):
return d
elif not any(isinstance(v,dict) for v in d.values()):
return list(d)
else:
return {k:stripBottom(v) for k,v in d.items()}
Output:
my_dict = {
'group1':{
'type1': {'val1' : 45, 'val2' : 12, 'val3' : 65},
'type2': {'val5' : 65, 'val6' : 132, 'val7' : 656},
},
'group2':{
'type3': {'val11' : 45, 'val12' : 123, 'val13' : 3},
'type4': {'val51' : 1, 'val61' : 2, 'val71' : 3, },
},
}
print(stripBottom(my_dict))
{'group1': {'type1': ['val1', 'val2', 'val3'],
'type2': ['val5', 'val6', 'val7']},
'group2': {'type3': ['val11', 'val12', 'val13'],
'type4': ['val51', 'val61', 'val71']}}
If your dictionary has variable levels of nesting or if you want to drop a specific level, you would need to add a parameter to the function:
def stripBottom(d,level=0):
if not isinstance(d,dict):
return d
elif level == 0:
return list(d)
else:
return {k:stripBottom(v,level-1) for k,v in d.items()}
print(stripBottom(my_dict,1))
{'group1': ['type1', 'type2'], 'group2': ['type3', 'type4']}
You don't give an example of the actual code you are using to change the dict, but likely you will always need 2 levels of looping to iterate over groups, and then types. Something like:
newdict = {}
for k, grp in my_dict.items():
newdict[k] = {}
for typ, val in grp.items():
newdict[k][typ] = list(val)
One option to be more efficient is to not pre-compute these updted values, but instead when you come to use the 'group' data, wrap the output in list at that point. (Though obviously you will still be iterating when the dict is being used, to get these nested dicts).
Related
I have a dict as below:
dict1={
'item1':
{'result':
[{'val': 228, 'no': 202}]
},
'item2':
{'result':
[{'value': 148, 'year': 201}]
}
}
How can we insert a new key 'category' to each item so that the output looks like below:
output={
'item1':
{'category':
{'result':
[{'val': 228, 'no': 202}]
}
},
'item2':
{'category':
{'result':
[{'value': 148, 'year': 201}]
}
}
}
Currently, i have key:value and im looking to insert newkey which takes same value, key:newkey:value
I tried to do dict1['item1']['category1'] but this is adding a new key value pair.
Use to modify in-place the existing dictionary dict1:
for key, value in dict1.items():
dict1[key] = { "category" : value }
print(dict1)
Output
{'item1': {'category': {'result': [{'val': 228, 'no': 202}]}}, 'item2': {'category': {'result': [{'value': 148, 'year': 201}]}}}
As an alternative use update:
dict1.update((k, {"category": v}) for k, v in dict1.items())
Note that update receives both a dictionary or an iterable of key/value pairs, from the documentation:
update() accepts either another dictionary object or an iterable of
key/value pairs (as tuples or other iterables of length two).
Finally in Python 3.9+, you can use the merge operator (|=), as below:
dict1 |= ((k, {"category": v}) for k, v in dict1.items())
Try:
out = {k: {"category": v} for k, v in dict1.items()}
print(out)
Prints:
{
"item1": {"category": {"result": [{"val": 228, "no": 202}]}},
"item2": {"category": {"result": [{"value": 148, "year": 201}]}},
}
I have a following list of "dict" for example :
List1= [{'Europe': {'DE': {'Berlin': ['jack']}}},
{'Europe': {'DE': {'KL': ['Paul']}}},
{'Europe': {'FR': {'Paris': ['Jean', "Pierre"]}}}]
and I would like to transform it with Python into a dictionary having the tree structure like :
output = {"Europe": { "DE": { "KL": ["Paul"],
"Berlin": ["Jack"]
},
"FR" : { "Paris": ["Jean", "Pierre"]}
}
}
Can you help me ? my function get the list1 and return the output dict ?
Thank you
Here is a simple recursive solution. This solution is generic enough to be able to support any future changes in the structure of your dictionary, you can even use this for purposes other than this list of countries. By taking advantage of the mutability property of dict and list, we can pass it around each recursive call and perform:
If the element in lhs (left-hand side, here is List1) doesn't appear yet in the rhs (right-hand side, here is result), copy it as is.
If the element in lhs already appears in rhs, do a recursive call to merge the inner elements.
If the value in lhs is a list, add it to the rhs.
import copy
import json
List1= [
{'Europe': {'DE': {'Berlin': ['Jack']}}},
{'Europe': {'DE': {'KL': ['Paul']}}},
{'Asia': {'PH': {'Manila': ['Jose', 'Rizal']}}},
{'Europe': {'FR': {'Paris': ['Jean', "Pierre"]}}},
{'Asia': {'PH': {'Manila': ['Andres']}}},
{'Asia': {'KH': {'Siem Reap': ['Angkor']}}},
{'Europe': {'DE': {'Berlin': ['Jill']}}},
{'Asia': {'PH': {'Cebu': ['Lapulapu']}}},
{'Asia': {'PH': {'Manila': ['Bonifacio']}}},
{'Europe': {'ES': {'Valencia': ['Paella']}}},
{'Asia': {'KH': {'Phnom Penh': ['Wat']}}},
{'Europe': {'ES': {'Valencia': ['Me gusta mucho!']}}},
{'Asia': {'PH': {'Palawan': ['Beach']}}},
{'Asia': {'PH': {'Palawan': ['Cave']}}},
{'Asia': {'PH': {'Palawan': []}}},
]
result = {}
def merge(lhs, rhs):
if isinstance(lhs, dict):
for key, value in lhs.items():
if key not in rhs:
rhs[key] = copy.deepcopy(value) # Thanks to #sabik for the code review (see comments section). To avoid updating the source data List1, here we would perform a deep copy instead of just <rhs[key] = value>.
else:
merge(value, rhs[key])
elif isinstance(lhs, list):
rhs.extend(lhs)
for item in List1:
merge(item, result)
print(json.dumps(result, indent=4))
Output:
{
"Europe": {
"DE": {
"Berlin": [
"Jack",
"Jill"
],
"KL": [
"Paul"
]
},
"FR": {
"Paris": [
"Jean",
"Pierre"
]
},
"ES": {
"Valencia": [
"Paella",
"Me gusta mucho!"
]
}
},
"Asia": {
"PH": {
"Manila": [
"Jose",
"Rizal",
"Andres",
"Bonifacio"
],
"Cebu": [
"Lapulapu"
],
"Palawan": [
"Beach",
"Cave"
]
},
"KH": {
"Siem Reap": [
"Angkor"
],
"Phnom Penh": [
"Wat"
]
}
}
}
I think you can use nested defaultdict to do it. Here is the solution, it inspired by Nested defaultdict of defaultdict
import json
from collections import defaultdict
# List1 = [{'Europe': {'DE': {'Berlin': ['jack']}}},
# {'Europe': {'DE': {'KL': ['Paul']}}},
# {'Europe': {'FR': {'Paris': ['Jean', "Pierre"]}}}]
List1 = [{'Europe': {'DE': {'Berlin': {'a': ['jack']}}}},
{'Europe': {'DE': {'Berlin': {'b': {'b1': ['xxx']}}}}},
{'Europe': {'FR': {'Paris': ['Jean', "Pierre"]}}}]
def create_tree():
return defaultdict(create_tree)
def set_node(i, node, k):
if isinstance(i, dict):
for k, v in i.items():
if isinstance(v, list):
node[k] = v
else:
node = node[k]
set_node(v, node, k)
def get_tree():
tree = create_tree()
for d in List1:
set_node(d, tree, None)
return tree
tree = get_tree()
# if you want to get the dict of tree, use:json.loads(json.dumps(tree))
print(json.dumps(tree, indent=4))
This works, although I feel like there is probably a more elegant way of doing it.
all_country_codes = set([list(x['Europe'].keys())[0] for x in List1])
output = []
for code in all_country_codes:
results = [x['Europe'][code] for x in List1 if code in x['Europe'].keys()]
country_code_dict = {}
for country_dictionary in results:
country_code_dict.update(country_dictionary)
output.append({code: country_code_dict})
almost_there_dict = {}
for reformatted_dict in output:
almost_there_dict.update(reformatted_dict)
final_dict = {}
final_dict['Europe'] = almost_there_dict
If the depth of your nested dict does not change, you can just try looping and use setdefault and update if a key is found:-
List1= [{'Europe': {'DE': {'Berlin': ['Jack']}}},
{'Europe': {'DE': {'KL': ['Paul']}}},
{'Europe': {'FR': {'Paris': ['Jean', "Pierre"]}}}]
output = {}
for l in List1:
for k,v in l.items():
#outer key
output.setdefault(k, {})
for key,val in v.items():
#inner key
output[k].setdefault(key, {})
#update inner key
output[k][key].update(val)
which returns (keep in mind depending on your Python version, dict might be sorted or not):
{'Europe': {'DE': {'Berlin': ['Jack'], 'KL': ['Paul']},
'FR': {'Paris': ['Jean', 'Pierre']}}}
This question already has answers here:
Update value of a nested dictionary of varying depth
(28 answers)
Closed 4 years ago.
I have two dictionaries, the first is what the default values of the second one should fall back to if they are not present or not defined, and it is somewhat like this:
default_dict = {
'lorem': {
'foo': 'white',
'bar': 'black',
},
'ipsum': {
'xyz': '',
'abc': {},
'qwe': {}
}
}
And the second looks like this:
custom_dict = {
'lorem': {
'bar': 'blue',
},
'ipsum': {
'xyz': 'apple',
'qwe': { 'one': 'strawberry' }
}
}
Is there any way I can "update" from the default_dict with the values from the custom_dict?
The desired result would look something like this:
custom_dict = {
'lorem': {
'foo': 'white',
'bar': 'blue',
},
'ipsum': {
'xyz': 'apple',
'abc': {},
'qwe': { 'one': 'strawberry' }
}
}
I've tried doing default_dict.update(custom_dict) and then custom_dict = default_dict, but as you can imagine I'm just getting custom_dict back untouched... So the keys of default_dict are getting removed when it is updated.
Use:
d={a:b for k,v in custom_dict.items() for a,b in v.items()}
print({k:{a:d.get(a,b) for a,b in v.items()} for k,v in default_dict.items()})
A dictionary comprehension + a nested dictionary comprehension would work.
Output:
{'lorem': {'foo': 'white', 'bar': 'blue'}, 'ipsum': {'xyz': 'apple', 'abc': {}, 'qwe': {'one': 'strawberry'}}}
if the structure of your dictionaries are always like above so the following code works fine:
for item in default_dict:
for value in default_dict[item].keys():
if value not in custom_dict[item].keys():
custom_dict[item].update({value: default_dict[item][value]})
good luck
Using Python's requests package, I have a body for a POST that has a bunch of None values, ex (dictionary, not JSON):
{
'name': 'John',
'surname': None,
'somelist': [
{
'a': 1,
'b': None
},
{
'a': None,
'b': 2
}
],
'otherdict': {
'c': False,
'd': None
}
}
and I would like the actual body that gets sent out has all the entries with values None removed, as opposed to converted to JSON null:
{
'name': 'John',
'somelist': [
{
'a': 1
},
{
'b': 2
}
],
'otherdict': {
'c': False
}
}
Does the requests package have an option that does this or do I need to do it on my end?
You can do this with recursion (remove keys with None values or recursively call the clean method if the value is not None):
def clean(d):
if type(d) == list:
return [clean(e) for e in d]
elif type(d) == dict:
for k, v in list(d.items()):
if v is None:
del d[k]
else:
d[k] = clean(v)
return d
print(clean(data))
Output
{'somelist': [{'a': 1}, {'b': 2}], 'otherdict': {'c': False}, 'name': 'John'}
I have a defaultdict(dict) values as:
defaultdict(<class 'dict'>, {
'AL2G22360.t1_Sp': {
'locus': 'AL2G22360.t1',
'length': '663',
'strain': 'Sp'},
'AL2G22360.t1_My': {
'locus': 'AL2G22360.t1',
'length': '389',
'strain': 'My'},
'AL2G22220.t1_My': {
'locus': 'AL2G22220.t1',
'length': '865',
'strain': 'My'},
'AL2G22220.t1_My': {
'locus': 'AL2G22220.t1',
'length': '553',
'strain': 'My' ........}})
Say, I want to change it in a way that the major key is value of variable **locus**. Since, there is a duplicate for locus value (non unique, but some may be unique), I want to have another sub keys My vs. Sp from variable **strain**. The rest of the data can be as it is.
Expected output:
defaultdict(<class 'dict'>, {
'AL2G22360.t1': {
'Sp': {
'keys': 'AL2G22360.t1_Sp',
'length': '663'},
'My': {
'keys': 'AL2G22360.t1_My',
'length': '389'}},
'AL2G22220.t1': {
'My': {
'keys': 'AL2G22220.t1_My',
'length': '865'},
'Sp': {
'keys': 'AL2G22220.t1_Sp',
'length': '553'}, .....}})
I'd do it like this:
result = defaultdict(lambda: defaultdict(dict))
for k, v in a.items():
result[v['locus']][v['strain']] = { 'keys': k, 'length': v['length'] }
return result
This creates a defaultdict whose values again are defaultdicts whose values are dicts. (This matches your specified output.) Then this gets filled in a straight forward manner by iterating through the original and copying all values into the new format.