How to 'deep-merge' dicts? - python

Imagine the following dicts:
a = {'key1': {'subkey1': [1, 2, 3]}}
b = {'key1': {'subkey2': [1, 2, 3]}}
I'd like to merge them to get
c = {'key1': {'subkey1': [1, 2, 3],
'subkey2': [1, 2, 3]}}
Extra nice would be a solution that returns deep-copies from a and b which I can alter without altering a or b.
c = {**a, **b}
looks nice but seems to be the same as c = copy(a).update(b) which returns same as b in my case because key1 gets overwritten by the update.
You can of course do this by hand like this (found in another answer):
def combine_dict(map1: dict, map2: dict):
def update(d: dict, u: dict):
for k, v in u.items():
if isinstance(v, collections.Mapping):
r = update(d.get(k, {}), v)
d[k] = r
else:
d[k] = u[k]
return d
_result = {}
update(_result, map1)
update(_result, map2)
return _result
But we have Python 3.5 now - maybe things have changed?

You need recursion to accomplish this. Luckily milanboers on GitHub saved us from hours of work and possible brain damage.
def deep_merge(dict1: dict, dict2: dict) -> dict:
""" Merges two dicts. If keys are conflicting, dict2 is preferred. """
def _val(v1, v2):
if isinstance(v1, dict) and isinstance(v2, dict):
return deep_merge(v1, v2)
return v2 or v1
return {k: _val(dict1.get(k), dict2.get(k)) for k in dict1.keys() | dict2.keys()}
a = {'key1': {'subkey1': [1, 2, 3]}}
b = {'key1': {'subkey2': [1, 2, 3]}}
a = deep_merge(a, b)
print(a)
Results in:
{'key1': {'subkey2': [1, 2, 3], 'subkey1': [1, 2, 3]}}

Related

Python: Merge 2 dictionaries and add 0 if key is empty

I have 2 dicts as follows:
d1 = {a:2, b:4}
d2 = {a:13, b:3, c:5}
How to merge these two and add 0 if one key is empty?
The result should be
= {a:[2,13], b:[4,3], c:[0,5]}
I tried to use this
d1 = {'a':2, 'b':4}
d2 = {'a':13, 'b':3, 'c':5}
dd = defaultdict(list)
for d in (d1, d2):
for key, value in d.items():
dd[key].append(value)
but I don't know how to add 0, I'm really new to programming.
You can get the list of keys then generate the required dictionary by using dict.get and default value as 0
d1 = {'a':2, 'b':4}
d2 = {'a':13, 'b':3, 'c':5}
keys = set((*d1.keys(), *d2.keys()))
new_dict = {k: [d.get(k, 0) for d in [d1, d2]] for k in keys}
print(new_dict) #{'a': [2, 13], 'b': [4, 3], 'c': [0, 5]}
Use below code.
d1 = {'a':2, 'b':4}
d2 = {'a':13, 'b':3, 'c':5}
keys = set((*d1.keys(), *d2.keys()))
result={}
for k in keys:
result[k]=list((d1.get(k, 0), d2.get(k,0)))
print(result)#{'a': [2, 13], 'b': [4, 3], 'c': [0, 5]}

IS THERE ANY SOLUTIONN no imports just combinations

python
def comb_gen(num_list):
num_list = [1,2,3]
should be output:
this should be any
out_put = [{1},{2},{3},{1, 2},{2, 3},{1, 3},{1, 2, 3}]*
List is mutable structure, so you cannot have it as a key of dictionary. One solution is to convert it to tuple:
my_dict = {"name": "John", "values": [1, 2, 3], "values2": True}
out = {tuple(v) if isinstance(v, list) else v: k for k, v in my_dict.items()}
print(out)
Prints:
{'John': 'name', (1, 2, 3): 'values', True: 'values2'}
EDIT:
out = [(v, k) for k, v in my_dict.items()]
print(out)
Prints:
[('John', 'name'), ([1, 2, 3], 'values'), (True, 'values2')]

Update dictionary items with a for loop

I would like update a dictionary items in a for loop here is what I have:
>>> d = {}
>>> for i in range(0,5):
... d.update({"result": i})
>>> d
{'result': 4}
But I want d to have following items:
{'result': 0,'result': 1,'result': 2,'result': 3,'result': 4}
As mentioned, the whole idea of dictionaries is that they have unique keys.
What you can do is have 'result' as the key and a list as the value, then keep appending to the list.
>>> d = {}
>>> for i in range(0,5):
... d.setdefault('result', [])
... d['result'].append(i)
>>> d
{'result': [0, 1, 2, 3, 4]}
Keys have to be unique in a dictionnary, so what you are trying to achieve is not possible. When you assign another item with the same key, you simply override the previous entry, hence the result you see.
Maybe this would be useful to you?
>>> d = {}
>>> for i in range(3):
... d['result_' + str(i)] = i
>>> d
{'result_0': 0, 'result_1': 1, 'result_2': 2}
You can modify this to fit your needs.
PHA in dictionary the key cant be same :p in your example
{'result': 0,'result': 1,'result': 2,'result': 3,'result': 4}
you can use list of multiplw dict:
[{},{},{},{}]
You can't have different values for the same key in your dictionary. One option would be to number the result:
d = {}
for i in range(0,5):
result = 'result' + str(i)
d[result] = i
d
>>> {'result0': 0, 'result1': 1, 'result4': 4, 'result2': 2, 'result3': 3}
d = {"key1": [8, 22, 38], "key2": [7, 3, 12], "key3": [5, 6, 71]}
print(d)
for key, value in d.items():
value_new = [sum(value)]
d.update({key: value_new})
print(d)
>>> d = {"result": []}
>>> for i in range(0,5):
... d["result"].append(i)
...
>>> d
{'result': [0, 1, 2, 3, 4]}

Is there any pythonic way to combine two dicts (making a list for common values)?

For example, I have two dicts.
A = {'a':1, 'b':10, 'c':2}
B = {'b':3, 'c':4, 'd':10}
I want a result like this:
{'a':1, 'b': [10, 3], 'c':[2, 4], 'd':10}
If a key appears in both the dicts, I want to list of both the values.
I'd make all values lists:
{k: filter(None, [A.get(k), B.get(k)]) for k in A.viewkeys() | B}
using dictionary view objects.
Demo:
>>> A = {'a':1, 'b':10, 'c':2}
>>> B = {'b':3, 'c':4, 'd':10}
>>> {k: filter(None, [A.get(k), B.get(k)]) for k in A.viewkeys() | B}
{'a': [1], 'c': [2, 4], 'b': [10, 3], 'd': [10]}
This at least keeps your value types consistent.
To produce your output, you need to use the set intersection and symmetric differences between the two dictionaries:
dict({k: [A[k], B[k]] for k in A.viewkeys() & B},
**{k: A.get(k, B.get(k)) for k in A.viewkeys() ^ B})
Demo:
>>> dict({k: [A[k], B[k]] for k in A.viewkeys() & B},
... **{k: A.get(k, B.get(k)) for k in A.viewkeys() ^ B})
{'a': 1, 'c': [2, 4], 'b': [10, 3], 'd': 10}
In Python 3, dict.keys() is a dictionary view, so you can just replace all .viewkeys() calls with .keys() to get the same functionality there.
I would second the notion of Martijn Pieters that you problably want to have the same type for all the values in your result dict.
To give a second option:
you could also use the defaultdict to achieve your result quite intuitively.
a defaultdict is like a dict, but it has a default constructor that is called if the key doesn't exist yet.
so you would go:
from collections import defaultdict
A = {'a':1, 'b':10, 'c':2}
B = {'b':3, 'c':4, 'd':10}
result = defaultdict(list)
for d in [A, B]:
for k, v in d.items():
result[k].append(v)
then in a later stage you still easily add more values to your result.
you can also switch to
defaultdict(set)
if you don't want duplicate values

Filter a dict of dict

I new in Python and I am not sure it is a good idea to use dict of dict but here is my question.
I have a dict of dict and I want to filter by the key of the inside dict:
a ={ 'key1' : {'id1' :[0,1,2] , 'id2' :[0,1,2], 'id3' :[4,5,6]}
'key2' : {'id3' :[0,1,2] , 'id4' :[0,1,2]}
'key3' : {'id3' :[0,1,2] , 'id1' :[4,5,6]}
}
For exemple , I want to filter by 'id1' to have :
result = { 'key1' : {'id1' :[0,1,2] }
'key3' : {'id1' :[4,5,6]}
}
I have tried the filter method by I get all the value:
r = [('key1' ,{'id1' :[0,1,2] , 'id2' :[0,1,2], 'id3' :[4,5,6]})
('key3' , {'id3' :[0,1,2] , 'id1' :[4,5,6]})
]
Furthermore the filter method returns a list and I want to keep the format as a dict.
Thanks in advance
Try this:
>>> { k: v['id1'] for k,v in a.items() if 'id1' in v }
{'key3': [4, 5, 6], 'key1': [0, 1, 2]}
For Python 2.x you might prefer to use iteritems() instead of items() and you'll still need a pretty recent python (2.7 I think) for a dictionary comprehension: for older pythons use:
dict((k, v['id1']) for k,v in a.iteritems() if 'id1' in v )
If you want to extract multiple values then I think you are best to just write the loops out in full:
def query(data, wanted):
result = {}
for k, v in data.items():
v2 = { k2:v[k2] for k2 in wanted if k2 in v }
if v2:
result[k] = v2
return result
giving:
>>> query(a, ('id1', 'id2'))
{'key3': {'id1': [4, 5, 6]}, 'key1': {'id2': [0, 1, 2], 'id1': [0, 1, 2]}}
According to the precision you gave to Duncan, here is another filtering on a list using dictionary comprehension:
>>> my_list = ['id1', 'id2']
>>> {k1 : {k2: v2 for (k2, v2) in a[k1].iteritems() if k2 in my_list} for k1 in a}
{'key3': {'id1': [4, 5, 6]}, 'key2': {}, 'key1': {'id2': [0, 1, 2], 'id1': [0, 1, 2]}}
EDIT: you can also remove empty values with another dict compreehension, but that "begins" to be difficult to read... :-)
>>> {k3: v3 for k3, v3 in {k1 : {k2: v2 for (k2, v2) in a[k1].iteritems() if k2 in my_list} for k1 in a}.iteritems() if v3}
{'key3': {'id1': [4, 5, 6]}, 'key1': {'id2': [0, 1, 2], 'id1': [0, 1, 2]}}
You can do with a dictionary comprehension:
def query(data, query):
return {key : {query : data[key][query]}
for key in data if query in data[key]}
You have to look at each entry of the dictionary, which can cost a lot of time if you have many entries or do this a lot. A database with a index can speed this up.
field = 'id1'
dict( (k,{field: d[field]}) for k,d in a.items() if field in d)

Categories