How can I turn a list of dicts like [{'a':1}, {'b':2}, {'c':1}, {'d':2}], into a single dict like {'a':1, 'b':2, 'c':1, 'd':2}?
Answers here will overwrite keys that match between two of the input dicts, because a dict cannot have duplicate keys. If you want to collect multiple values from matching keys, see How to merge dicts, collecting values from matching keys?.
This works for dictionaries of any length:
>>> result = {}
>>> for d in L:
... result.update(d)
...
>>> result
{'a':1,'c':1,'b':2,'d':2}
As a comprehension:
# Python >= 2.7
{k: v for d in L for k, v in d.items()}
# Python < 2.7
dict(pair for d in L for pair in d.items())
In case of Python 3.3+, there is a ChainMap collection:
>>> from collections import ChainMap
>>> a = [{'a':1},{'b':2},{'c':1},{'d':2}]
>>> dict(ChainMap(*a))
{'b': 2, 'c': 1, 'a': 1, 'd': 2}
Also see:
What is the purpose of collections.ChainMap?
Little improvement for #dietbuddha answer with dictionary unpacking from PEP 448, for me, it`s more readable this way, also, it is faster as well:
from functools import reduce
result_dict = reduce(lambda a, b: {**a, **b}, list_of_dicts)
But keep in mind, this works only with Python 3.5+ versions.
This is similar to #delnan but offers the option to modify the k/v (key/value) items and I believe is more readable:
new_dict = {k:v for list_item in list_of_dicts for (k,v) in list_item.items()}
for instance, replace k/v elems as follows:
new_dict = {str(k).replace(" ","_"):v for list_item in list_of_dicts for (k,v) in list_item.items()}
unpacks the k,v tuple from the dictionary .items() generator after pulling the dict object out of the list
For flat dictionaries you can do this:
from functools import reduce
reduce(lambda a, b: dict(a, **b), list_of_dicts)
You can use join function from funcy library:
from funcy import join
join(list_of_dicts)
>>> L=[{'a': 1}, {'b': 2}, {'c': 1}, {'d': 2}]
>>> dict(i.items()[0] for i in L)
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
Note: the order of 'b' and 'c' doesn't match your output because dicts are unordered
if the dicts can have more than one key/value
>>> dict(j for i in L for j in i.items())
If you don't need the singleton dicts anymore:
>>> L = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
>>> dict(map(dict.popitem, L))
{'a': 1, 'b': 2, 'c': 1, 'd': 2}
dict1.update( dict2 )
This is asymmetrical because you need to choose what to do with duplicate keys; in this case, dict2 will overwrite dict1. Exchange them for the other way.
EDIT: Ah, sorry, didn't see that.
It is possible to do this in a single expression:
>>> from itertools import chain
>>> dict( chain( *map( dict.items, theDicts ) ) )
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
No credit to me for this last!
However, I'd argue that it might be more Pythonic (explicit > implicit, flat > nested ) to do this with a simple for loop. YMMV.
this way worked for me:
object = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
object = {k: v for dct in object for k, v in dct.items()}
printing object:
object = {'a':1,'b':2,'c':1,'d':2}
thanks Axes
>>> dictlist = [{'a':1},{'b':2},{'c':1},{'d':2, 'e':3}]
>>> dict(kv for d in dictlist for kv in d.iteritems())
{'a': 1, 'c': 1, 'b': 2, 'e': 3, 'd': 2}
>>>
Note I added a second key/value pair to the last dictionary to show it works with multiple entries.
Also keys from dicts later in the list will overwrite the same key from an earlier dict.
Related
Is there a better/pythonic way to do the following:
I have a function that merges dictionaries:
def merge_dicts(a, *dict_args):
for dictionary in dict_args:
for k, v in dictionary.items():
if k not in a:
a[k] = v
return a
Here is a sample run:
a = {'A': 1, 'B': 2}
b = {'B': 3, 'C': 4}
c = merge_dicts(a, b) # {'A': 1, 'B': 2, 'C': 4}
I am using python2.7.
You can use update. Since the earlier dicts have priority you have to update in reverse order, and update with a last:
def merge_dicts(a, *dict_args):
d = {}
for dictionary in reversed(dict_args):
d.update(dictionary)
d.update(a)
return d
Or as a one-liner, using itertools.chain:
from itertools import chain
def merge_dicts(a, *dict_args):
# chain (key, value) items in order of increasing priority
return dict(chain.from_iterable(d.iteritems() for d in dict_args[::-1]+(a,)))
> merge_dicts(a, b)
{'A': 1, 'C': 4, 'B': 2}
If I may add, why not remove a from the function signature altogether:
def merge_dicts(*dict_args):
return dict(chain.from_iterable(d.iteritems() for d in dict_args[::-1]))
# If you provide 0 or 1 dict,
# this will return an empty dict or the single dict (a copy thereof) itself
You don't need to check the existence of keys in dictionaries, since you want to preserve the first key you can use a dict comprehension by looping through the list of dictionaries backward:
{k: v for d in list_of_dict[::-1] for k, v in d.items()}
Python will replace the existence keys with new ones, each time it encounter a duplicate one, and since you are looping through the list backward, it will comes up with the first keys in your aggregated dictionary.
Based on your example:
>>> {k: v for d in l[::-1] for k, v in d.items()}
{'A': 1, 'C': 4, 'B': 2}
I have a dictionary of lists in which some of the values are empty:
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
At the end of creating these lists, I want to remove these empty lists before returning my dictionary. I tried doing it like this:
for i in d:
if not d[i]:
d.pop(i)
but I got a RuntimeError. I am aware that you cannot add/remove elements in a dictionary while iterating through it...what would be a way around this then?
See Modifying a Python dict while iterating over it for citations that this can cause problems, and why.
In Python 3.x and 2.x you can use use list to force a copy of the keys to be made:
for i in list(d):
In Python 2.x calling keys made a copy of the keys that you could iterate over while modifying the dict:
for i in d.keys():
But note that in Python 3.x this second method doesn't help with your error because keys returns an a view object instead of copying the keys into a list.
You only need to use copy:
This way you iterate over the original dictionary fields and on the fly can change the desired dict d.
It works on each Python version, so it's more clear.
In [1]: d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
In [2]: for i in d.copy():
...: if not d[i]:
...: d.pop(i)
...:
In [3]: d
Out[3]: {'a': [1], 'b': [1, 2]}
(BTW - Generally to iterate over copy of your data structure, instead of using .copy for dictionaries or slicing [:] for lists, you can use import copy -> copy.copy (for shallow copy which is equivalent to copy that is supported by dictionaries or slicing [:] that is supported by lists) or copy.deepcopy on your data structure.)
Just use dictionary comprehension to copy the relevant items into a new dict:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.items() if v}
>>> d
{'a': [1], 'b': [1, 2]}
For this in Python 2:
>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.iteritems() if v}
>>> d
{'a': [1], 'b': [1, 2]}
This worked for me:
d = {1: 'a', 2: '', 3: 'b', 4: '', 5: '', 6: 'c'}
for key, value in list(d.items()):
if value == '':
del d[key]
print(d)
# {1: 'a', 3: 'b', 6: 'c'}
Casting the dictionary items to list creates a list of its items, so you can iterate over it and avoid the RuntimeError.
I would try to avoid inserting empty lists in the first place, but, would generally use:
d = {k: v for k,v in d.iteritems() if v} # re-bind to non-empty
If prior to 2.7:
d = dict( (k, v) for k,v in d.iteritems() if v )
or just:
empty_key_vals = list(k for k in k,v in d.iteritems() if v)
for k in empty_key_vals:
del[k]
To avoid "dictionary changed size during iteration error".
For example: "when you try to delete some key",
Just use 'list' with '.items()'. Here is a simple example:
my_dict = {
'k1':1,
'k2':2,
'k3':3,
'k4':4
}
print(my_dict)
for key, val in list(my_dict.items()):
if val == 2 or val == 4:
my_dict.pop(key)
print(my_dict)
Output:
{'k1': 1, 'k2': 2, 'k3': 3, 'k4': 4}
{'k1': 1, 'k3': 3}
This is just an example. Change it based on your case/requirements.
For Python 3:
{k:v for k,v in d.items() if v}
You cannot iterate through a dictionary while it’s changing during a for loop. Make a casting to list and iterate over that list. It works for me.
for key in list(d):
if not d[key]:
d.pop(key)
Python 3 does not allow deletion while iterating (using the for loop above) a dictionary. There are various alternatives to do it; one simple way is to change the line
for i in x.keys():
with
for i in list(x)
The reason for the runtime error is that you cannot iterate through a data structure while its structure is changing during iteration.
One way to achieve what you are looking for is to use a list to append the keys you want to remove and then use the pop function on dictionary to remove the identified key while iterating through the list.
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
pop_list = []
for i in d:
if not d[i]:
pop_list.append(i)
for x in pop_list:
d.pop(x)
print (d)
For situations like this, I like to make a deep copy and loop through that copy while modifying the original dict.
If the lookup field is within a list, you can enumerate in the for loop of the list and then specify the position as the index to access the field in the original dict.
Nested null values
Let's say we have a dictionary with nested keys, some of which are null values:
dicti = {
"k0_l0":{
"k0_l1": {
"k0_l2": {
"k0_0":None,
"k1_1":1,
"k2_2":2.2
}
},
"k1_l1":None,
"k2_l1":"not none",
"k3_l1":[]
},
"k1_l0":"l0"
}
Then we can remove the null values using this function:
def pop_nested_nulls(dicti):
for k in list(dicti):
if isinstance(dicti[k], dict):
dicti[k] = pop_nested_nulls(dicti[k])
elif not dicti[k]:
dicti.pop(k)
return dicti
Output for pop_nested_nulls(dicti)
{'k0_l0': {'k0_l1': {'k0_l2': {'k1_1': 1,
'k2_2': 2.2}},
'k2_l1': 'not '
'none'},
'k1_l0': 'l0'}
The Python "RuntimeError: dictionary changed size during iteration" occurs when we change the size of a dictionary when iterating over it.
To solve the error, use the copy() method to create a shallow copy of the dictionary that you can iterate over, e.g., my_dict.copy().
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in my_dict.copy():
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
You can also convert the keys of the dictionary to a list and iterate over the list of keys.
my_dict = {'a': 1, 'b': 2, 'c': 3}
for key in list(my_dict.keys()):
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
If the values in the dictionary were unique too, then I used this solution:
keyToBeDeleted = None
for k, v in mydict.items():
if(v == match):
keyToBeDeleted = k
break
mydict.pop(keyToBeDeleted, None)
I have a dictionary
d = {'a':1, 'b':2, 'c':3}
I need to remove a key, say c and return the dictionary without that key in one function call
{'a':1, 'b':2}
d.pop('c') will return the key value - 3 - instead of the dictionary.
I am going to need one function solution if it exists, as this will go into comprehensions
How about this:
{i:d[i] for i in d if i!='c'}
It's called Dictionary Comprehensions and it's available since Python 2.7.
or if you are using Python older than 2.7:
dict((i,d[i]) for i in d if i!='c')
Why not roll your own? This will likely be faster than creating a new one using dictionary comprehensions:
def without(d, key):
new_d = d.copy()
new_d.pop(key)
return new_d
If you need an expression that does this (so you can use it in a lambda or comprehension) then you can use this little hack trick: create a tuple with the dictionary and the popped element, and then get the original item back out of the tuple:
(foo, foo.pop(x))[0]
For example:
ds = [{'a': 1, 'b': 2, 'c': 3}, {'a': 4, 'b': 5, 'c': 6}]
[(d, d.pop('c'))[0] for d in ds]
assert ds == [{'a': 1, 'b': 2}, {'a': 4, 'b': 5}]
Note that this actually modifies the original dictionary, so despite being a comprehension, it's not purely functional.
When you invoke pop the original dictionary is modified in place.
You can return that one from your function.
>>> a = {'foo': 1, 'bar': 2}
>>> a.pop('foo')
1
>>> a
{'bar': 2}
solution from me
item = dict({"A": 1, "B": 3, "C": 4})
print(item)
{'A': 1, 'B': 3, 'C': 4}
new_dict = (lambda d: d.pop('C') and d)(item)
print(new_dict)
{'A': 1, 'B': 3}
this will work,
(lambda dict_,key_:dict_.pop(key_,True) and dict_)({1:1},1)
EDIT
this will drop the key if exist in the dictionary and will return the dictionary without the key,value pair
in python there are functions that alter an object in place, and returns a value instead of the altered object, {}.pop function is an example.
we can use a lambda function as in the example, or more generic below
(lambda func:obj:(func(obj) and False) or obj)
to alter this behavior, and get a the expected behavior.
I have a dictionary as:
default = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
I wish to eliminate the empty values as:
default = {'a': ['alpha'], 'b': ['beta','gamma']}
I wrote a function (following an example found on the web)
def remove_empty_keys(d):
for k in d.keys():
try:
if len(d[k]) < 1:
del[k]
except:
pass
return(d)
I have the following questions:
1- I didn't find the mistake why it always returns following -
remove_empty_keys(default)
{'a': ['alpha'], 'b': ['beta'], 'g': []}
2- Is there a built-in function to eliminate/delete Null/None/empty values from Python dictionary without creating a copy of the original dictionary?
There's no builtin for this (AFAIK), but you can do it easily with a dict comprehension:
new_dict = {k:v for k,v in original_dict.items() if v}
If you're stuck with an older version of python (pre 2.7 without dict comprehensions), you can use the dict constructor:
new_dict = dict((k,v) for k,v in original_dict.items() if v)
Note that this doesn't operate in place (as per your second question). And dictionaries don't support slice assignment like lists do, so the best* you can really do to get this all done in place is:
new_dict = {k:v for k,v in original_dict.items() if v}
original_dict.clear()
original_dict.update(new_dict)
*of course the term "best" is completely subjective.
To fix your function, change del[k] to del d[k]. There is no function to delete values in place from a dictionary.
What you are doing is deleting the variable k, not changing the dictionary at all. This is why the original dictionary is always returned.
Rewritten, your function might look like:
def remove_empty_keys(d):
for k in d.keys():
if not d[k]:
del d[k]
This assumes you want to eliminate both empty list and None values, and actually removes any item with a "false" value.
You can use dict comprehension: -
>>> default = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
>>> {key: value for key, value in default.iteritems() if value}
{'a': ['alpha'], 'b': ['beta', 'gamma']}
dict((k, v) for k, v in default.iteritems() if v)
This filters all items which are not empty strings, empty dict/tuple/list.
One more option is the following (without creating a new dict):
for e in [k for k,v in default.iteritems() if len(v) == 0]: default.pop(e)
Michael's answer is correct.
Stepping back, you might be able to avoid creating those empty lists at all, by use of collections.defaultdict(list)
>>> import collections
>>> d = collections.defaultdict(list)
>>> d
defaultdict(<type 'list'>, {})
>>> d["hobbits"].append("Frodo")
>>> d["hobbits"].append("Sam")
>>> d
defaultdict(<type 'list'>, {'hobbits': ['Frodo', 'Sam']})
If the values are all lists, so you can use their truth's, you could use itertools.compress:
from itertools import compress
d = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
new_d = dict(compress(d.items(), d.values()))
print(new_d) # {'a': ['alpha'], 'b': ['beta', 'gamma']}
How can I turn a list of dicts like [{'a':1}, {'b':2}, {'c':1}, {'d':2}], into a single dict like {'a':1, 'b':2, 'c':1, 'd':2}?
Answers here will overwrite keys that match between two of the input dicts, because a dict cannot have duplicate keys. If you want to collect multiple values from matching keys, see How to merge dicts, collecting values from matching keys?.
This works for dictionaries of any length:
>>> result = {}
>>> for d in L:
... result.update(d)
...
>>> result
{'a':1,'c':1,'b':2,'d':2}
As a comprehension:
# Python >= 2.7
{k: v for d in L for k, v in d.items()}
# Python < 2.7
dict(pair for d in L for pair in d.items())
In case of Python 3.3+, there is a ChainMap collection:
>>> from collections import ChainMap
>>> a = [{'a':1},{'b':2},{'c':1},{'d':2}]
>>> dict(ChainMap(*a))
{'b': 2, 'c': 1, 'a': 1, 'd': 2}
Also see:
What is the purpose of collections.ChainMap?
Little improvement for #dietbuddha answer with dictionary unpacking from PEP 448, for me, it`s more readable this way, also, it is faster as well:
from functools import reduce
result_dict = reduce(lambda a, b: {**a, **b}, list_of_dicts)
But keep in mind, this works only with Python 3.5+ versions.
This is similar to #delnan but offers the option to modify the k/v (key/value) items and I believe is more readable:
new_dict = {k:v for list_item in list_of_dicts for (k,v) in list_item.items()}
for instance, replace k/v elems as follows:
new_dict = {str(k).replace(" ","_"):v for list_item in list_of_dicts for (k,v) in list_item.items()}
unpacks the k,v tuple from the dictionary .items() generator after pulling the dict object out of the list
For flat dictionaries you can do this:
from functools import reduce
reduce(lambda a, b: dict(a, **b), list_of_dicts)
You can use join function from funcy library:
from funcy import join
join(list_of_dicts)
>>> L=[{'a': 1}, {'b': 2}, {'c': 1}, {'d': 2}]
>>> dict(i.items()[0] for i in L)
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
Note: the order of 'b' and 'c' doesn't match your output because dicts are unordered
if the dicts can have more than one key/value
>>> dict(j for i in L for j in i.items())
If you don't need the singleton dicts anymore:
>>> L = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
>>> dict(map(dict.popitem, L))
{'a': 1, 'b': 2, 'c': 1, 'd': 2}
dict1.update( dict2 )
This is asymmetrical because you need to choose what to do with duplicate keys; in this case, dict2 will overwrite dict1. Exchange them for the other way.
EDIT: Ah, sorry, didn't see that.
It is possible to do this in a single expression:
>>> from itertools import chain
>>> dict( chain( *map( dict.items, theDicts ) ) )
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
No credit to me for this last!
However, I'd argue that it might be more Pythonic (explicit > implicit, flat > nested ) to do this with a simple for loop. YMMV.
this way worked for me:
object = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
object = {k: v for dct in object for k, v in dct.items()}
printing object:
object = {'a':1,'b':2,'c':1,'d':2}
thanks Axes
>>> dictlist = [{'a':1},{'b':2},{'c':1},{'d':2, 'e':3}]
>>> dict(kv for d in dictlist for kv in d.iteritems())
{'a': 1, 'c': 1, 'b': 2, 'e': 3, 'd': 2}
>>>
Note I added a second key/value pair to the last dictionary to show it works with multiple entries.
Also keys from dicts later in the list will overwrite the same key from an earlier dict.