Can I delete items from a dictionary in Python while iterating over it?
I want to remove elements that don't meet a certain condition from the dictionary, instead of creating an entirely new dictionary. Is the following a good solution, or are there better ways?
for k, v in mydict.items():
if k == val:
del mydict[k]
For Python 3+:
>>> mydict
{'four': 4, 'three': 3, 'one': 1}
>>> for k in list(mydict.keys()):
... if mydict[k] == 3:
... del mydict[k]
>>> mydict
{'four': 4, 'one': 1}
The other answers work fine with Python 2 but raise a RuntimeError for Python 3:
RuntimeError: dictionary changed size during iteration.
This happens because mydict.keys() returns an iterator not a list.
As pointed out in comments simply convert mydict.keys() to a list by list(mydict.keys()) and it should work.
For Python 2:
A simple test in the console shows you cannot modify a dictionary while iterating over it:
>>> mydict = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k, v in mydict.iteritems():
... if k == 'two':
... del mydict[k]
------------------------------------------------------------
Traceback (most recent call last):
File "<ipython console>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
As stated in delnan's answer, deleting entries causes problems when the iterator tries to move onto the next entry. Instead, use the keys() method to get a list of the keys and work with that:
>>> for k in mydict.keys():
... if k == 'two':
... del mydict[k]
>>> mydict
{'four': 4, 'three': 3, 'one': 1}
If you need to delete based on the items value, use the items() method instead:
>>> for k, v in mydict.items():
... if v == 3:
... del mydict[k]
>>> mydict
{'four': 4, 'one': 1}
You could also do it in two steps:
remove = [k for k in mydict if k == val]
for k in remove: del mydict[k]
My favorite approach is usually to just make a new dict:
# Python 2.7 and 3.x
mydict = { k:v for k,v in mydict.items() if k!=val }
# before Python 2.7
mydict = dict((k,v) for k,v in mydict.iteritems() if k!=val)
Iterate over a copy instead, such as the one returned by items():
for k, v in list(mydict.items()):
You can't modify a collection while iterating it. That way lies madness - most notably, if you were allowed to delete and deleted the current item, the iterator would have to move on (+1) and the next call to next would take you beyond that (+2), so you'd end up skipping one element (the one right behind the one you deleted). You have two options:
Copy all keys (or values, or both, depending on what you need), then iterate over those. You can use .keys() et al for this (in Python 3, pass the resulting iterator to list). Could be highly wasteful space-wise though.
Iterate over mydict as usual, saving the keys to delete in a seperate collection to_delete. When you're done iterating mydict, delete all items in to_delete from mydict. Saves some (depending on how many keys are deleted and how many stay) space over the first approach, but also requires a few more lines.
It's cleanest to use list(mydict):
>>> mydict = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k in list(mydict):
... if k == 'three':
... del mydict[k]
...
>>> mydict
{'four': 4, 'two': 2, 'one': 1}
This corresponds to a parallel structure for lists:
>>> mylist = ['one', 'two', 'three', 'four']
>>> for k in list(mylist): # or mylist[:]
... if k == 'three':
... mylist.remove(k)
...
>>> mylist
['one', 'two', 'four']
Both work in python2 and python3.
With python3, iterate on dic.keys() will raise the dictionary size error. You can use this alternative way:
Tested with python3, it works fine and the Error "dictionary changed size during iteration" is not raised:
my_dic = { 1:10, 2:20, 3:30 }
# Is important here to cast because ".keys()" method returns a dict_keys object.
key_list = list( my_dic.keys() )
# Iterate on the list:
for k in key_list:
print(key_list)
print(my_dic)
del( my_dic[k] )
print( my_dic )
# {}
You can use a dictionary comprehension.
d = {k:d[k] for k in d if d[k] != val}
You could first build a list of keys to delete, and then iterate over that list deleting them.
dict = {'one' : 1, 'two' : 2, 'three' : 3, 'four' : 4}
delete = []
for k,v in dict.items():
if v%2 == 1:
delete.append(k)
for i in delete:
del dict[i]
There is a way that may be suitable if the items you want to delete are always at the "beginning" of the dict iteration
while mydict:
key, value = next(iter(mydict.items()))
if should_delete(key, value):
del mydict[key]
else:
break
The "beginning" is only guaranteed to be consistent for certain Python versions/implementations. For example from What’s New In Python 3.7
the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec.
This way avoids a copy of the dict that a lot of the other answers suggest, at least in Python 3.
I tried the above solutions in Python3 but this one seems to be the only one working for me when storing objects in a dict. Basically you make a copy of your dict() and iterate over that while deleting the entries in your original dictionary.
tmpDict = realDict.copy()
for key, value in tmpDict.items():
if value:
del(realDict[key])
One-liner:
my_dict = {k: v for k, v in my_dict.copy().items() if not k == value}
The my_dict.copy() object is used for the iteration only and will not be available outside the scope of the dictionary comprehension. This avoids editing the object over which you are currently iterating, as advised against by #user395760 in their answer.
You can split this over multiple lines for clarity:
my_dict = {
k: v
for k, v in my_dict.copy().items()
if not k == value
}
Related
I want to filter a dictionary based on a set/list/(dict) of keys.
Currently I'm using a generator like this:
def filter_dict(in_dict, in_iterator):
for key, value in in_dict.items():
if key in in_iterator:
yield key, value
d = {'one': 1, 'two': 2, 'three': 3}
l = ['one', 'two']
for key, value in filter_dict(d, l):
print(key, value)
Which works great and filters correctly with the result:
one 1
two 2
Is there a better or more standardized way to do this?
Maybe something like filter(d, l) or d.items(l)
Instead of iterating through all the keys and then checking if it is present in another list, you could simply walk through the list and pick items from the dict and create a new dict.
new_dict = dict( ((key, d[key]) for key in l) )
if items of your list not in your dictionaries you get error if you use this:
it takes about 5 seconds to complete it
new_dict = dict(((key, d[key]) for key in l))
you can use something like this:
and you can be sure you won't get the error if items of your list not in your dictionaries
keys = set(l).intersection(d)
result = {key:d[key] for key in keys}
but it takes about 17 seconds, and it doesn't good performance
you can use this and it takes about 5 seconds to complete it and you can be sure you won't get any error:
result = {}
for k in set(d1).intersection(l1):
result[k]= d1[k]
One possibility is to perform set operation on dict.keys():
d = {'one': 1, 'two': 2, 'three': 3}
l = ['one', 'two']
for k in d.keys() & set(l):
print(k, d[k])
Prints:
one 1
two 2
def invert_dict(d):
inv = dict()
for key in d:
val = d[key]
if val not in inv:
inv[val] = [key]
else:
inv[val].append(key)
return inv
This is an example from Think Python book, a function for inverting(swapping) keys and values in a dictionary. New values (former keys) are stored as lists, so if there was multiple dictionary values (bound to a different keys) that were equal before inverting, then this function simply appends them to the list of former keys.
Example:
somedict = {'one': 1, 'two': 2, 'doubletwo': 2, 'three': 3}
invert_dict(somedict) ---> {1: ['one'], 2: ['doubletwo', 'two'], 3: ['three']}
My question is, can the same be done with dictionary comprehensions? This function creates an empty dict inv = dict(), which is then checked later in the function with if/else for the presence of values. Dict comprehension, in this case, should check itself. Is that possible, and how the syntax should look like?
General dict comprehension syntax for swapping values is:
{value:key for key, value in somedict.items()}
but if I want to add an 'if' clausule, what it should look like? if value not in (what)?
Thanks.
I don't think it's possible with simple dict comprehension without using other functions.
Following code uses itertools.groupby to group keys that have same values.
>>> import itertools
>>> {k: [x[1] for x in grp]
for k, grp in itertools.groupby(
sorted((v,k) for k, v in somedict.iteritems()),
key=lambda x: x[0])
}
{1: ['one'], 2: ['doubletwo', 'two'], 3: ['three']}
You can use a set comprehension side effect:
somedict = {'one': 1, 'two': 2, 'doubletwo': 2, 'three': 3}
invert_dict={}
{invert_dict.setdefault(v, []).append(k) for k, v in somedict.items()}
print invert_dict
# {1: ['one'], 2: ['doubletwo', 'two'], 3: ['three']}
Here is a good answer:
fts = {1:1,2:1,3:2,4:1}
new_dict = {dest: [k for k, v in fts.items() if v == dest] for dest in set(fts.values())}
Reference: Head First Python ,2nd Edition, Page(502)
I have a dictionary as:
default = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
I wish to eliminate the empty values as:
default = {'a': ['alpha'], 'b': ['beta','gamma']}
I wrote a function (following an example found on the web)
def remove_empty_keys(d):
for k in d.keys():
try:
if len(d[k]) < 1:
del[k]
except:
pass
return(d)
I have the following questions:
1- I didn't find the mistake why it always returns following -
remove_empty_keys(default)
{'a': ['alpha'], 'b': ['beta'], 'g': []}
2- Is there a built-in function to eliminate/delete Null/None/empty values from Python dictionary without creating a copy of the original dictionary?
There's no builtin for this (AFAIK), but you can do it easily with a dict comprehension:
new_dict = {k:v for k,v in original_dict.items() if v}
If you're stuck with an older version of python (pre 2.7 without dict comprehensions), you can use the dict constructor:
new_dict = dict((k,v) for k,v in original_dict.items() if v)
Note that this doesn't operate in place (as per your second question). And dictionaries don't support slice assignment like lists do, so the best* you can really do to get this all done in place is:
new_dict = {k:v for k,v in original_dict.items() if v}
original_dict.clear()
original_dict.update(new_dict)
*of course the term "best" is completely subjective.
To fix your function, change del[k] to del d[k]. There is no function to delete values in place from a dictionary.
What you are doing is deleting the variable k, not changing the dictionary at all. This is why the original dictionary is always returned.
Rewritten, your function might look like:
def remove_empty_keys(d):
for k in d.keys():
if not d[k]:
del d[k]
This assumes you want to eliminate both empty list and None values, and actually removes any item with a "false" value.
You can use dict comprehension: -
>>> default = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
>>> {key: value for key, value in default.iteritems() if value}
{'a': ['alpha'], 'b': ['beta', 'gamma']}
dict((k, v) for k, v in default.iteritems() if v)
This filters all items which are not empty strings, empty dict/tuple/list.
One more option is the following (without creating a new dict):
for e in [k for k,v in default.iteritems() if len(v) == 0]: default.pop(e)
Michael's answer is correct.
Stepping back, you might be able to avoid creating those empty lists at all, by use of collections.defaultdict(list)
>>> import collections
>>> d = collections.defaultdict(list)
>>> d
defaultdict(<type 'list'>, {})
>>> d["hobbits"].append("Frodo")
>>> d["hobbits"].append("Sam")
>>> d
defaultdict(<type 'list'>, {'hobbits': ['Frodo', 'Sam']})
If the values are all lists, so you can use their truth's, you could use itertools.compress:
from itertools import compress
d = {'a': ['alpha'], 'b': ['beta','gamma'], 'g': []}
new_d = dict(compress(d.items(), d.values()))
print(new_d) # {'a': ['alpha'], 'b': ['beta', 'gamma']}
In python... I have a list of elements 'my_list', and a dictionary 'my_dict' where some keys match in 'my_list'.
I would like to search the dictionary and retrieve key/value pairs for the keys matching the 'my_list' elements.
I tried this...
if any(x in my_dict for x in my_list):
print set(my_list)&set(my_dict)
But it doesn't do the job.
(I renamed list to my_list and dict to my_dict to avoid the conflict with the type names.)
For better performance, you should iterate over the list and check for membership in the dictionary:
for k in my_list:
if k in my_dict:
print(k, my_dict[k])
If you want to create a new dictionary from these key-value pairs, use
new_dict = {k: my_dict[k] for k in my_list if k in my_dict}
Don't use dict and list as variable names. They shadow the built-in functions. Assuming list l and dictionary d:
kv = [(k, d[k]) for k in l if k in d]
new_dict = dict((k, v) for k, v in dict.iteritems() if k in list)
Turning list into a set set(list) may yield a noticeable speed increase
Try This:
mydict = {'one': 1, 'two': 2, 'three': 3}
mykeys = ['three', 'one','ten']
newList={k:mydict[k] for k in mykeys if k in mydict}
print newList
{'three': 3, 'one': 1}
What about print([kv for kv in dict.items() if kv[0] in list])
Here is a one line solution for that
{i:my_dict[i] for i in set(my_dict.keys()).intersection(set(my_list))}
This question already has answers here:
Reverse / invert a dictionary mapping
(32 answers)
Closed 10 months ago.
I receive a dictionary as input, and would like to to return a dictionary whose keys will be the input's values and whose value will be the corresponding input keys. Values are unique.
For example, say my input is:
a = dict()
a['one']=1
a['two']=2
I would like my output to be:
{1: 'one', 2: 'two'}
To clarify I would like my result to be the equivalent of the following:
res = dict()
res[1] = 'one'
res[2] = 'two'
Any neat Pythonic way to achieve this?
Python 2:
res = dict((v,k) for k,v in a.iteritems())
Python 3 (thanks to #erik):
res = dict((v,k) for k,v in a.items())
new_dict = dict(zip(my_dict.values(), my_dict.keys()))
From Python 2.7 on, including 3.0+, there's an arguably shorter, more readable version:
>>> my_dict = {'x':1, 'y':2, 'z':3}
>>> {v: k for k, v in my_dict.items()}
{1: 'x', 2: 'y', 3: 'z'}
You can make use of dict comprehensions:
Python 3
res = {v: k for k, v in a.items()}
Python 2
res = {v: k for k, v in a.iteritems()}
Edited: For Python 3, use a.items() instead of a.iteritems(). Discussions about the differences between them can be found in iteritems in Python on SO.
In [1]: my_dict = {'x':1, 'y':2, 'z':3}
Python 3
In [2]: dict((value, key) for key, value in my_dict.items())
Out[2]: {1: 'x', 2: 'y', 3: 'z'}
Python 2
In [2]: dict((value, key) for key, value in my_dict.iteritems())
Out[2]: {1: 'x', 2: 'y', 3: 'z'}
The current leading answer assumes values are unique which is not always the case. What if values are not unique? You will loose information!
For example:
d = {'a':3, 'b': 2, 'c': 2}
{v:k for k,v in d.iteritems()}
returns {2: 'b', 3: 'a'}.
The information about 'c' was completely ignored.
Ideally it should had be something like {2: ['b','c'], 3: ['a']}. This is what the bottom implementation does.
Python 2.x
def reverse_non_unique_mapping(d):
dinv = {}
for k, v in d.iteritems():
if v in dinv:
dinv[v].append(k)
else:
dinv[v] = [k]
return dinv
Python 3.x
def reverse_non_unique_mapping(d):
dinv = {}
for k, v in d.items():
if v in dinv:
dinv[v].append(k)
else:
dinv[v] = [k]
return dinv
You could try:
Python 3
d={'one':1,'two':2}
d2=dict((value,key) for key,value in d.items())
d2
{'two': 2, 'one': 1}
Python 2
d={'one':1,'two':2}
d2=dict((value,key) for key,value in d.iteritems())
d2
{'two': 2, 'one': 1}
Beware that you cannot 'reverse' a dictionary if
More than one key shares the same value. For example {'one':1,'two':1}. The new dictionary can only have one item with key 1.
One or more of the values is unhashable. For example {'one':[1]}. [1] is a valid value but not a valid key.
See this thread on the python mailing list for a discussion on the subject.
res = dict(zip(a.values(), a.keys()))
new_dict = dict( (my_dict[k], k) for k in my_dict)
or even better, but only works in Python 3:
new_dict = { my_dict[k]: k for k in my_dict}
Another way to expand on Ilya Prokin's response is to actually use the reversed function.
dict(map(reversed, my_dict.items()))
In essence, your dictionary is iterated through (using .items()) where each item is a key/value pair, and those items are swapped with the reversed function. When this is passed to the dict constructor, it turns them into value/key pairs which is what you want.
Suggestion for an improvement for Javier answer :
dict(zip(d.values(),d))
Instead of d.keys() you can write just d, because if you go through dictionary with an iterator, it will return the keys of the relevant dictionary.
Ex. for this behavior :
d = {'a':1,'b':2}
for k in d:
k
'a'
'b'
Can be done easily with dictionary comprehension:
{d[i]:i for i in d}
dict(map(lambda x: x[::-1], YourDict.items()))
.items() returns a list of tuples of (key, value). map() goes through elements of the list and applies lambda x:[::-1] to each its element (tuple) to reverse it, so each tuple becomes (value, key) in the new list spitted out of map. Finally, dict() makes a dict from the new list.
Hanan's answer is the correct one as it covers more general case (the other answers are kind of misleading for someone unaware of the duplicate situation). An improvement to Hanan's answer is using setdefault:
mydict = {1:a, 2:a, 3:b}
result = {}
for i in mydict:
result.setdefault(mydict[i],[]).append(i)
print(result)
>>> result = {a:[1,2], b:[3]}
Using loop:-
newdict = {} #Will contain reversed key:value pairs.
for key, value in zip(my_dict.keys(), my_dict.values()):
# Operations on key/value can also be performed.
newdict[value] = key
If you're using Python3, it's slightly different:
res = dict((v,k) for k,v in a.items())
Adding an in-place solution:
>>> d = {1: 'one', 2: 'two', 3: 'three', 4: 'four'}
>>> for k in list(d.keys()):
... d[d.pop(k)] = k
...
>>> d
{'two': 2, 'one': 1, 'four': 4, 'three': 3}
In Python3, it is critical that you use list(d.keys()) because dict.keys returns a view of the keys. If you are using Python2, d.keys() is enough.
I find this version the most comprehensive one:
a = {1: 'one', 2: 'two'}
swapped_a = {value : key for key, value in a.items()}
print(swapped_a)
output :
{'one': 1, 'two': 2}
An alternative that is not quite as readable (in my opinion) as some of the other answers:
new_dict = dict(zip(*list(zip(*old_dict.items()))[::-1]))
where list(zip(*old_dict.items()))[::-1] gives a list of 2 tuples, old_dict's values and keys, respectively.