What is a good 1-liner (other than putting this into a function) to achieve this:
# like dict1.update(dict2) but does not add any keys from dict2
# that are not already in dict1
for k in dict1:
if k in dict2:
dict1[k]=dict2[k]
I guess that this would work, but uses a "protected" function:
[dict1.__setitem__(k, dict2[k]) for k in dict1 if k in dict2]
dict1.update((k, dict2[k]) for k in set(dict2).intersection(dict1))
is how I'd do it in Python 2.6 or below (see further on how to do this in later versions).
Next to another mapping, dict.update() can also take an iterable of (key, value) tuples, which we generate based on the set intersection of the two dictionaries (so all keys they have in common).
Demo:
>>> dict1 = {'foo':'bar', 'ham': 'eggs'}
>>> dict2 = {'ham': 'spam', 'bar': 'baz'}
>>> dict1.update((k, dict2[k]) for k in set(dict2).intersection(dict1))
>>> dict1
{'foo': 'bar', 'ham': 'spam'}
In python 2.7 you can use the new Dict views to achieve the same without casting to sets:
dict1.update((k, dict2[k]) for k in dict1.viewkeys() & dict2.viewkeys())
In Python 3, dict views are the default, so you can instead spell this as:
dict1.update((k, dict2[k]) for k in dict1.keys() & dict2.keys())
Non-destructively:
dict((k, dict2.get(k, v)) for k, v in dict1.items())
Modifying dict1:
dict1.update((k, v) for k, v in dict2.items() if k in dict1)
Using map/zip
As two lines for readability:
match_keys = dict1.keys() & dict2.keys()
dict2.update(**dict(zip(match_keys, map(dict2.get, match_keys))))
Or as a one-liner:
dict2.update(**dict(zip(dict1.keys() & dict2.keys(), map(dict2.get, dict1.keys() & dict2.keys()))))
non-destrctively:
new_dict = {**dict2, **dict(zip(dict1.keys() & dict2.keys(), map(dict2.get, dict1.keys() & dict2.keys())))}
Related
encoder.load_state_dict({k:v for k,v in encoder_dict.items() if k in model_dict})
This syntax is an operation of the dictionary, but I can't understand that what does "k:v" acts?
encoder.load_state_dict({k:v for k,v in encoder_dict.items() if k in model_dict})
This is dictionary comprehension. An equivalent code in simple terms would be:
new_dict = dict()
for k,v in encoder_dict.items():
if k in model_dict:
new_dict[k] = v
encoder.load_state_dict(new_dict)
where k and v corresponds to the key and value pair returned by encoder_dict.items()
Say I have a nested dict that I want to flatten. The dict is only one layer deep and the nested elements are either dict or str. So given the example data below:
data = {
'foo': 'bar',
'sample': {
'a': 213,
'b': 634298,
'c': 1},
'doo': 'val',
'spaz': {
'x': 4,
'y': 32,
'z': 18}}
Desired output is a flattened dict as follows:
{'foo': 'bar',
'sample_a': 213,
'sample_b': 634298,
'sample_c': 1,
'doo': 'val',
'spaz_x': 4,
'spaz_y': 32,
'spaz_z': 18}
The traditional way I'd do this is a loop such as this:
data_out = {}
for k, v in data.items():
if isinstance(v, dict):
for ik, iv in v.items():
data_out[f'{k}_{ik}'] = iv
else:
data_out[k] = v
Although I'm looking to see how I can do this cleanly with a comprehension. My attempt uses a conditional nested loop along with itertools.chain.from_iterable
data_out = dict(chain.from_iterable(((f'{k}_{ik}', iv) for ik, iv in v.items()) if isinstance(v, dict) else ((k, v),) for k, v in data.items()))
Below is the same thing as above only broken down by line for readability:
data_out = dict(chain.from_iterable(
((f'{k}_{ik}', iv) for ik, iv in v.items())
if isinstance(v, dict)
else ((k, v),)
for k, v in data.items()))
While this works I'm convinced this more than needed, I just cannot think of a better way preferably as a traditional dict comprehension and not using dict on a generator comprehension. Only way I can really shorten the code that I can think of is removing .from_iterable and just star unpacking the generator although I don't think this is more optimal for this approach.
You could invert the order of the loops in your dict comprehension and thus create the "flat" dictionary directly, without chain.from_iterable, but that does not really make it any clearer.
>>> {k: v for (k1, v1) in data.items()
... for (k, v) in (((f'{k1}_{k2}', v2) for (k2, v2) in v1.items())
... if isinstance(v1, dict) else ((k1, v1),))}
...
{'doo': 'val',
'foo': 'bar',
'sample_a': 213,
'sample_b': 634298,
'sample_c': 1,
'spaz_x': 4,
'spaz_y': 32,
'spaz_z': 18}
Another way might be to create two dicts and then **-combine them to the final dict:
>>> {**{ k1: v1 for (k1, v1) in data.items() if isinstance(v1, str)},
... **{f'{k1}_{k2}': v2 for (k1, v1) in data.items() if isinstance(v1, dict)
... for (k2, v2) in v1.items()}}
However, IMHO your "traditional" nested loop is much cleaner than either of those three versions. Alternatively, just use any of the other recursive "flatten dictionary with arbitrary depth" functions found here.
I actually figured out a way I can do this using getattr as a shortcut to checking if its a dict or not. To me this is more efficient than other ways I’ve seen:
data_out = {'_'.join(filter((0).__ne__, [pk, sk])): v
for pk, v in data.items()
for sk, v in getattr(v, 'items', {0: v}.items)()}
Legend:
pk - parent_key
sk - suffix_key or sub_key
And if you may have 0's in your keys I just used a "trasher" object:
fill = object(); nfill = fill.__ne__
data_out = {'_'.join(filter(nfill, [pk, sk])): v
for pk, v in data.items()
for sk, v in getattr(v, 'items', {fill:v}.items)()}
So, I have 2 dictionaries, I have to check for missing keys and for matching keys, check if they have same or different values.
dict1 = {..}
dict2 = {..}
#key values in a list that are missing in each
missing_in_dict1_but_in_dict2 = []
missing_in_dict2_but_in_dict1 = []
#key values in a list that are mismatched between the 2 dictionaries
mismatch = []
What's the most efficient way to do this?
You can use dictionary view objects, which act as sets. Subtract sets to get the difference:
missing_in_dict1_but_in_dict2 = dict2.keys() - dict1
missing_in_dict2_but_in_dict1 = dict1.keys() - dict2
For the keys that are the same, use the intersection, with the & operator:
mismatch = {key for key in dict1.keys() & dict2 if dict1[key] != dict2[key]}
If you are still using Python 2, use dict.viewkeys().
Using dictionary views to produce intersections and differences is very efficient, the view objects themselves are very lightweight the algorithms to create the new sets from the set operations can make direct use of the O(1) lookup behaviour of the underlying dictionaries.
Demo:
>>> dict1 = {'foo': 42, 'bar': 81}
>>> dict2 = {'bar': 117, 'spam': 'ham'}
>>> dict2.keys() - dict1
{'spam'}
>>> dict1.keys() - dict2
{'foo'}
>>> [key for key in dict1.keys() & dict2 if dict1[key] != dict2[key]]
{'bar'}
and a performance comparison with creating separate set() objects:
>>> import timeit
>>> import random
>>> def difference_views(d1, d2):
... missing1 = d2.keys() - d1
... missing2 = d1.keys() - d2
... mismatch = {k for k in d1.keys() & d2 if d1[k] != d2[k]}
... return missing1, missing2, mismatch
...
>>> def difference_sets(d1, d2):
... missing1 = set(d2) - set(d1)
... missing2 = set(d1) - set(d2)
... mismatch = {k for k in set(d1) & set(d2) if d1[k] != d2[k]}
... return missing1, missing2, mismatch
...
>>> testd1 = {random.randrange(1000000): random.randrange(1000000) for _ in range(10000)}
>>> testd2 = {random.randrange(1000000): random.randrange(1000000) for _ in range(10000)}
>>> timeit.timeit('d(d1, d2)', 'from __main__ import testd1 as d1, testd2 as d2, difference_views as d', number=1000)
1.8643521590274759
>>> timeit.timeit('d(d1, d2)', 'from __main__ import testd1 as d1, testd2 as d2, difference_sets as d', number=1000)
2.811345119960606
Using set() objects is slower, especially when your input dictionaries get larger.
One easy way is to create sets from the dict keys and subtract them:
>>> dict1 = { 'a': 1, 'b': 1 }
>>> dict2 = { 'b': 1, 'c': 1 }
>>> missing_in_dict1_but_in_dict2 = set(dict2) - set(dict1)
>>> missing_in_dict1_but_in_dict2
set(['c'])
>>> missing_in_dict2_but_in_dict1 = set(dict1) - set(dict2)
>>> missing_in_dict2_but_in_dict1
set(['a'])
Or you can avoid casting the second dict to a set by using .difference():
>>> set(dict1).difference(dict2)
set(['a'])
>>> set(dict2).difference(dict1)
set(['c'])
I have,
dict1={a:1, b:2, c:3}
dict2={a:3, c:7}
I want to find out what keys I have in dict1 that I don't have in dict2. So I do
diff_as_set = set(dict1.keys()) - set (dict2.keys())
This gives me: b
However, I want a dictionary which contains all the key value mappings from dict1 for all keys that are not in dict 2 so I then do:
diff_as_dict = {k:v for k,v in dict1 if k in diff_as_set}
I get:
diff_as_dict = {k:v for k, v in dict1 if k in diff_as_set}
ValueError: too many values to unpack (expected 2)
Any ideas?
Looping over a dict only provides keys, you need to use:
diff_as_dict = {k:v for k, v in dict1.iteritems() if k in diff_as_set}
^^^^^^^^^^^
Or use .items() for Python 3.x
Instead of going through the entire dict to pick out the ones that match your set, just iterate the set.
diff_as_dict = {k:dict1[k] for k in diff_as_set}
Example:
>>> dict1={'a':1, 'b':2, 'c':3}
>>> dict2={'a':3, 'c':7}
>>> diff_as_set = set(dict1.keys()) - set (dict2.keys())
>>> diff_as_set
set(['b'])
>>> diff_as_dict = {k:dict1[k] for k in diff_as_set}
>>> diff_as_dict
{'b': 2}
You're missing the .iteritems() part:
dict1 = {'a':1, 'b':2, 'c':3}
dict2 = {'a':3, 'c':7}
newdict = {k : v for k,v in dict1.iteritems() if not(k in dict2)}
After this, newdict is equal to {'b': 2}. This does everything in one go.
In python... I have a list of elements 'my_list', and a dictionary 'my_dict' where some keys match in 'my_list'.
I would like to search the dictionary and retrieve key/value pairs for the keys matching the 'my_list' elements.
I tried this...
if any(x in my_dict for x in my_list):
print set(my_list)&set(my_dict)
But it doesn't do the job.
(I renamed list to my_list and dict to my_dict to avoid the conflict with the type names.)
For better performance, you should iterate over the list and check for membership in the dictionary:
for k in my_list:
if k in my_dict:
print(k, my_dict[k])
If you want to create a new dictionary from these key-value pairs, use
new_dict = {k: my_dict[k] for k in my_list if k in my_dict}
Don't use dict and list as variable names. They shadow the built-in functions. Assuming list l and dictionary d:
kv = [(k, d[k]) for k in l if k in d]
new_dict = dict((k, v) for k, v in dict.iteritems() if k in list)
Turning list into a set set(list) may yield a noticeable speed increase
Try This:
mydict = {'one': 1, 'two': 2, 'three': 3}
mykeys = ['three', 'one','ten']
newList={k:mydict[k] for k in mykeys if k in mydict}
print newList
{'three': 3, 'one': 1}
What about print([kv for kv in dict.items() if kv[0] in list])
Here is a one line solution for that
{i:my_dict[i] for i in set(my_dict.keys()).intersection(set(my_list))}