Related
I tried different approaces with itertools, but just can't figure it out.
I need to find different combinations of dictionaries:
letters = ['a','b','c']
combinations = []
for i in range(3):
for t in letters:
one_combi = {str(t):i}
combinations.append(one_combi)
Now have a list of dictionaries {letter:number}
Now I need to create a list of combinations where the key (letter) only appear once.
Expected output looks something like this:
[{'a':0,'b':0,'c':0},
{'a':1,'b':0,'c':0},
{'a':1,'b':1,'c':0},
{'a':1,'b':1,'c':1},
{'a':2,'b':0,'c':0},
...
{'a':2,'b':2,'c':2}]
Would be great if someone can help me out on this one!
You can generate all combinations of integers from a range derived from the length of the input, and then use zip:
letters = ['a','b','c']
def combos(d, c = []):
if len(c) == len(d):
yield dict(zip(letters, c))
else:
for i in d:
yield from combos(d, c+[i])
print(list(combos(range(len(letters))))
Output:
[{'a': 0, 'b': 0, 'c': 0},
{'a': 0, 'b': 0, 'c': 1},
{'a': 0, 'b': 0, 'c': 2},
{'a': 0, 'b': 1, 'c': 0},
{'a': 0, 'b': 1, 'c': 1},
...
{'a': 2, 'b': 2, 'c': 2}]
What you are looking for is itertools.product
from itertools import product
lst = []
for a, b, c in product([0, 1, 2], repeat=3):
lst.append({'a': a, 'b': b, 'c': c})
print(lst)
Output:
[{'a': 0, 'b': 0, 'c': 0},
{'a': 0, 'b': 0, 'c': 1},
{'a': 0, 'b': 0, 'c': 2},
{'a': 0, 'b':1, 'c': 0},
{'a': 0, 'b': 1, 'c': 1},
{'a': 0, 'b': 1, 'c': 2},
{'a': 0, 'b': 2, 'c': 0},
{'a': 0, 'b': 2, 'c': 1},...
Update
We can compact everything into a single line using list comprehension.
letters = ['a','b','c']
lst = [dict(zip(letters, x)) for x in product(range(len(letters)), repeat=len(letters))]
print(lst)
I have a dict looks like this :
my_dict = {
"a":[1, 2, 3],
"b":[10],
"c":[4, 5],
"d":[11]
}
And I would like to obtain a list containig all combinations keeping keys and value like this:
result = [
{"a":1, "b":10, "c":4, "d":11},
{"a":1, "b":10, "c":5, "d":11},
{"a":2, "b":10, "c":4, "d":11},
{"a":2, "b":10, "c":5, "d":11},
{"a":3, "b":10, "c":4, "d":11},
{"a":3, "b":10, "c":5, "d":11}
]
Do someone have a solution for this ?
Is there any existing solution to do this, or how should I proceed to do it myself ?
Thank you.
A task for itertools.product:
>>> from itertools import product
>>> for dict_items in product(*[product([k],v) for k, v in my_dict.items()]):
... print(dict(dict_items))
{'a': 1, 'b': 10, 'c': 4, 'd': 11}
{'a': 1, 'b': 10, 'c': 5, 'd': 11}
{'a': 2, 'b': 10, 'c': 4, 'd': 11}
{'a': 2, 'b': 10, 'c': 5, 'd': 11}
{'a': 3, 'b': 10, 'c': 4, 'd': 11}
{'a': 3, 'b': 10, 'c': 5, 'd': 11}
Small explanation:
The inner product(...) will expand the dict to a list such as [[(k1, v11), (k1, v12), ...], [(k2, v21), (k2, v22), ...], ...].
The outer product(...) will reassemble the items lists by choosing one tuple from each list.
dict(...) will create a dictionary from a sequence of (k1, v#), (k2, v#), ... tuples.
Try:
def permute(d):
k = d.keys()
perms = itertools.product(*d.values())
return [dict(zip(k, v)) for v in perms]
Example usage:
>>> d = {'a': [1, 2, 3], 'b': [10], 'c': [4, 5], 'd': [11]}
>>> pprint(permute(d))
[{'a': 1, 'b': 10, 'c': 4, 'd': 11},
{'a': 1, 'b': 10, 'c': 5, 'd': 11},
{'a': 2, 'b': 10, 'c': 4, 'd': 11},
{'a': 2, 'b': 10, 'c': 5, 'd': 11},
{'a': 3, 'b': 10, 'c': 4, 'd': 11},
{'a': 3, 'b': 10, 'c': 5, 'd': 11}]
Assuming that you are only interested in my_dict having 4 keys, it is simple enough to use nested for loops:
my_dict = {
"a": [1, 2, 3],
"b": [10],
"c": [4, 5],
"d": [11]
}
result = []
for a_val in my_dict['a']:
for b_val in my_dict['b']:
for c_val in my_dict['c']:
for d_val in my_dict['d']:
result.append({'a': a_val, 'b': b_val, 'c': c_val, 'd': d_val})
print(result)
This gives the expected result.
You can use:
from itertools import product
allNames = sorted(my_dict)
values= list(product(*(my_dict[Name] for Name in allNames)))
d = list(dict(zip(['a','b','c','d'],i)) for i in values)
Output:
[{'a': 1, 'c': 4, 'b': 10, 'd': 11},
{'a': 1, 'c': 5, 'b': 10, 'd': 11},
{'a': 2, 'c': 4, 'b': 10, 'd': 11},
{'a': 2, 'c': 5, 'b': 10, 'd': 11},
{'a': 3, 'c': 4, 'b': 10, 'd': 11},
{'a': 3, 'c': 5, 'b': 10, 'd': 11}]
itertools.product produces the combinations of a list of iterators.
dict.values() gets the list needed.
For each combination, zip up the dict.keys() with the combination.
Use a list comprehension to collect them up:
from itertools import product
from pprint import pprint
my_dict = {
"a":[1, 2, 3],
"b":[10],
"c":[4, 5],
"d":[11]
}
result = [dict(zip(my_dict,i)) for i in product(*my_dict.values())]
pprint(result)
Output:
[{'a': 1, 'b': 10, 'c': 4, 'd': 11},
{'a': 1, 'b': 10, 'c': 5, 'd': 11},
{'a': 2, 'b': 10, 'c': 4, 'd': 11},
{'a': 2, 'b': 10, 'c': 5, 'd': 11},
{'a': 3, 'b': 10, 'c': 4, 'd': 11},
{'a': 3, 'b': 10, 'c': 5, 'd': 11}]
here is my list of dict:
l = [{'a': 2, 'c': 1, 'b': 3},
{'a': 2, 'c': 3, 'b': 1},
{'a': 1, 'c': 2, 'b': 3},
{'a': 1, 'c': 3, 'b': 2},
{'a': 2, 'c': 5, 'b': 3}]
and now I want to sort the list by keys and orders provided by the user. for instance:
keys = ['a', 'c', 'b']
orders = [1, -1, 1]
I tried to using lambda in sort()method but it failed in a weird way :
>>> l.sort(key=lambda x: (order * x[key] for (key, order) in zip(keys, orders)))
>>> l
[{'a': 2, 'c': 5, 'b': 3},
{'a': 1, 'c': 3, 'b': 2},
{'a': 1, 'c': 2, 'b': 3},
{'a': 2, 'c': 3, 'b': 1},
{'a': 2, 'c': 1, 'b': 3}]
Anyone know how to solve this?
You were almost there; your lambda produces generator expressions and those happen to be ordered by their memory address (in Python 2) and produce a TypeError: '<' not supported between instances of 'generator' and 'generator' exception in Python 3.
Use a list comprehension instead:
l.sort(key=lambda x: [order * x[key] for (key, order) in zip(keys, orders)])
Demo:
>>> l = [{'a': 1, 'c': 2, 'b': 3},
... {'a': 1, 'c': 3, 'b': 2},
... {'a': 2, 'c': 1, 'b': 3},
... {'a': 2, 'c': 5, 'b': 3},
... {'a': 2, 'c': 3, 'b': 1}]
>>> keys = ['a', 'c', 'b']
>>> orders = [1, -1, 1]
>>> l.sort(key=lambda x: [order * x[key] for (key, order) in zip(keys, orders)])
>>> from pprint import pprint
>>> pprint(l)
[{'a': 1, 'b': 2, 'c': 3},
{'a': 1, 'b': 3, 'c': 2},
{'a': 2, 'b': 3, 'c': 5},
{'a': 2, 'b': 1, 'c': 3},
{'a': 2, 'b': 3, 'c': 1}]
I have the following dict structure:
{12345: {2006: [{'a': 1, 'b': 2}, {'a': 3, 'b': 4}, {'a': 1, 'b': 5}]}, 12346: {2007: [{'a': 2, 'b': 7}, {'a': 1, 'b': 9}, {'a': 1, 'b': 12}]}}
I want to be able to filter based on the keys of 'a' or 'b'
for example if 'a' is 1 the my filtered dict would look like:
{12345: {2006: [{'a': 1, 'b': 2}, {'a': 1, 'b': 5}]}, 12346: {2007: [{'a': 1, 'b': 9}, {'a': 1, 'b': 12}]}}
I have the following for loop which gets me down to where I have the inner dict's I want, but I am not sure how to put it back into a dict of the same structure.
d = {12345: {2006: [{'a': 1, 'b': 2}, {'a': 3, 'b': 4}, {'a': 1, 'b': 5}]}, 12346: {2007: [{'a': 2, 'b': 7}, {'a': 1, 'b': 9}, {'a': 1, 'b': 12}]}}
d_filter = {}
for item_code in d.keys():
for year in d[item_code]:
for item_dict in d[item_code][year]:
if item_dict['a'] == 1:
print(item_dict) # how to put this back in d_filter?
producing:
{'a': 1, 'b': 2}
{'a': 1, 'b': 5}
{'a': 1, 'b': 9}
{'a': 1, 'b': 12}
I am guessing there is a better way to filter that I can not find, or something with dictionary comprehension that my small mind can not grasp.
Any help would be appreciated.
Here's a dictionary comprehension that does just that; dct is your initial dictionary:
d = {k: {ky: [d for d in vl if d['a']==1] for ky, vl in v.items()}
for k, v in dct.items()}
print d
# {12345: {2006: [{'a': 1, 'b': 2}, {'a': 1, 'b': 5}]}, 12346: {2007: [{'a': 1, 'b': 9}, {'a': 1, 'b': 12}]}}
You can change the inner filter (i.e. d['a']==1) to the dict key and/or value of your choice.
You could do something like this:
filtered = {
item_code: {
year: [item for item in items if item['a'] == 1]
for year, items in years.items()
}
for item_code, years in d.items()
}
Which results in:
{12345: {2006: [{'a': 1, 'b': 2}, {'a': 1, 'b': 5}]},
12346: {2007: [{'a': 1, 'b': 9}, {'a': 1, 'b': 12}]}}
This may be a duplicate but the closest I could find was Comparing 2 lists consisting of dictionaries with unique keys in python which did not work for me.
So I have two lists of dictionaries.
y = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
How do I compare these two lists so my compare results in the intersection of the two lists. I can't convert it to set since it says unhashable type (dict)
Your question and it's title seem at odds with each other.
The intersection of the 2 lists would be the common elements of both list. The question title requests the elements that are not in both lists. Which is it that you want?
For the intersection, it is not very efficient (being O(n^2) in time), but this list comprehension will do it:
>>> a = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
>>> b = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
>>> [d for d in a if d in b]
[{'a': 1, 'b': 2, 'c': 3}]
y1 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y2 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
print [x for x in y1 if x in y2] # prints [{'a': 1, 'c': 3, 'b': 2}]
A dict (or list) is not hashable, however, a tuple is. You can convert the list of dicts to a set of tuples. Perform the intersection and then convert back
the code to convert to a set-of-tuples
y_tupleset = set(tuple(sorted(d.items())) for d in y)
the code to convert back the intersected set-of-tuples to a list-of-dicts
y_dictlist = [dict(it) for it in list(y_tupleset)]
Thus, the full code would be:
y0 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y1 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
y0_tupleset = set(tuple(sorted(d.items())) for d in y0)
y1_tupleset = set(tuple(sorted(d.items())) for d in y1)
y_inter = y0_tupleset.intersection(y1_tupleset)
y_inter_dictlist = [dict(it) for it in list(y_inter)]
print(y_inter_dictlist)
# prints the following line
[{'a': 1, 'c': 3, 'b': 2}]
edit: d.items() is valid on python3, for python2, it should be replaced with d.iteritems()
Pick your poison:
y1 = [{'a': 3, 'b': 4, 'c': 5}, {'a': 1, 'b': 2, 'c': 3}]
y2 = [{'a': 4, 'b': 5, 'c': 6}, {'a': 1, 'b': 2, 'c': 3}]
y3 = [{'a': 1, 'b': 2, 'c': 3}, {'a': 4, 'b': 2, 'c': 6}]
# Returns a list of keys that are in both dictionaries
def intersect_keys(d1, d2):
return [k for k in d1 if k in d2]
# Returns a list of values that are in both dictionaries
def intersect_vals(d1, d2):
return [v for v in d1.itervalues() if v in d2.itervalues()]
# Returns a list of (key,value) pairs that are in both dictionaries
def intersect_pairs(d1, d2):
return [(k,v) for (k,v) in d1.iteritems() if k in d2 and d2[k] == v]
print(intersect_keys(*y1)) # ['a', 'c', 'b']
print(intersect_vals(*y1)) # [3]
print(intersect_pairs(*y1)) # []
print(intersect_keys(*y2)) # ['a', 'c', 'b']
print(intersect_vals(*y2)) # []
print(intersect_pairs(*y2)) # []
print(intersect_keys(*y3)) # ['a', 'c', 'b']
print(intersect_vals(*y3)) # [2]
print(intersect_pairs(*y3)) # [('b', 2)]
Note: the examples compare the two elements of the y* list, which was how I interpreted your question. You could of course use something like:
print(intersect_pairs(y1[0], y2[0]))
To compute the intersection the first dictionary in the y1 and y2 lists.