Is there a simple way to append an integer to each item in an itertools iterator? If I use itertools.product, I do not receive the expected output. For example:
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
((0, 1), 4)
((0, 2), 4)
((0, 3), 4)
((1, 2), 4)
((1, 3), 4)
((2, 3), 4)
But I would expect (and I want) is
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
I know that I can "flatten" the output, but I would rather construct the iterator to produce tuples, not tuples of tuples.
I have many different iterators floating around, and I want to keep the code the same for products of itertool iterators and plain itertool iterators
These two alternatives each produce an iterator. In the first case, the iterator is created by a generator expression. In the second, the iterator is created by the use of a generator function.
In [9]: for i in (tup + (4,) for tup in itertools.combinations(np.arange(4),2)):
...: print(i)
...:
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
In [10]:
A generator function might be more readable at the call site, especially if the function name describes its behavior.
import itertools
import numpy as np
def adder(it, addend):
for x in it:
yield x + addend
for i in adder(itertools.combinations(np.arange(4),2), (4,)):
print(i)
Related
I have list with repeated elements, for example array = [2,2,2,7].
If I use the solution suggested in this answer (using itertools.combinations()), I get:
()
(7,)
(2,)
(2,)
(2,)
(7, 2)
(7, 2)
(7, 2)
(2, 2)
(2, 2)
(2, 2)
(7, 2, 2)
(7, 2, 2)
(7, 2, 2)
(2, 2, 2)
(7, 2, 2, 2)
As you can see some of the 'combinations' are repeated, e.g. (7,2,2) appears 3 times.
The output I would like is:
()
(7,)
(2,)
(7, 2)
(2, 2)
(7, 2, 2)
(2, 2, 2)
(7, 2, 2, 2)
I could check the output for repeated combinations but I don't feel like that is the best solution to this problem.
You can take the set of the combinations and then chain them together.
from itertools import chain, combinations
arr = [2, 2, 2, 7]
list(chain.from_iterable(set(combinations(arr, i)) for i in range(len(arr) + 1)))
# [(), (7,), (2,), (2, 7), (2, 2), (2, 2, 2), (2, 2, 7), (2, 2, 2, 7)]
You would need to maintain a set of tuples that are sorted in the same fashion:
import itertools as it
desired=set([(),(7,),(2,),(7, 2),(2, 2),(7, 2, 2),(2, 2, 2),(7, 2, 2, 2)])
result=set()
for i in range(len(array)+1):
for combo in it.combinations(array, i):
result.add(tuple(sorted(combo, reverse=True)))
>>> result==desired
True
Without using itertools.combinations() and set's:
from collections import Counter
import itertools
def powerset(bag):
for v in itertools.product(*(range(r + 1) for r in bag.values())):
yield Counter(zip(bag.keys(), v))
array = [2, 2, 2, 7]
for s in powerset(Counter(array)):
# Convert `Counter` object back to a list
s = list(itertools.chain.from_iterable(itertools.repeat(*mv) for mv in s))
print(s)
I believe your problem could alternatively be stated as finding the power set of a multiset, at least according to this definition.
However it's worth noting that the method shown above will be slower than the solutions in other answers such as this one which simply group the results from itertools.combinations() into a set to remove duplicates, despite being seemingly less efficient, it is still faster in practice as iterating in Python is much slower than in C (see itertoolsmodule.c for the implementation of itertools.combinations()).
Through my limited testing, the method shown in this answer will outperform the previously cited method when there are approximately 14 distinct elements in your array, each with an average multiplicity of 2 (at which point the other method begins to pull away and run many times slower), however the running time for either method under those circumstances are >30Â seconds, so if performance is of concern, then you might want to consider implementing this part of your application in C.
I'm trying to dig into some code I found online here to better understand Python.
This is the code fragment I'm trying to get a feel for:
from itertools import chain, product
def generate_groupings(word_length, glyph_sizes=(1,2)):
cartesian_products = (
product(glyph_sizes, repeat=r)
for r in range(1, word_length + 1)
)
Here, word_length is 3.
I'm trying to evaluate the contents of the cartesian_products generator. From what I can gather after reading the answer at this SO question, generators do not iterate (and thus, do not yield a value) until they are called as part of a collection, so I've placed the generator in a list:
list(cartesian_products)
Out[6]:
[<itertools.product at 0x1025d1dc0>,
<itertools.product at 0x1025d1e10>,
<itertools.product at 0x1025d1f50>]
Obviously, I now see inside the generator, but I was hoping to get more specific information than the raw details of the itertools.product objects. Is there a way to accomplish this?
if you don't care about exhausting the generator, you can use:
list(map(list,cartesian_products))
You will get the following for word_length = 3
Out[1]:
[[(1,), (2,)],
[(1, 1), (1, 2), (2, 1), (2, 2)],
[(1, 1, 1),
(1, 1, 2),
(1, 2, 1),
(1, 2, 2),
(2, 1, 1),
(2, 1, 2),
(2, 2, 1),
(2, 2, 2)]]
I'm trying to include multiple operations in the lambda function with variables that have different lengths, i.e. something like:
$ serial_result = map(lambda x,y:(x**2,y**3), range(20), range(10))
but this doesn't work. Could someone tell me how to get around this?
I understand that:
$ serial_result = map(lambda x,y:(x**2,y**3), range(0,20,2), range(10))
works because the arrays of "x" and "y" have the same length.
If you want the product of range items you can use itertools.product :
>>> from itertools import product
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), product(range(20), range(10)))
If you want to pass the pairs to lambda like second case you can use itertools.zip_longest (in python 2 use izip_longest)and pass a fillvalue to fill the missed items,
>>> from itertools import zip_longest
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), zip_longest(range(20), range(10),fillvalue=1))
Note that if you are in python 2 you can pass multiple argument to lambda as a tuple :
>>> serial_result = map(lambda (x,y):(x**2,y**3), product(range(20), range(10)))
See the difference of izip_longest and product in following example :
>>> list(product(range(5),range(3)))
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]
>>> list(zip_longest(range(5),range(3)))
[(0, 0), (1, 1), (2, 2), (3, None), (4, None)]
>>> list(zip_longest(range(5),range(3),fillvalue=1))
[(0, 0), (1, 1), (2, 2), (3, 1), (4, 1)]
It sounds like you may be confused as to how exactly you want to use all the values in these two variables. There are several ways combine them...
If you want a result for every combination of an element in a and an element in b: itertools.product(a, b).
If you want to stop once you get to the end of the shorter: zip(a, b)
If you want to continue on until you've used all of the longest: itertools.zip_longer(a, b) (izip_longer in python 2). Once a runs out of elements it will be filled in with None, or a default you provide.
I am having trouble finding a way to do this in a Pythonic way. I assume I can use itertools somehow because I've done something similar before but can't remember what I did.
I am trying to generate all non-decreasing lists of length L where each element can take on a value between 1 and N. For example if L=3 and N=3 then [1,1,1],[1,1,2],[1,1,3],[1,2,2],[1,2,3], etc.
You can do this using itertools.combinations_with_replacement:
>>> L, N = 3,3
>>> cc = combinations_with_replacement(range(1, N+1), L)
>>> for c in cc: print(c)
(1, 1, 1)
(1, 1, 2)
(1, 1, 3)
(1, 2, 2)
(1, 2, 3)
(1, 3, 3)
(2, 2, 2)
(2, 2, 3)
(2, 3, 3)
(3, 3, 3)
This works because c_w_r preserves the order of the input, and since we're passing a nondecreasing sequence in, we only get nondecreasing tuples out.
(It's easy to convert to lists if you really need those as opposed to tuples.)
I have a function which returns a list of tuples, that I would like to iterate through:
def get_parameter_product(num_parameters, lower_range, upper_range):
param_lists = [ xrange(lower_range, upper_range) for _ in xrange(num_parameters)]
return list(itertools.product(*param_lists))
for p in get_parameter_product(3, 0, 5):
print p,
(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3), (0, 0, 4), ... , (4, 4, 2), (4, 4, 3), (4, 4, 4)
However, for larger values of num_parameters is take a lot of memory to allocate. Is it possible to convert this to a generator?
itertools.product is already a generator. You can just return it instead of converting it to a list.
Your method is adding an extraneous step that is not needed.
prod_list = get_parameter_product(3, 0, 5)
can be replaced with
prod_genr = product(range(0, 5), repeat=3)
Anything in itertools returns a generator rather than list or tuple. You may wish to look at the documentation to check you really want product as opposed to combinations or permutations.
You don't need to convert a list to a generator—just don't convert your generators into lists.
First, to build up param_lists, use a generator expression instead of a list comprehension, and you'll have an iterator instead of a list.
Second, don't call list in the return statement.
>>> def get_parameter_product(num_parameters, lower_range, upper_range):
... param_lists = (xrange(lower_range, upper_range) for _ in xrange(num_parameters))
... return itertools.product(*param_lists)
>>> p = get_parameter_product(3, 0, 5)
>>> print(p)
<generator object <genexpr> at 0x101d650f0>
>>> print(list(p))
[(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3), (0, 0, 4), ... , (4, 4, 2), (4, 4, 3), (4, 4, 4)]