How do I convert a list to a generator? - python

I have a function which returns a list of tuples, that I would like to iterate through:
def get_parameter_product(num_parameters, lower_range, upper_range):
param_lists = [ xrange(lower_range, upper_range) for _ in xrange(num_parameters)]
return list(itertools.product(*param_lists))
for p in get_parameter_product(3, 0, 5):
print p,
(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3), (0, 0, 4), ... , (4, 4, 2), (4, 4, 3), (4, 4, 4)
However, for larger values of num_parameters is take a lot of memory to allocate. Is it possible to convert this to a generator?

itertools.product is already a generator. You can just return it instead of converting it to a list.

Your method is adding an extraneous step that is not needed.
prod_list = get_parameter_product(3, 0, 5)
can be replaced with
prod_genr = product(range(0, 5), repeat=3)
Anything in itertools returns a generator rather than list or tuple. You may wish to look at the documentation to check you really want product as opposed to combinations or permutations.

You don't need to convert a list to a generator—just don't convert your generators into lists.
First, to build up param_lists, use a generator expression instead of a list comprehension, and you'll have an iterator instead of a list.
Second, don't call list in the return statement.
>>> def get_parameter_product(num_parameters, lower_range, upper_range):
... param_lists = (xrange(lower_range, upper_range) for _ in xrange(num_parameters))
... return itertools.product(*param_lists)
>>> p = get_parameter_product(3, 0, 5)
>>> print(p)
<generator object <genexpr> at 0x101d650f0>
>>> print(list(p))
[(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3), (0, 0, 4), ... , (4, 4, 2), (4, 4, 3), (4, 4, 4)]

Related

Python Assign Multiple Variables with Map Function

With Python, I can assign multiple variables like so:
a, b = (1, 2)
print(a)
print(b)
# 1
# 2
Can I do something similar with the map function?
def myfunc(a):
return (a+1, a-1)
a_plus_one, a_minus_one = map(myfunc, (1, 2, 3))
# or
a_plus_one, a_minus_one = list(map(myfunc, (1,2,3)))
print(a_plus_one)
print(a_minus_one)
These attempts give me too many values to unpack error.
Edit:
Desired output is two new lists.
a_plus_one = (2, 3, 4)
a_minus_one = (0, 1, 2)
It looks like you're misunderstanding how map works. Look at list(map(myfunc, (1,2,3))):
[(2, 0), (3, 1), (4, 2)]
You want to transpose that using zip:
>>> a_plus_one, a_minus_one = zip(*map(myfunc, (1,2,3)))
>>> a_plus_one
(2, 3, 4)
>>> a_minus_one
(0, 1, 2)
For more info: Transpose list of lists

Concatenating returned elements to list in a recursive function

This one has been giving me a headache for too long
I am trying to create a list of tuples from a recursion, but I can't quite figure out if how I'm approaching this is going to work or not.
Below, foo() and A are aliases for more complicated methods and structures, but I'd like foo() below to return the following:
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8,
8)]
First attempt
When I try adding them together as lists, it nests all the lists.
A = [(num, num) for num in np.arange(9)]
def foo(A):
if len(A)==1:
return(A[0])
else:
return([A[0]] + [foo(A[1:])])
print(foo(A))
output: [(0, 0), [(1, 1), [(2, 2), [(3, 3), [(4, 4), [(5, 5), [(6, 6),
[(7, 7), (8, 8)]]]]]]]]
Second attempt
I can understand why this is wrong, so I tried appending the returned values to the list at the higher level, nothing returns:
A = [(num, num) for num in np.arange(9)]
def foo(A):
if len(A)==1:
return(A[0])
else:
return([A[0]].append(foo(A[1:])))
print(foo(A))
output: None
Current solution (there's got to be a better way)
def foo(A):
if len(A)==1:
return(A[0])
else:
return(A[0] + foo(A[1:]))
output: (0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8)
...then:
temp = np.array(foo(A)).reshape(-1,2)
output = [tuple(temp[i, :]) for i in range(np.shape(temp)[0])]
print(output)
which gives the desired output...
Can someone give some advice on how to do this correctly using recursion?
I'm not sure what you are asking for since A is already the structure you want. But for your first attempt, you mess up the return type. The first if returns a number, but the second if returns a list. So make sure the first if returns a list, and remove the list conversion in the second if. It should be like this:
import numpy as np
A = [(num, num) for num in np.arange(9)]
def foo(A):
if len(A)==1:
return([A[0]])
else:
return([A[0]] + foo(A[1:]))
print(foo(A))
You pretty much had it on your first try. By adding an unpacking and altering your original base case, we can get your desired result.
def foo(A):
if len(A)==1:
return([A[0]])
else:
return([A[0]] + [*foo(A[1:])])
print(foo(A))

How to append to an itertools generator

Is there a simple way to append an integer to each item in an itertools iterator? If I use itertools.product, I do not receive the expected output. For example:
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
((0, 1), 4)
((0, 2), 4)
((0, 3), 4)
((1, 2), 4)
((1, 3), 4)
((2, 3), 4)
But I would expect (and I want) is
>>> for i in itertools.product(itertools.combinations(np.arange(4),2),(4,)):
... print(i)
...
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
I know that I can "flatten" the output, but I would rather construct the iterator to produce tuples, not tuples of tuples.
I have many different iterators floating around, and I want to keep the code the same for products of itertool iterators and plain itertool iterators
These two alternatives each produce an iterator. In the first case, the iterator is created by a generator expression. In the second, the iterator is created by the use of a generator function.
In [9]: for i in (tup + (4,) for tup in itertools.combinations(np.arange(4),2)):
...: print(i)
...:
(0, 1, 4)
(0, 2, 4)
(0, 3, 4)
(1, 2, 4)
(1, 3, 4)
(2, 3, 4)
In [10]:
A generator function might be more readable at the call site, especially if the function name describes its behavior.
import itertools
import numpy as np
def adder(it, addend):
for x in it:
yield x + addend
for i in adder(itertools.combinations(np.arange(4),2), (4,)):
print(i)

Python: Print a generator expression's values when those values are itertools.product objects

I'm trying to dig into some code I found online here to better understand Python.
This is the code fragment I'm trying to get a feel for:
from itertools import chain, product
def generate_groupings(word_length, glyph_sizes=(1,2)):
cartesian_products = (
product(glyph_sizes, repeat=r)
for r in range(1, word_length + 1)
)
Here, word_length is 3.
I'm trying to evaluate the contents of the cartesian_products generator. From what I can gather after reading the answer at this SO question, generators do not iterate (and thus, do not yield a value) until they are called as part of a collection, so I've placed the generator in a list:
list(cartesian_products)
Out[6]:
[<itertools.product at 0x1025d1dc0>,
<itertools.product at 0x1025d1e10>,
<itertools.product at 0x1025d1f50>]
Obviously, I now see inside the generator, but I was hoping to get more specific information than the raw details of the itertools.product objects. Is there a way to accomplish this?
if you don't care about exhausting the generator, you can use:
list(map(list,cartesian_products))
You will get the following for word_length = 3
Out[1]:
[[(1,), (2,)],
[(1, 1), (1, 2), (2, 1), (2, 2)],
[(1, 1, 1),
(1, 1, 2),
(1, 2, 1),
(1, 2, 2),
(2, 1, 1),
(2, 1, 2),
(2, 2, 1),
(2, 2, 2)]]

Is it possible to include mutiple equations in a lambda function

I'm trying to include multiple operations in the lambda function with variables that have different lengths, i.e. something like:
$ serial_result = map(lambda x,y:(x**2,y**3), range(20), range(10))
but this doesn't work. Could someone tell me how to get around this?
I understand that:
$ serial_result = map(lambda x,y:(x**2,y**3), range(0,20,2), range(10))
works because the arrays of "x" and "y" have the same length.
If you want the product of range items you can use itertools.product :
>>> from itertools import product
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), product(range(20), range(10)))
If you want to pass the pairs to lambda like second case you can use itertools.zip_longest (in python 2 use izip_longest)and pass a fillvalue to fill the missed items,
>>> from itertools import zip_longest
>>> serial_result = map(lambda x:(x[0]**2,x[1]**3), zip_longest(range(20), range(10),fillvalue=1))
Note that if you are in python 2 you can pass multiple argument to lambda as a tuple :
>>> serial_result = map(lambda (x,y):(x**2,y**3), product(range(20), range(10)))
See the difference of izip_longest and product in following example :
>>> list(product(range(5),range(3)))
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2)]
>>> list(zip_longest(range(5),range(3)))
[(0, 0), (1, 1), (2, 2), (3, None), (4, None)]
>>> list(zip_longest(range(5),range(3),fillvalue=1))
[(0, 0), (1, 1), (2, 2), (3, 1), (4, 1)]
It sounds like you may be confused as to how exactly you want to use all the values in these two variables. There are several ways combine them...
If you want a result for every combination of an element in a and an element in b: itertools.product(a, b).
If you want to stop once you get to the end of the shorter: zip(a, b)
If you want to continue on until you've used all of the longest: itertools.zip_longer(a, b) (izip_longer in python 2). Once a runs out of elements it will be filled in with None, or a default you provide.

Categories