This is my piece of code with two generators defined:
one_line_gen = (x for x in range(3))
def three_line_gen():
yield 0
yield 1
yield 2
When I execute:
for x in one_line_gen:
print x
for x in one_line_gen:
print x
The result is as expected:
0
1
2
However, if I execute:
for x in three_line_gen():
print x
for x in three_line_gen():
print x
The result is:
0
1
2
0
1
2
Why? I thought any generator can be used only once.
three_line_gen is not a generator, it's a function. What it returns when you call it is a generator, a brand new one each time you call it. Each time you put parenthesis like this:
three_line_gen()
It is a brand new generator to be iterated on. If however you were to first do
mygen = three_line_gen()
and iterate over mygen twice, the second time will fail as you expect.
no, you can not iterate over a generator twice. a generator is exhausted once you have iterated over it. you may make a copy of a generator with tee though:
from itertools import tee
one_line_gen = (x for x in range(3))
gen1, gen2 = tee(one_line_gen)
# or:
# gen1, gen2 = tee(x for x in range(3))
for item in gen1:
print(item)
for item in gen2:
print(item)
for the other issues see Ofer Sadan's answer.
Yes, generator can be used only once. but you have two generator object.
# Python 3
def three_line_gen():
yield 0
yield 1
yield 2
iterator = three_line_gen()
print(iterator)
for x in iterator:
print(id(iterator), x)
iterator2 = three_line_gen()
print(iterator2)
for x in iterator2:
print(id(iterator2), x)
And the result is:
<generator object three_line_gen at 0x1020401b0>
4328784304 0
4328784304 1
4328784304 2
<generator object three_line_gen at 0x1020401f8>
4328784376 0
4328784376 1
4328784376 2
Why? I thought any generator can be used only once.
Because every call to three_line_gen() creates a new generator.
Otherwise, you're correct that generators only run forward until exhausted.
Can generator be used more than once?
Yes, it is possible if the results are buffered outside the generator. One easy way is to use itertools.tee():
>>> from itertools import tee
>>> def three_line_gen():
yield 0
yield 1
yield 2
>>> t1, t2 = tee(three_line_gen())
>>> next(t1)
0
>>> next(t2)
0
>>> list(t1)
[1, 2]
>>> list(t2)
[1, 2]
Because in One liner is Generator Object while the three liner is a function.
They meant to be different.
These two are similar.
def three_line_gen_fun():
yield 0
yield 1
yield 2
three_line_gen = three_line_gen_fun()
one_line_gen = (x for x in range(3))
type(three_line_gen) == type(one_line_gen)
Related
I want to use next to skip one or more items returned from a generator. Here is a simplified example designed to skip one item per loop (in actual use, I'd test n and depending on the result, may repeat the next() and the generator is from a package I don't control):
def gen():
for i in range(10):
yield i
for g in gen():
n = next(gen())
print(g, n)
I expected the result to be
0 1
2 3
etc.
Instead I got
0 0
1 0
etc.
What am I doing wrong?
You're making a new generator each time you call gen(). Each new generator starts from 0.
Instead, you can call it once and capture the return value.
def gen():
for i in range(10):
yield i
x = gen()
for g in x:
n = next(x)
print(g, n)
What is the Pythonic way to make a generator that also produces aggregate results? In meta code, something like this (but not for real, as my Python version does not support mixing yield and return):
def produce():
total = 0
for item in find_all():
total += 1
yield item
return total
As I see it, I could:
Not make produce() a generator, but pass it a callback function to call on every item.
With every yield, also yield the aggregate results up until now. I'd rather not calculate the intermediate results with every yield, only when finishing.
Send a dict as argument to produce() that will be populated with the aggregate results.
Use a global to store aggregate results.
All of them don't seem very attractive...
NB. total is a simple example, my actual code requires complex aggregations. And I need intermediate results before produce() finishes, hence a generator.
Maybe you shouldn't use a generator but an iterator.
def findall(): # no idea what your "find_all" does so I use this instead. :-)
yield 1
yield 2
yield 3
class Produce(object):
def __init__(self, iterable):
self._it = iterable
self.total = 0
def __iter__(self):
return self
def __next__(self):
self.total += 1
return next(self._it)
next = __next__ # only necessary for python2 compatibility
Maybe better to see this with an example:
>>> it = Produce(findall())
>>> it.total
0
>>> next(it)
1
>>> next(it)
2
>>> it.total
2
you can use enumerate to count stuff, for example
i=0
for i,v in enumerate(range(10), 1 ):
print(v)
print("total",i)
(notice the start value of the enumerate)
for more complex stuff, you can use the same principle, make produce a generator that yield both values and ignore one in the iteration and use it later when finished.
other alternative is passing a modifiable object, for example
def produce(mem):
t=0
for x in range(10):
t+=1
yield x
mem.append(t)
aggregate=[]
for x in produce(aggregate):
print(x)
print("total",aggregate[0])
in either case the result is the same for this example
0
1
2
3
4
5
6
7
8
9
total 10
Am I missing something? Why not:
def produce():
total = 0
for item in find_all():
total += 1
yield item
yield total
How can I check if the some blackbox generator is awaiting the value or it is returning the value now? I mean managing the following generator:
def gen():
a = yield
yield a
yield a+1
yield a+2
can be the following:
g = gen()
g.next()
print g.send(5)
print g.next()
print g.next()
and for the different generator, for example:
def gen():
a = yield
b = yield
yield a+b
it needs to be different also, for example:
g = gen()
g.next()
g.send(1)
print g.send(2)
So the question is how can I choose between sending value in generator and getting results from it only in case of blackbox (third-party) generator? I need to write the following code:
values = [1, 2, 3]
results = list()
g = gen()
g.next()
for v in values:
# needs this magic
if g.__awaits__: # in case of "x = yield" expression
results.append(g.send(v))
elif g.__yields__: # in case of "yield x" expression
results.append(g.next())
You can't, because there isn't any difference in terms of the allowable operations on those two "kinds" of generators. You can always send a value to the generator as soon as you have called next on it once, even if the generator doesn't do anything with the sent value:
>>> def gen():
... yield 1
... yield 2
... yield 3
>>> x = gen()
>>> next(x)
1
>>> x.send('hello')
2
>>> x.send('hello')
3
There's no way to tell whether the generator "needs" you to send a value except by reading the documentation for whatever generator you're using.
I'm trying to write an infinite generator that will repeat every positive integer n times. So for example, if I create f = inf_repeat(3), printing the output of f 10 times would result in:
1 1 1 2 2 2 3 3 3 4
I am close but not quite there. Here's what I've got:
# courtesy of http://stackoverflow.com/questions/279561/what-is-the-python-equivalent-of-static-variables-inside-a-function
# a generator that yields items instead of returning a list
def static_var(varname, value):
def decorate(func):
setattr(func, varname, value)
return func
return decorate
def inf_repeat(k):
count_limit = k
#static_var("counter", 0)
#static_var("number", 0)
def func():
while True:
if func.counter == count_limit:
func.counter = 0
func.number += 1
func.counter += 1
yield func.number
return func
My problem is that this doesn't behave entirely like an iterator. The following commands work:
f3 = inf_repeat(3)
print next(f3())
But it's irritating to have to call f3 with parens. I'd like to be able to use the standard iterator syntax I've seen, such as:
print(f3.next())
and
new_list = [iter(f3)]*5
What do I need to modify in my function to get to that point? Looking at a variety of generator tutorials, it seemed that yield was sufficient to create a generator, but clearly that's not the case.
Also I have no objective to using a module. I checked itertools but maybe I missed something that could do what I want without all this code?
You just need to call the generator object (what you called f3) at some point. You can call it when you create it:
f3 = inf_repeat(3)()
or even inside inf_repeat
# change last line to this
return func()
Putting yield in your function makes it a generator function --- that is, a function that, when called, returns a generator. If you want to get the generator, you need to call your generator function at some point.
Incidentally, your implementation is needlessly complex. You can get your desired behavior much more simply without all those decorators and nested functions:
def inf_repeat(k):
number = k
while True:
yield number // k
number += 1
Then:
>>> list(itertools.islice(inf_repeat(3), 10))
[1, 1, 1, 2, 2, 2, 3, 3, 3, 4]
>>> list(itertools.islice(inf_repeat(4), 13))
[1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4]
Here's a simple solution:
def inf_repeat(N):
i = 1
while True:
for n in range(N):
yield i
i += 1
# Testing:
f = inf_repeat(3)
for j in range(10):
print f.next()
Here's a solution using itertools:
def inf_repeat(N):
return chain.from_iterable(repeat(i, N) for i in count(1))
Another solution using itertools
import itertools as it
def inf_repeat(k):
for i in it.count(1):
for j in [i]*k:
yield j
for n in inf_repeat(3): print n
produces
1
1
1
2
2
2
...
def f(n):
i = 0
while True:
yield i // n
i += 1
I am using python 3.3.3. i am learning generator and as far as I have gone, I have learnt two ways to initialize a generator.
using generator comprehension
gen1 = (i * i for i in range(6))
using function and yield
def fib():
a,b=0,1
while True:
yield a
a,b=b,a+b
gen2=fib()
now when i use next(gen1) again and again, i get 0,1,4,9,... as output.
but when i use next(fib()) again and again, i get 0, 0 ,0,0 as output. i was thinking the output should be instead 0,1,1,2,3,5,....
print(next(fib())
print(next(fib())
print(next(fib())
print(next(fib())
is the code that prints 0,0,0,0
>>> def fib():
... a,b=0,1
... while True:
... yield a
... a,b=b,a+b
...
>>> next(fib())
0
>>> next(fib())
0
>>> gen = fib()
>>> next(gen)
0
>>> next(gen)
1
>>>
When you do next(fib()) you are creating the generator from scratch each time.
The following program prints fibonacci numbers neatly in Python 3.3.3 Can you show me your full program?
def fib():
a,b=0,1
while True:
yield a
a,b=b,a+b
gen2=fib()
for i in gen2:
print(i)
if i > 100: #otherwise this will go into an infinite loop
break
Prints out
0
1
1
2
3
..
144