While I try to understand the mechanism of generator's send() method, I encountered a strange behaviour in my example. (VS Code Python 3.10)
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
yield num
num += 1
#test
iter = MultipleOfThree()
for m3 in iter:
print(m3)
This code is working as expected and prints
>>> 3,6,9,12,15,18,21.....
When I add the send() statement in for loop, and arrange the MultipleOfThree() func like below
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
i = yield num
if i is not None:
num = i
num += 1
#test
iter = MultipleOfThree()
for m3 in iter:
print(m3)
iter.send(m3) #for simplicity send the m3 itself
it prints
>>> 3,9,15,21,27
I couldn't understand the mechanism of send() here, why escapes the for loop.
I study the subject from this page How to Use Generators and yield in Python.
The generator makes a stop at each yield of some value. It continues after the value is consumed by calling next on the corresponding iterator. It continues until the next yield (or generator exit).
A for loop uses this iterator protocol to consume the yielded values one at a time and runs the loop body with each one.
A send is similar to next. It consumes a value, makes the generator to advance, but in addition to next It also sends a value to the generator. The sent value becomes the value returned by the yield (in the generator).
Your code is interacting with the generator at two places: in the for statement and in the send statement. But you are printing only one of the values, that's why you see only every second.
Add a print and you'll see all values again:
for m3 in iter:
print(m3) # print 1 (yielded by for loop)
print(iter.send(m3)) # print 2 (yielded by send)
First initialize the generator:
iter = MultipleOfThree()
Get the first value:
m3= next(iter)
Then continue in a while loop:
while True:
print(m3)
m3 = iter.send(m3)
You wish to get the first value using next(iter) or iter.send(None) in order to reach the first multiple of three, without any input.
Then, you want to send back the input and continue from there using iter.send(m3).
Overall code:
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
i = yield num
if i is not None:
num = i
num += 1
iter = MultipleOfThree()
m3= next(iter)
while True:
print(m3)
m3 = iter.send(m3)
Related
I was looking into this article by RealPython about Python generators. It uses a palindrome generator as a exemplary tool to explain the .send, .throw and .close methods. Building a program following their instructions gives the following code:
def infinite_palindromes():
num = 0
while True:
if is_palindrome(num):
i = (yield num)
if i is not None:
num = i
num += 1
def is_palindrome(num):
# Skip single-digit inputs
if num // 10 == 0:
return False
temp = num
reversed_num = 0
while temp != 0:
reversed_num = (reversed_num * 10) + (temp % 10)
temp = temp // 10
if num == reversed_num:
return True
else:
return False
pal_gen = infinite_palindromes()
for e in pal_gen:
digits = len(str(e))
pal_gen.send(10**(digits))
I watched the flow of the program in debugging and what happens is basically, when the function finds 11, the first palindrome, it enters into the for e loop block, the digits variable is defined, and 10 to the power of that variable value is sent back to the generator. 100 is sent back, and since it is different from None, num is equated to i and becomes 100, then it's summed with 1 and becomes 101, the next palindrome. Here is what I don't get: when that number is throw and correctly identified as a palindrome, the code goes back to the for e in pal_gen: line, but it doesn't enter the block. Only the next palindrome, 111 is sent back down and enters the block, where the digits variable is defined again, and so on and so forth, and this patterns repeats, with only every other palindrome entering the block. Why is that?
Can someone, please, explain to me why that happens?
To understand the behaviour, we need to understand how for loops are implemented in Python. This code:
for e in pal_gen:
digits = len(str(e))
pal_gen.send(10**(digits))
is basically equivalent to (except that no name _iter is actually generated):
_iter = iter(pal_gen)
while True:
try:
e = next(_iter)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
However, because pal_gen is a generator object (created by calling the generator function infinite_palindromes), it is already an iterator; calling iter on it returns the same object unchanged. Thus, we effectively have:
while True:
try:
e = next(pal_gen)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
The other thing to note here is that calling .send on a generator advances the generator, and so does passing it to next (i.e., using it like any other iterator). It's just that the code was written to ignore the value retrieved using .send.
In fact, next on a generator is equivalent to calling .send and passing None (by convention; Python defines the behaviour this way so that the assignment in the generator code, i = (yield num), has something to assign).
So, each time through the loop, two values are sent to the generator:
while True:
try:
e = pal_gen.send(None)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
The control flow is like so:
None is sent to the generator. Because it hasn't yielded yet, this value is discarded (Python requires it to be None, since it can't be used by the generator).
The generator iterates its own loop until 11 is found and yielded. The execution of the generator pauses here: i is not assigned yet.
The main loop receives 11 as a value for e, computes 100 as the next value to work with, and sends that to the generator.
The generator receives 100 and assigns it to i, then proceeds with its logic. The next step is if i is not None:, and that is indeed the case - thus, 100 is assigned to num. num is incremented at the end of the loop.
The loop inside the generator runs into the next iteration, and immediately finds that 101 is a palindrome. Thus, the .send call returns a value of 101, but this isn't assigned anywhere.
Similarly, in the second iteration of the main loop, 111 will be found and yielded (as the next palindrome after 101) and assigned to e in the first call; then 1000 is sent explicitly, causing 1001 to be yielded and ignored. In the third iteration, 1111 is assigned to e, then 10001 is ignored. In the fourth iteration, 10101 (not 11111! This is the next palindrome after 10001) is assigned, and 100001 is ignored. Etc.
If the code is tested at the REPL, the values returned from the explicit .send will be displayed. This is just a quirk of the REPL.
I've been learning how send method and yield assignment work in generator and met a problem:
def gen():
for i in range(5):
x = yield i
print(x)
s = gen()
print(next(s))
print(s.send(15))
output:
0 #after print(next(s))
15
1
so it prints 15 and 1 after print(s.send(15)). It broke my understanding of yield, because I don't understand why it yields 1 after printing x. I'm wondering if someone knows the answer.
When you call s.send(15) the generator resumes running. The value of yield is the argument to s.send(), so it does x = 15 and the generator prints that. Then the for loop repeats with i = 1, and it does yield i.
The value of s.send() is the next value that's yielded, so print(s.send(15)) prints that 1.
I am trying to understand generator functions and from what I have read, the .throw() method is used when dealing with generators if you want to pass an externally generated value to the middle of a function, rather than at the beginning as an argument.
I created a generator function to return the smallest 1 digit palindrome, the smallest 2 digit palindrome and so on till the number of digits equal to 10.
def isPalindrome(num):
if num == int(str(num)[::-1]):
return True
return False
def palindrome_special():
num = 0
while True:
if isPalindrome(num):
num = (yield num) - 1
num = num + 1
for number in palindrome_special():
print(number)
num_digits = len(str(number))
if num_digits <= 10:
palindrome_special().send(10 ** num_digits)
This code is returning a TypeError though:
Traceback (most recent call last):
File "C:/Users/anura/PycharmProjects/Advanced Python/advanced_generators.py", line 17, in
palindrome_special().send(10 ** num_digits)
TypeError: can't send non-None value to a just-started generator
You have to note that you can only pass "not None" value to the generator/coroutine if it's state is "suspended".
Generators have 4 states : CREATED - SUSPENDED - CLOSED - RUNNING
def coroutine():
var = yield
print(var)
obj = coroutine()
try:
obj.send(4)
except StopIteration:
pass
In this case you'll get that error again because it's state is CREATED.
now let's try:
def coroutine():
print('changing state to SUSPEND')
var = yield
print(var)
obj = coroutine()
next(obj)
try:
obj.send(4)
except StopIteration:
pass
As you can see I first change it's state to SUSPEND using next() so that it is now ready to accept value. The execution point for the generator is now right before the 'yield' statement.
Let me add that, next() doing the same thing as send(None). So you could use obj.send(None) in the fist example that I wrote.
I have some textbook code that calls itself recursively. I don't understand the program flow. Here is the code:
def Recur_Factorial_Data(DataArray):
numbers = list(DataArray)
num_counter = 0
list_of_results = []
for num_float in numbers:
n = int(num_float)
1. result = Recur_Factorial(n)
list_of_results.append(result)
def Recur_Factorial(num):
if num == 1:
2. return num
else:
result = num*Recur_Factorial(num-1)
3. return result
if num < 0:
return -1
elif num == 0:
return 0
else:
return 0
In Recur_Factorial_Data, I loop through the data elements and call Recur_Factorial, which returns its value back to the calling function (Recur_Factorial_Data). I would expect that the lines marked 2 ("return num") and 3 ("return result") would always return a value back to the calling function, but that's not the case. For example, where the initial value (from the array DataArray) is 11, the function repeatedly calls itself until num is 1; at that point, the program falls to the line marked 2, but it does not loop back to the line marked 1. Instead, it falls through to the next line. The same happened on the line marked 3 -- I would expect it to return the result back to the calling function, but it does that in some cases and not in others.
I hope this is enough description to understand my question -- I just don't know why each return does not loop back to return the result to the calling function.
EDIT: The question at Understanding how recursive functions work is very helpful, and I recommend it to anyone interested in recursion. My question here is a bit different -- I asked it in the context of the program flow of the Python code where the recursive function is called.
If it call itself recursively 10 times, it will be at the 10th level of recursion and should go back 10 times at the point where there was a recursive call with an intermediate result and only then exit from the recursive function to the place where it was called. Learn more about recursion
Also try to rearrange instructions in Recur_Factorial function in this way:
def Recur_Factorial(num):
if num < 0:
return -1
elif num == 0:
return 0
elif num == 1:
return num
else:
return num * Recur_Factorial(num-1)
Why must I use a variable to obtain next() values from a Python generator?
def my_gen():
i = 0
while i < 4:
yield 2 * i
i += 1
#p = my_gen()
#for i in range(4):
# print(next(p))
##for i in range(4):
## print(next(my_gen()))
In the above, the # block works, while the ## block returns 4 copies of the first "yield."
print(next(my_gen()))
Each time this runs, you're calling next() on a separate (and brand-new) generator returned by my_gen().
If you want to call next() on the same generator more than once, you'll need some way to keep that generator around for long enough to reuse it.
just gonna add my $0.02...
this may help clear up some misconceptions:
def function_that_creates_a_generator(n):
while n>0:
yield n
n - 1
# this
for x in function_that_creates_a_generator(5):
print(x)
#is almost exactly the same as this
generator = function_that_creates_a_generator(5)
while True:
try:
x = generator.next() #same as: x = next(generator)
except StopIteration:
break
print(x)
For loops are really just a prettier way of writing a while loop that continually asks an iterable object for its next element. when you ask an iterable for another object and it has none, it will raise a StopIteration exception that is automatically caught in for loops; signaling a break from the loop. In the while loop we simply break these things out individually instead of having them hidden by the interpreter.