I was looking into this article by RealPython about Python generators. It uses a palindrome generator as a exemplary tool to explain the .send, .throw and .close methods. Building a program following their instructions gives the following code:
def infinite_palindromes():
num = 0
while True:
if is_palindrome(num):
i = (yield num)
if i is not None:
num = i
num += 1
def is_palindrome(num):
# Skip single-digit inputs
if num // 10 == 0:
return False
temp = num
reversed_num = 0
while temp != 0:
reversed_num = (reversed_num * 10) + (temp % 10)
temp = temp // 10
if num == reversed_num:
return True
else:
return False
pal_gen = infinite_palindromes()
for e in pal_gen:
digits = len(str(e))
pal_gen.send(10**(digits))
I watched the flow of the program in debugging and what happens is basically, when the function finds 11, the first palindrome, it enters into the for e loop block, the digits variable is defined, and 10 to the power of that variable value is sent back to the generator. 100 is sent back, and since it is different from None, num is equated to i and becomes 100, then it's summed with 1 and becomes 101, the next palindrome. Here is what I don't get: when that number is throw and correctly identified as a palindrome, the code goes back to the for e in pal_gen: line, but it doesn't enter the block. Only the next palindrome, 111 is sent back down and enters the block, where the digits variable is defined again, and so on and so forth, and this patterns repeats, with only every other palindrome entering the block. Why is that?
Can someone, please, explain to me why that happens?
To understand the behaviour, we need to understand how for loops are implemented in Python. This code:
for e in pal_gen:
digits = len(str(e))
pal_gen.send(10**(digits))
is basically equivalent to (except that no name _iter is actually generated):
_iter = iter(pal_gen)
while True:
try:
e = next(_iter)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
However, because pal_gen is a generator object (created by calling the generator function infinite_palindromes), it is already an iterator; calling iter on it returns the same object unchanged. Thus, we effectively have:
while True:
try:
e = next(pal_gen)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
The other thing to note here is that calling .send on a generator advances the generator, and so does passing it to next (i.e., using it like any other iterator). It's just that the code was written to ignore the value retrieved using .send.
In fact, next on a generator is equivalent to calling .send and passing None (by convention; Python defines the behaviour this way so that the assignment in the generator code, i = (yield num), has something to assign).
So, each time through the loop, two values are sent to the generator:
while True:
try:
e = pal_gen.send(None)
digits = len(str(e))
pal_gen.send(10**(digits))
except StopIteration:
break
The control flow is like so:
None is sent to the generator. Because it hasn't yielded yet, this value is discarded (Python requires it to be None, since it can't be used by the generator).
The generator iterates its own loop until 11 is found and yielded. The execution of the generator pauses here: i is not assigned yet.
The main loop receives 11 as a value for e, computes 100 as the next value to work with, and sends that to the generator.
The generator receives 100 and assigns it to i, then proceeds with its logic. The next step is if i is not None:, and that is indeed the case - thus, 100 is assigned to num. num is incremented at the end of the loop.
The loop inside the generator runs into the next iteration, and immediately finds that 101 is a palindrome. Thus, the .send call returns a value of 101, but this isn't assigned anywhere.
Similarly, in the second iteration of the main loop, 111 will be found and yielded (as the next palindrome after 101) and assigned to e in the first call; then 1000 is sent explicitly, causing 1001 to be yielded and ignored. In the third iteration, 1111 is assigned to e, then 10001 is ignored. In the fourth iteration, 10101 (not 11111! This is the next palindrome after 10001) is assigned, and 100001 is ignored. Etc.
If the code is tested at the REPL, the values returned from the explicit .send will be displayed. This is just a quirk of the REPL.
Related
While I try to understand the mechanism of generator's send() method, I encountered a strange behaviour in my example. (VS Code Python 3.10)
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
yield num
num += 1
#test
iter = MultipleOfThree()
for m3 in iter:
print(m3)
This code is working as expected and prints
>>> 3,6,9,12,15,18,21.....
When I add the send() statement in for loop, and arrange the MultipleOfThree() func like below
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
i = yield num
if i is not None:
num = i
num += 1
#test
iter = MultipleOfThree()
for m3 in iter:
print(m3)
iter.send(m3) #for simplicity send the m3 itself
it prints
>>> 3,9,15,21,27
I couldn't understand the mechanism of send() here, why escapes the for loop.
I study the subject from this page How to Use Generators and yield in Python.
The generator makes a stop at each yield of some value. It continues after the value is consumed by calling next on the corresponding iterator. It continues until the next yield (or generator exit).
A for loop uses this iterator protocol to consume the yielded values one at a time and runs the loop body with each one.
A send is similar to next. It consumes a value, makes the generator to advance, but in addition to next It also sends a value to the generator. The sent value becomes the value returned by the yield (in the generator).
Your code is interacting with the generator at two places: in the for statement and in the send statement. But you are printing only one of the values, that's why you see only every second.
Add a print and you'll see all values again:
for m3 in iter:
print(m3) # print 1 (yielded by for loop)
print(iter.send(m3)) # print 2 (yielded by send)
First initialize the generator:
iter = MultipleOfThree()
Get the first value:
m3= next(iter)
Then continue in a while loop:
while True:
print(m3)
m3 = iter.send(m3)
You wish to get the first value using next(iter) or iter.send(None) in order to reach the first multiple of three, without any input.
Then, you want to send back the input and continue from there using iter.send(m3).
Overall code:
def MultipleOfThree():
num = 1
while True:
if num % 3 == 0:
i = yield num
if i is not None:
num = i
num += 1
iter = MultipleOfThree()
m3= next(iter)
while True:
print(m3)
m3 = iter.send(m3)
I'm using yield from, but I don't know about the influence of while for yield. If I put yield from in a while loop, it works well, but when I cancel the loop at the mean time an exception occurs.
final_result = {}
def sales_sum(pro_name):
total = 0
nums = []
while True:
x = yield
print(pro_name+" Sales volume: ", x)
if not x:
break
total += x
nums.append(x)
return total, nums
def middle(key):
while True:
final_result[key] = yield from sales_sum(key)
def middle2(key):
final_result[key] = yield from sales_sum(key)
def main(fun):
data_sets = { "A": [1200, 1500], "B": [28,55,98]}
for key, data_set in data_sets.items():
m = fun(key)
m.send(None)
for value in data_set:
m.send(value)
m.send(None)
if __name__ == '__main__':
main(middle) # work well
main(middle2) # StopIteration
I expect main(middle2) to work well as main(middle), but there is a StopIteration exception.
The cause of the unexpected StopIteration exception in main is that your m.send(None) call causes your middle2 generator to be exhausted (after the sub-generator sales_sum breaks out of its loop in response to the falsey value it received). When a generator is exhausted, it raises StopIteration. Normally that's invisible because you consume iterators in for loops, but in this case, it breaks your code.
There are a few ways you could fix this. One would be to use a two-argument call to next instead of using m.send(None):
next(m, None)
This does the same thing as m.send(None), but has the added benefit of suppressing the StopIteration. Note that the None in the call to next is not really the same as the one in send. It's the default return value in the case of an exhausted iterator, not the value that gets sent in (which is always None when using next).
Another approach would be to change middle2 so that it doesn't end when the sales_sum generator does. You could add an extra yield statement at the end, so that it yields control one last time after doing its assignment to final_result when its sub-generator returns.
A final idea would be to replace m.send(None) with m.close(). This would require some changes in final_result, as the close call will throw a GeneratorExit exception into the generator. If you expect it, you could use that as your signal to be done instead of looking for a falsey value:
def sales_sum(pro_name):
total = 0
nums = []
while True:
try:
x = yield
except GeneratorExit:
return total, nums
print(pro_name+" Sales volume: ", x)
total += x
nums.append(x)
With this change, middle2 would not need any modification.
sales_sum is a finite iterator. middle2 iterates over it exactly once; middle tries to iterate over it multiple times.
Why must I use a variable to obtain next() values from a Python generator?
def my_gen():
i = 0
while i < 4:
yield 2 * i
i += 1
#p = my_gen()
#for i in range(4):
# print(next(p))
##for i in range(4):
## print(next(my_gen()))
In the above, the # block works, while the ## block returns 4 copies of the first "yield."
print(next(my_gen()))
Each time this runs, you're calling next() on a separate (and brand-new) generator returned by my_gen().
If you want to call next() on the same generator more than once, you'll need some way to keep that generator around for long enough to reuse it.
just gonna add my $0.02...
this may help clear up some misconceptions:
def function_that_creates_a_generator(n):
while n>0:
yield n
n - 1
# this
for x in function_that_creates_a_generator(5):
print(x)
#is almost exactly the same as this
generator = function_that_creates_a_generator(5)
while True:
try:
x = generator.next() #same as: x = next(generator)
except StopIteration:
break
print(x)
For loops are really just a prettier way of writing a while loop that continually asks an iterable object for its next element. when you ask an iterable for another object and it has none, it will raise a StopIteration exception that is automatically caught in for loops; signaling a break from the loop. In the while loop we simply break these things out individually instead of having them hidden by the interpreter.
Does Python have anything in the fashion of a "redo" statement that exists in some languages?
(The "redo" statement is a statement that (just like "break" or "continue") affects looping behaviour - it jumps at the beginning of innermost loop and starts executing it again.)
No, Python doesn't have direct support for redo. One option would something faintly terrible involving nested loops like:
for x in mylist:
while True:
...
if shouldredo:
continue # continue becomes equivalent to redo
...
if shouldcontinue:
break # break now equivalent to continue on outer "real" loop
...
break # Terminate inner loop any time we don't redo
but this mean that breaking the outer loop is impossible within the "redo-able" block without resorting to exceptions, flag variables, or packaging the whole thing up as a function.
Alternatively, you use a straight while loop that replicates what for loops do for you, explicitly creating and advancing the iterator. It has its own issues (continue is effectively redo by default, you have to explicitly advance the iterator for a "real" continue), but they're not terrible (as long as you comment uses of continue to make it clear you intend redo vs. continue, to avoid confusing maintainers). To allow redo and the other loop operations, you'd do something like:
# Create guaranteed unique sentinel (can't use None since iterator might produce None)
sentinel = object()
iterobj = iter(mylist) # Explicitly get iterator from iterable (for does this implicitly)
x = next(iterobj, sentinel) # Get next object or sentinel
while x is not sentinel: # Keep going until we exhaust iterator
...
if shouldredo:
continue
...
if shouldcontinue:
x = next(iterobj, sentinel) # Explicitly advance loop for continue case
continue
...
if shouldbreak:
break
...
# Advance loop
x = next(iterobj, sentinel)
The above could also be done with a try/except StopIteration: instead of two-arg next with a sentinel, but wrapping the whole loop with it risks other sources of StopIteration being caught, and doing it at a limited scope properly for both inner and outer next calls would be extremely ugly (much worse than the sentinel based approach).
No, it doesn't. I would suggest using a while loop and resetting your check variable to the initial value.
count = 0
reset = 0
while count < 9:
print 'The count is:', count
if not someResetCondition:
count = count + 1
This is my solution using iterators:
class redo_iter(object):
def __init__(self, iterable):
self.__iterator = iter(iterable)
self.__started = False
self.__redo = False
self.__last = None
self.__redone = 0
def __iter__(self):
return self
def redo(self):
self.__redo = True
#property
def redone(self):
return self.__redone
def __next__(self):
if not (self.__started and self.__redo):
self.__started = True
self.__redone = 0
self.__last = next(self.__iterator)
else:
self.__redone += 1
self.__redo = False
return self.__last
# Display numbers 0-9.
# Display 0,3,6,9 doubled.
# After a series of equal numbers print --
iterator = redo_iter(range(10))
for i in iterator:
print(i)
if not iterator.redone and i % 3 == 0:
iterator.redo()
continue
print('---')
Needs explicit continue
redone is an extra feature
For Python2 use def next(self) instead of def __next__(self)
requires iterator to be defined before the loop
I just meet the same question when I study perl,and I find this page.
follow the book of perl:
my #words = qw(fred barney pebbles dino wilma betty);
my $error = 0;
my #words = qw(fred barney pebbles dino wilma betty);
my $error = 0;
foreach (#words){
print "Type the word '$_':";
chomp(my $try = <STDIN>);
if ($try ne $_){
print "Sorry - That's not right.\n\n";
$error++;
redo;
}
}
and how to achieve it on Python ??
follow the code:
tape_list=['a','b','c','d','e']
def check_tape(origin_tape):
errors=0
while True:
tape=raw_input("input %s:"%origin_tape)
if tape == origin_tape:
return errors
else:
print "your tape %s,you should tape %s"%(tape,origin_tape)
errors += 1
pass
all_error=0
for char in tape_list:
all_error += check_tape(char)
print "you input wrong time is:%s"%all_error
Python has not the "redo" syntax,but we can make a 'while' loop in some function until get what we want when we iter the list.
Not very sophiscated but easy to read, using a while and an increment at the end of the loop. So any continue in between will have the effect of a redo. Sample to redo every multiple of 3:
redo = True # To ends redo condition in this sample only
i = 0
while i<10:
print(i, end='')
if redo and i % 3 == 0:
redo = False # To not loop indifinively in this sample
continue # Redo
redo = True
i += 1
Result: 00123345667899
There is no redo in python.
A very understandable solution is as follow:
for x in mylist:
redo = True
while redo:
redo = False
If should_redo:
redo = True
It's clear enough to do not add comments
Continue will work as if it was in the for loop
But break is not useable, this solution make break useable but the code is less clear.
Here is a solution for python 3.8+ since now we have the := operator:
for key in mandatory_attributes: # example with a dictionary
while not (value := input(f"{key} (mandatory): ")):
print("You must enter a value")
mandatory_attributes[key] = value
Consider the following code:
def print_mah(n):
if n <= 0:
return
else:
print('mah')
print_mah(n-1)
print_mah(3)
Here Python checks if n is less than or equal to 0, it finds that this is False so it prints 'mah' and calls the same function with n-1 until n is equal to 0, so 'mah' is printed 3 times.
But consider this manipulated code:
def print_mah(n):
if n <= 0:
return
else:
print_mah(n-1)
print('mah')
print_mah(3)
Python checks if n is less than or equal to 0, it finds that this is False so calls the same function again with n-1, and 'mah' is printed 3 times, too.
My question is that why 'mah' isn't printed just one time, in other words why print_mah isn't called with n=2, then Python finds that the condition is False, so it calls it with n=1, and finds the condition is False, so it calls it with n==0, and finds that the condition is True, so the function returns, and after that 'mah' is printed, only, once.
Python finds that the condition is False, so it calls it with n=1, and finds the condition is False, so it calls it with n==0, and finds that the condition is True, so the function returns
That actually is the exact execution path of the second version. Maybe you don't see, though, that when the function returns it picks back up where it left off after the recursive call returns, just like any other method call.
Therefore, when n==1 and it recurses with n==0, then it is returned and mah is printed the first time, then that returns and mah is printed when n==2, then that returns and mah is printed a third and final time.
return doesn't break out of your entire call stack, just the current function call. In this case:
def print_mah(n):
if n <= 0:
return
else:
print_mah(n-1)
print('mah')
print_mah(3)
print_mah is called three times before returning 0. You can think about it as if they were just nested logic, like this:
def print_mah(n):
if n <= 0:
return
else:
print_mah(n-1) # Replace this line with new call
print('mah')
Where we just call the function again inside the else statement at the comment.
def print_mah(n):
if n <= 0:
#return
else:
n = n-1
if n <= 0:
#return
else:
print_mah(n-1)
print('mah')
print('mah')
You can see that print('mah') shows up in sequence at the bottom, which, as it's written, will print sequentially as the functions backtrack out of the call stack.
Here is a "trace" of what the second program is doing:
print_mah(3) ->
print_mah(2) ->
print_mah(1) ->
print_mah(0) ->
# Does nothing, essentially.
<-
# print_mah(1) continues running.
print('mah') # The last thing that print_mah(1) does.
<-
# print_mah(2) continues running.
print('mah')
<-
# print_mah(3) continues running.
print('mah')
<-
What we see here is that print('mah') occurs three times (therefore, 'mah' gets printed three times).
My question is that why 'mah' isn't printed just one time, in other words why print_mah isn't called with n=2, then Python finds that the condition is False, so it calls it with n=1, and finds the condition is False, so it calls it with n==0, and finds that the condition is True, so the function returns, and after that 'mah' is printed, only, once.
The function only returns out of the most-inner function. The two function levels where the else condition is used will then continue execution after the call to print_mah and actually print. Here's a line-by-line walk through for print_mah(2) for brevity.
print_mah(2)
# Enter print_mah - Level 1
if n <= 0: # n = 2
# ...
# False, continue to else
else:
print_mah(n-1) # n = 2
# Enter print_mah - Level 2
if n <= 0: # n = 1
# ...
# False, continue to else
else:
print_mah(n-1) # n = 1
# Enter print_mah - Level 3
if n <= 0: # n = 0
return
# Return None to print_mah - Level 2
print('mah')
# Function is complete, return None to print_mah - Level 1
print('mah')
# Function is complete, return None to the execution scope
To understand the different maybe this can help.
Algorithm 1
def print_n(n):
if n <= 0:
return
else:
print_n(n-1)
print(n)
Algorithm 2
def print_n(n):
if n <= 0:
return
else:
print(n)
print_n(n-1)
These algorithms should provide different results, maybe this is a good starting point for further research.
Some help
If you call a function (f2) inside an other function (f1) the current function (f1) will wait until the called function (f2) finished.
Some keywords for research
function call
stack
Both functions will print mah three times, because you are not returning anything after the recursive call. When you call the function inside itself you should handle the stopping condition (which you already did in your first if-condition), then after that it will NOT exit the program because a recursive call builds a stack of operations. After the last operation gets stopped (returns something), then it will start to compile the rest of the function below the recursive call on its way back, until the end of the function is reached.