I want to do something analogous to the following:
def normal(start):
term = start
while True:
yield term
term = term + 1
def iterate(start, inc):
if inc == 1:
return normal(start)
else:
term = start
while True:
yield term
term = term + inc
Now this gives the following error.
SyntaxError: 'return' with argument inside generator
How I can return a generator to one function through another?
(Note: Example shown here doesn't require this kind of functionality but it shows what I need to do)
Thanks in advance.
Starting in Python 3.3, you can use yield from normal(start) as described in PEP 380. In earlier versions, you must manually iterate over the other generator and yield what it yields:
if inc == 1:
for item in normal(start):
yield item
Note that this isn't "returning the generator", it's one generator yielding the values yielded by another generator. You could use yield normal(start) to yield the "normal" generator object itself, but judging from your example that isn't what you're looking for.
You cant have returns inside a generator expression.
In Python 2.X you have to chain the generators manually:
def normal(start):
term = start
while True:
yield term
term = term + 1
def iterate(start, inc):
if inc == 1:
for item in normal(start):
yield item
else:
term = start
while True:
yield term
term = term + inc
I'm assuming you know both of these examples will run forever :)
FWIW in your original example you could clean it up by making two generators (say, 'mormal' and 'abnormal') and then returning one or the other from the iterate function. As long as you're not mixing generators and returns you can return generators... so to speak...
#using original 'normal'
def abnormal(start, inc):
term = start
while True:
yield term
term = term + inc
def iterate (start, inc):
if inc == 1:
return normal(start)
return abnormal(start, inc)
Related
I'm trying to use pyresttest's benchmarking framework to generate a sequence of entries in my flask_sqlalchemy-based database. I would like to read input values from a pre-defined list as advertised by this framework's benchmarking generator type 'fixed_sequence', but it's only picking up the first element of the list.
Here is the issue that explains my problem in detail, with an example: https://github.com/svanoort/pyresttest/issues/264
Any pointer in the right direction will be greatly appreciated
I looked into the code, it is jsut a bug, this feature was never used by anyone.
https://github.com/svanoort/pyresttest/blob/master/pyresttest/generators.py#L100
instead of:
```
def factory_fixed_sequence(values):
""" Return a generator that runs through a list of values in order, looping after end """
def seq_generator():
my_list = list(values)
i = 0
while(True):
yield my_list[i]
if i == len(my_list):
i = 0
return seq_generator
It should be:
def factory_fixed_sequence(values):
""" Return a generator that runs through a list of values in order, looping after end """
def seq_generator():
my_list = list(values)
i = 0
while(True):
yield my_list[i]
i += 1
if i == len(my_list):
i = 0
return seq_generator
```
The i += 1 is missing
Now I plan to learn more about yield in python. And I found some codes about yield, which implemented the algorithm Reservoir Sampling as following:
def RandomSelect(knum, rand=None):
''' (int, func) -> list
Reservoir Sampling implementation
'''
selection = None
k_elems_list = []
count = 0
if rand is None:
rand = Random()
while True:
item = yield selection
if len(k_elems_list) < knum:
k_elems_list.append(item)
else:
# Randomly replace elements in the reservoir with a decreasing probability
r = rand.randint(0, count)
if r < knum:
k_elems_list[r] = item
count += 1
print k_elems_list
In order to break the while loop, I just add some codes after item = yield selection
if item == -1: # reach to the end of data, just break
break
Question 1, Is there any better way to break out the while loop?
To call the function RandomSelect,
myList = [1,2,3,4,5,6,7,8,-1]
cr = RandomSelect(3);
cr.next() # advance to the yield statement, otherwise I can't call send
try:
for val in myList:
cr.send(val)
except StopIteration:
pass
finally:
del cr
I have to catch the StopIteration exception explicitly.
Question 2, is there any better way to swallow the StopIteration in the codes?
I think a slightly cleaner way to accomplish what is being done — which addresses both your questions — would be to explicitly close the generator by calling itsclose()method to terminate it and break out of the loop. Doing so also means aStopIterationdoesn't need to be "swallowed". Another benefit is it's no longer necessary to add the -1 sentinel value at the end of the list.
def RandomSelect(knum, rand=None):
''' (int, func) -> list
Reservoir Sampling implementation
'''
selection = None
k_elems_list = []
count = 0
if rand is None:
rand = Random()
while True:
try:
item = yield selection
except GeneratorExit:
break
if len(k_elems_list) < knum:
k_elems_list.append(item)
else:
# Randomly replace elements in the reservoir with a decreasing probability
r = rand.randint(0, count)
if r < knum:
k_elems_list[r] = item
count += 1
print k_elems_list
myList = [1,2,3,4,5,6,7,8]
cr = RandomSelect(3)
cr.next() # advance to the yield statement, otherwise I can't call send
for val in myList:
cr.send(val)
cr.close()
del cr
A minor additional enhancement (about something you didn't ask about) would be to make it so it wasn't necessary to manually advance to theyieldstatement before callingsend(). A good way to accomplish that would be with a decorator function similar to the one namedconsumer()David Beazley described in his Generator Tricks
For Systems Programmers presentation at PyCon 2008:
def coroutine(func):
""" Decorator that takes care of starting a coroutine automatically. """
def start(*args, **kwargs):
cr = func(*args, **kwargs)
cr.next()
return cr
return start
#coroutine
def RandomSelect(knum, rand=None):
.
.
.
print k_elems_list
myList = [1,2,3,4,5,6,7,8]
cr = RandomSelect(3)
#cr.next() # NO LONGER NECESSARY
for val in myList:
cr.send(val)
cr.close()
del cr
Sometimes, when rewriting recursive functions as generators, I miss the brevity of return.
"""
Returns a list of all length n strings that can be made out of a's and/or b's.
"""
def ab_star(n):
if n == 0:
return [""]
results = []
for s in ab_star(n - 1):
results.append("a" + s)
results.append("b" + s)
return results
turns into
"""
Generator for all length n strings that can be made out of a's and/or b's.
"""
def ab_star(n):
if n == 0:
yield ""
else:
for s in ab_star(n - 1):
yield "a" + s
yield "b" + s
It's that else that bugs me. I wish there was a way to say "yield, and this is it, so exit the function". Is there a way?
Don't miss return, use it.
You can return right after you yield.
def ab_star(n):
if n == 0:
yield ""
return
for s in ab_star(n - 1):
yield "a" + s
yield "b" + s
An alternative is to use return in both cases, where the first case returns a sequence of length 1, and the second returns a generator-expression:
def ab_star(n):
if n == 0:
return ( "", )
return ( c+s for s in ab_star(n - 1) for c in 'ab' )
This avoidance of yield avoids the limitation that you cannot use both return <value> and yield in the same function.
(This works in your case because your function doesn't have to be a generator. Since you only iterate over the results, it can also return a tuple.)
There isn't. When I wrote the "Simple Generators PEP", I noted:
Q. Then why not allow an expression on "return" too?
A. Perhaps we will someday. In Icon, "return expr" means both "I'm
done", and "but I have one final useful value to return too, and
this is it". At the start, and in the absence of compelling uses
for "return expr", it's simply cleaner to use "yield" exclusively
for delivering values.
But that never gained traction. Until it does ;-), you can make your generator look more like your first function by writing the first part as:
if n == 0:
yield ""
return
Then you can drop the else: statement and dedent the rest.
In short. How do I write something else than this: for another in combinationOfK(K-1, L[i+1:]): My function combinationOfK(...) is not iterable.
I am trying to understand the code from here, solution to. Problem 26: Generate the combinations of K distinct objects chosen from the N elements of a listI know what yield does. But I am trying to write the code without a yield statement. Code with yield statement is this.
def combination(K, L):
if K<=0:
yield []
return
for i in range(len(L)):
thisone = L[i:i+1]
for another in combination(K-1, L[i+1:]):
yield thisone + another
The question, yield-keyword-explained gave me the idea that I could replace yield. The recepie they give, which is not working for me, is:
When you see a function with yield statements, apply this easy
trick to understand what will happen:
Insert a line result = [] at the start of the function.
Replace each yield expr with result.append(expr).
Insert a line return result at the bottom of the function.
Yay - no more yield statements! Read and figure out code.
Revert function to original definition.
Using this to get code without yield give me this. The code is not working (the function is not iterable). What do I have to write to get this code working without yield?
def combinationOfK(K,L):
result = []
if K <= 0:
result.append([])
return
for i in range(len(L)):
thisone = L[i:i+1]
for another in combinationOfK(K-1, L[i+1:]): # the error
result.append(thisone + another)
return result
I am using this code to test the function,
the_list = ['a','b','c','d','e']
print list(combinationOfK(2, the_list))
raising error TypeError: 'NoneType' object is not iterable.
The problem is that your original code uses return in an unusual way.
def combination(K, L):
if K<=0:
yield []
return # <--- hmmm
Most of the time you won't see return in a generator, because you don't often need it. Usually, generators simply "fall off" at the end; the interpreter reaches the end of the generator without encountering a return statement, and then it knows to throw StopIteration.
In this case, the writer of the code has inserted a return statement to "hurry up" the process. When K <= 0, there's no more work to be done, so the generator can throw StopIteration -- but without the return statement, it would go into the for loop, producing incorrect results. In my opinion, a clearer way to do this would have been like so:
def combination(K, L):
if K<=0:
yield []
else:
for i in range(len(L)):
thisone = L[i:i+1]
for another in combination(K-1, L[i+1:]):
yield thisone + another
Now the conversion works as expected:
def combination2(K, L):
result = []
if K <= 0:
result.append([])
else:
for i in range(len(L)):
thisone = L[i:i + 1]
for another in combination2(K - 1, L[i + 1:]):
result.append(thisone + another)
return result
As Vincent mentioned, your function is returning None because of the 5th line. Change it to this:
def combinationOfK(K,L):
result = []
if K <= 0:
result.append([])
return result
for i in range(len(L)):
thisone = L[i:i+1]
for another in combinationOfK(K-1, L[i+1:]): # the error
result.append(thisone + another)
return result
However, why are you against yield? Generators make for readable, efficient code. The point of the Yield Keyword Explained article was not to dispense with it, but rather, to explain it.
In the generator code you posted:
def combination(K, L):
if K<=0:
yield []
return
for i in range(len(L)):
thisone = L[i:i+1]
for another in combination(K-1, L[i+1:]):
yield thisone + another
The return statement does NOT mean the same thing as return does in a normal function. In a generator, return immediately raises StopIteration, which causes the caller to stop iterating over the generator object.
Recently I wrote a function to generate certain sequences with nontrivial constraints. The problem came with a natural recursive solution. Now it happens that, even for relatively small input, the sequences are several thousands, thus I would prefer to use my algorithm as a generator instead of using it to fill a list with all the sequences.
Here is an example. Suppose we want to compute all the permutations of a string with a recursive function. The following naive algorithm takes an extra argument 'storage' and appends a permutation to it whenever it finds one:
def getPermutations(string, storage, prefix=""):
if len(string) == 1:
storage.append(prefix + string) # <-----
else:
for i in range(len(string)):
getPermutations(string[:i]+string[i+1:], storage, prefix+string[i])
storage = []
getPermutations("abcd", storage)
for permutation in storage: print permutation
(Please don't care about inefficiency, this is only an example.)
Now I want to turn my function into a generator, i.e. to yield a permutation instead of appending it to the storage list:
def getPermutations(string, prefix=""):
if len(string) == 1:
yield prefix + string # <-----
else:
for i in range(len(string)):
getPermutations(string[:i]+string[i+1:], prefix+string[i])
for permutation in getPermutations("abcd"):
print permutation
This code does not work (the function behaves like an empty generator).
Am I missing something?
Is there a way to turn the above recursive algorithm into a generator without replacing it with an iterative one?
def getPermutations(string, prefix=""):
if len(string) == 1:
yield prefix + string
else:
for i in xrange(len(string)):
for perm in getPermutations(string[:i] + string[i+1:], prefix+string[i]):
yield perm
Or without an accumulator:
def getPermutations(string):
if len(string) == 1:
yield string
else:
for i in xrange(len(string)):
for perm in getPermutations(string[:i] + string[i+1:]):
yield string[i] + perm
This avoids the len(string)-deep recursion, and is in general a nice way to handle generators-inside-generators:
from types import GeneratorType
def flatten(*stack):
stack = list(stack)
while stack:
try: x = stack[0].next()
except StopIteration:
stack.pop(0)
continue
if isinstance(x, GeneratorType): stack.insert(0, x)
else: yield x
def _getPermutations(string, prefix=""):
if len(string) == 1: yield prefix + string
else: yield (_getPermutations(string[:i]+string[i+1:], prefix+string[i])
for i in range(len(string)))
def getPermutations(string): return flatten(_getPermutations(string))
for permutation in getPermutations("abcd"): print permutation
flatten allows us to continue progress in another generator by simply yielding it, instead of iterating through it and yielding each item manually.
Python 3.3 will add yield from to the syntax, which allows for natural delegation to a sub-generator:
def getPermutations(string, prefix=""):
if len(string) == 1:
yield prefix + string
else:
for i in range(len(string)):
yield from getPermutations(string[:i]+string[i+1:], prefix+string[i])
The interior call to getPermutations -- it's a generator, too.
def getPermutations(string, prefix=""):
if len(string) == 1:
yield prefix + string
else:
for i in range(len(string)):
getPermutations(string[:i]+string[i+1:], prefix+string[i]) # <-----
You need to iterate through that with a for-loop (see #MizardX posting, which edged me out by seconds!)