Is it possible to use a generator or iterator in a while loop in Python? For example, something like:
i = iter(range(10))
while next(i):
# your code
The point of this would be to build iteration into the while loop statement, making it similar to a for loop, with the difference being that you can now additional logic into the while statement:
i = iter(range(10))
while next(i) and {some other logic}:
# your code
It then becomes a nice for loop/while loop hybrid.
Does anyone know how to do this?
In Python >= 3.8, you can do the following, using assignment expressions:
i = iter(range(10))
while (x := next(i, None)) is not None and x < 5:
print(x)
In Python < 3.8 you can use itertools.takewhile:
from itertools import takewhile
i = iter(range(10))
for x in takewhile({some logic}, i):
# do stuff
"Some logic" here would be a 1-arg callable receciving whatever next(i) yields:
for x in takewhile(lambda e: 5 > e, i):
print(x)
0
1
2
3
4
There are two problems with while next(i):
Unlike a for loop, the while loop will not catch the StopIteration exception that is raised if there is no next value; you could use next(i, None) to return a "falsey" value in that case, but then the while loop will also stop whenever the iterator returns an actual falsey value
The value returned by next will be consumed and no longer available in the loop's body. (In Python 3.8+, that could be solved with an assignment expression, see other answer.)
Instead, you could use a for loop with itertools.takewhile, testing the current element from the iterable, or just any other condition. This will loop until either the iterable is exhausted, or the condition evaluates to false.
from itertools import takewhile
i = iter(range(10))
r = 0
for x in takewhile(lambda x: r < 10, i):
print("using", x)
r += x
print("result", r)
Output:
using 0
...
using 4
result 10
You just need to arrange for your iterator to return a false-like value when it expires. E.g., if we reverse the range so that it counts down to 0:
>>> i = iter(range(5, -1, -1))
>>> while val := next(i):
... print('doing something here with value', val)
...
This will result in:
doing something here with value 5
doing something here with value 4
doing something here with value 3
doing something here with value 2
doing something here with value 1
a = iter(range(10))
try:
next(a)
while True:
print(next(a))
except StopIteration:
print("Stop iteration")
You can do
a = iter(range(10))
try:
a.next()
while True and {True or False logic}:
print("Bonjour")
a.next()
except StopIteration:
print("a.next() Stop iteration")
Related
In Python 3, zip(*iterables) as of the documentation
Returns an iterator of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The iterator stops when the shortest input iterable is exhausted.
As an example, I am running
for x in zip(a,b):
f(x)
Is there a way to find out which of the iterables, a or b, led to the stopping of the zip iterator?
Assume that len() is not reliable and iterating over both a and b to check their lengths is not feasible.
I found the following solution which replaces zip with a for loop over only the first iterable and iterates over the second one inside the loop.
ib = iter(b)
for r in a:
try:
s = next(ib)
except StopIteration:
print('Only b exhausted.')
break
print((r,s))
else:
try:
s = next(ib)
print('Only a exhausted.')
except StopIteration:
print('a and b exhausted.')
Here ib = iter(b) makes sure that it also works if b is a sequence or generator object. print((r,s)) would be replaced by f(x) from the question.
I think Jan has the best answer. Basically, you want to handle the last iteration from zip separately.
import itertools as it
a = (x for x in range(5))
b = (x for x in range(3))
iterables = ((it.chain(g,[f"generator {i} was exhausted"]) for i,g in enumerate([a,b])))
for i, j in zip(*iterables):
print(i, j)
# 0 0
# 1 1
# 2 2
# 3 generator 1 was exhausted
If you have only two iterables, you can use the below code. The exhausted[0] will have your indicator for which iterator was exhausted. Value of None means both were exhausted.
However I must say that I do not agree with len() not being reliable. In fact, you should depend on the len() call to determine the answer. (unless you tell us the reason why you can not.)
def f(val):
print(val)
def manual_iter(a,b, exhausted):
iters = [iter(it) for it in [a,b]]
iter_map = {}
iter_map[iters[0]] = 'first'
iter_map[iters[1]] = 'second'
while 1:
values = []
for i, it in enumerate(iters):
try:
value = next(it)
except StopIteration:
if i == 0:
try:
next(iters[1])
except StopIteration:
return None
exhausted.append(iter_map[it])
return iter_map[it]
values.append(value)
yield tuple(values)
if __name__ == '__main__':
exhausted = []
a = [1,2,3]
b = [10,20,30]
for x in manual_iter(a,b, exhausted):
f(x)
print(exhausted)
exhausted = []
a = [1,2,3,4]
b = [10,20,30]
for x in manual_iter(a,b, exhausted):
f(x)
print(exhausted)
exhausted = []
a = [1,2,3]
b = [10,20,30,40]
for x in manual_iter(a,b, exhausted):
f(x)
print(exhausted)
See below for by me written function zzip() which will do what you want to achieve. It uses the zip_longest method from the itertools module and returns a tuple with what zip would return plus a list of indices which if not empty shows at which 0-based position(s) was/were the iterable/iterables) becoming exhausted before other ones:
def zzip(*args):
""" Returns a tuple with the result of zip(*args) as list and a list
with ZERO-based indices of iterables passed to zzip which got
exhausted before other ones. """
from itertools import zip_longest
nanNANaN = 'nanNANaN'
Zipped = list(zip_longest(*args, fillvalue=nanNANaN))
ZippedT = list(zip(*Zipped))
Indx_exhausted = []
indx_nanNANaN = None
for i in range(len(args)):
try: # gives ValueError if nanNANaN is not in the column
indx_nanNANaN = ZippedT[i].index(nanNANaN)
Indx_exhausted += [(indx_nanNANaN, i)]
except ValueError:
pass
if Indx_exhausted: # list not empty, iterables were not same length
Indx_exhausted.sort()
min_indx_nanNANaN = Indx_exhausted[0][0]
Indx_exhausted = [
i for n, i in Indx_exhausted if n == min_indx_nanNANaN ]
return (Zipped[:min_indx_nanNANaN], Indx_exhausted)
else:
return (Zipped, Indx_exhausted)
assert zzip(iter([1,2,3]),[4,5],iter([6])) ==([(1,4,6)],[2])
assert zzip(iter([1,2]),[3,4,5],iter([6,7]))==([(1,3,6),(2,4,7)],[0,2])
assert zzip([1,2],[3,4],[5,6]) ==([(1,3,5),(2,4,6)],[])
The code above runs without raising an assertion error on the used test cases.
Notice that the 'for loop' in the function loops over the items of the passed parameter list and not over the elements of the passed iterables.
What exactly happens, when yield and return are used in the same function in Python, like this?
def find_all(a_str, sub):
start = 0
while True:
start = a_str.find(sub, start)
if start == -1: return
yield start
start += len(sub) # use start += 1 to find overlapping matches
Is it still a generator?
Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.
PEP 255 spells it out:
Specification: Return
A generator function can also contain return statements of the form:
"return"
Note that an expression_list is not allowed on return statements in
the body of a generator (although, of course, they may appear in the
bodies of non-generator functions nested within the generator).
When a return statement is encountered, control proceeds as in any
function return, executing the appropriate finally clauses (if any
exist). Then a StopIteration exception is raised, signalling that the
iterator is exhausted. A StopIteration exception is also raised if
control flows off the end of the generator without an explict return.
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
Note that return isn't always equivalent to raising StopIteration:
the difference lies in how enclosing try/except constructs are
treated. For example,
>>> def f1():
... try:
... return
... except:
... yield 1
>>> print list(f1())
[]
because, as in any function, return simply exits, but
>>> def f2():
... try:
... raise StopIteration
... except:
... yield 42
>>> print list(f2())
[42]
because StopIteration is captured by a bare "except", as is any
exception.
Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see #NPE's answer for details).
Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.
As pointed out by #BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.
From PEP 380:
In a generator, the statement
return value
is semantically equivalent to
raise StopIteration(value)
There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.
It probably is not as clean as you would want but it does do what you expect.
Here's an example:
def six(how_many=None):
if how_many is None or how_many < 1:
return None # returns value
if how_many == 1:
return 6 # returns value
def iter_func():
for count in range(how_many):
yield 6
return iter_func() # returns generator
Note: you don't get StopIteration exception with the example below.
def odd(max):
n = 0
while n < max:
yield n
n = n + 1
return 'done'
for x in odd(3):
print(x)
The for loop catches it. That's its signal to stop
But you can catch it in this way:
g = odd(3)
while True:
try:
x = next(g)
print(x)
except StopIteration as e:
print("g return value:", e.value)
break
What exactly happens, when yield and return are used in the same function in Python, like this?
def find_all(a_str, sub):
start = 0
while True:
start = a_str.find(sub, start)
if start == -1: return
yield start
start += len(sub) # use start += 1 to find overlapping matches
Is it still a generator?
Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.
PEP 255 spells it out:
Specification: Return
A generator function can also contain return statements of the form:
"return"
Note that an expression_list is not allowed on return statements in
the body of a generator (although, of course, they may appear in the
bodies of non-generator functions nested within the generator).
When a return statement is encountered, control proceeds as in any
function return, executing the appropriate finally clauses (if any
exist). Then a StopIteration exception is raised, signalling that the
iterator is exhausted. A StopIteration exception is also raised if
control flows off the end of the generator without an explict return.
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
Note that return isn't always equivalent to raising StopIteration:
the difference lies in how enclosing try/except constructs are
treated. For example,
>>> def f1():
... try:
... return
... except:
... yield 1
>>> print list(f1())
[]
because, as in any function, return simply exits, but
>>> def f2():
... try:
... raise StopIteration
... except:
... yield 42
>>> print list(f2())
[42]
because StopIteration is captured by a bare "except", as is any
exception.
Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see #NPE's answer for details).
Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.
As pointed out by #BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.
From PEP 380:
In a generator, the statement
return value
is semantically equivalent to
raise StopIteration(value)
There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.
It probably is not as clean as you would want but it does do what you expect.
Here's an example:
def six(how_many=None):
if how_many is None or how_many < 1:
return None # returns value
if how_many == 1:
return 6 # returns value
def iter_func():
for count in range(how_many):
yield 6
return iter_func() # returns generator
Note: you don't get StopIteration exception with the example below.
def odd(max):
n = 0
while n < max:
yield n
n = n + 1
return 'done'
for x in odd(3):
print(x)
The for loop catches it. That's its signal to stop
But you can catch it in this way:
g = odd(3)
while True:
try:
x = next(g)
print(x)
except StopIteration as e:
print("g return value:", e.value)
break
I know there are easier ways to create a function which gives you the largest number in a list of numbers but I wanted to use recursion. When I call the function greatest, i get none. For example greatest([1,3,2]) gives me none. If there are only two elements in the list, I get the right answer so I know the problem must be with the function calling itself. Not sure why though.
def compare(a,b):
if a==b:
return a
if a > b:
return a
if a < b:
return b
def greatest(x):
if len(x)==0:
return 0
i=0
new_list=[]
while i< len(x):
if len(x)-i>1:
c=compare(x[i],x[i+1])
else:
c=x[i]
new_list.append(c)
i=i+2
if len(new_list)>1:
greatest(new_list)
else:
return new_list[0]
print greatest([1,3,2])
This line:
if len(new_list)>1:
greatest(new_list) # <- this one here
calls greatest but doesn't do anything with the value it returns. You want
return greatest(new_list)
After fixing that, your function seems to behave (although I didn't look too closely):
>>> import itertools
>>> for i in range(1, 6):
... print i, all(max(g) == greatest(g) for g in itertools.product(range(-5, 5), repeat=i))
...
1 True
2 True
3 True
4 True
5 True
A simple recursion can be like this :
from random import *
def greatest(x,maxx=float("-inf")):
if len(x)>0:
if x[0] > maxx:
maxx=x[0]
return greatest(x[1:],maxx)
else:
return maxx
lis=range(10,50)
shuffle(lis)
print greatest(lis) #prints 49
I'm a big fan of Python's for...else syntax - it's surprising how often it's applicable, and how effectively it can simplify code.
However, I've not figured out a nice way to use it in a generator, for example:
def iterate(i):
for value in i:
yield value
else:
print 'i is empty'
In the above example, I'd like the print statement to be executed only if i is empty. However, as else only respects break and return, it is always executed, regardless of the length of i.
If it's impossible to use for...else in this way, what's the best approach to this so that the print statement is only executed when nothing is yielded?
You're breaking the definition of a generator, which should throw a StopIteration exception when iteration is complete (which is automatically handled by a return statement in a generator function)
So:
def iterate(i):
for value in i:
yield value
return
Best to let the calling code handle the case of an empty iterator:
count = 0
for value in iterate(range([])):
print value
count += 1
else:
if count == 0:
print "list was empty"
Might be a cleaner way of doing the above, but that ought to work fine, and doesn't fall into any of the common 'treating an iterator like a list' traps below.
There are a couple ways of doing this. You could always use the Iterator directly:
def iterate(i):
try:
i_iter = iter(i)
next = i_iter.next()
except StopIteration:
print 'i is empty'
return
while True:
yield next
next = i_iter.next()
But if you know more about what to expect from the argument i, you can be more concise:
def iterate(i):
if i: # or if len(i) == 0
for next in i:
yield next
else:
print 'i is empty'
raise StopIteration()
Summing up some of the earlier answers, it could be solved like this:
def iterate(i):
empty = True
for value in i:
yield value
empty = False
if empty:
print "empty"
so there really is no "else" clause involved.
As you note, for..else only detects a break. So it's only applicable when you look for something and then stop.
It's not applicable to your purpose not because it's a generator, but because you want to process all elements, without stopping (because you want to yield them all, but that's not the point).
So generator or not, you really need a boolean, as in Ber's solution.
If it's impossible to use for...else in this way, what's the best approach to this so that the print statement is only executed when nothing is yielded?
Maximum i can think of:
>>> empty = True
>>> for i in [1,2]:
... empty = False
... if empty:
... print 'empty'
...
>>>
>>>
>>> empty = True
>>> for i in []:
... empty = False
... if empty:
... print 'empty'
...
empty
>>>
What about simple if-else?
def iterate(i):
if len(i) == 0: print 'i is empty'
else:
for value in i:
yield value