What exactly happens, when yield and return are used in the same function in Python, like this?
def find_all(a_str, sub):
start = 0
while True:
start = a_str.find(sub, start)
if start == -1: return
yield start
start += len(sub) # use start += 1 to find overlapping matches
Is it still a generator?
Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.
PEP 255 spells it out:
Specification: Return
A generator function can also contain return statements of the form:
"return"
Note that an expression_list is not allowed on return statements in
the body of a generator (although, of course, they may appear in the
bodies of non-generator functions nested within the generator).
When a return statement is encountered, control proceeds as in any
function return, executing the appropriate finally clauses (if any
exist). Then a StopIteration exception is raised, signalling that the
iterator is exhausted. A StopIteration exception is also raised if
control flows off the end of the generator without an explict return.
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
Note that return isn't always equivalent to raising StopIteration:
the difference lies in how enclosing try/except constructs are
treated. For example,
>>> def f1():
... try:
... return
... except:
... yield 1
>>> print list(f1())
[]
because, as in any function, return simply exits, but
>>> def f2():
... try:
... raise StopIteration
... except:
... yield 42
>>> print list(f2())
[42]
because StopIteration is captured by a bare "except", as is any
exception.
Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see #NPE's answer for details).
Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.
As pointed out by #BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.
From PEP 380:
In a generator, the statement
return value
is semantically equivalent to
raise StopIteration(value)
There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.
It probably is not as clean as you would want but it does do what you expect.
Here's an example:
def six(how_many=None):
if how_many is None or how_many < 1:
return None # returns value
if how_many == 1:
return 6 # returns value
def iter_func():
for count in range(how_many):
yield 6
return iter_func() # returns generator
Note: you don't get StopIteration exception with the example below.
def odd(max):
n = 0
while n < max:
yield n
n = n + 1
return 'done'
for x in odd(3):
print(x)
The for loop catches it. That's its signal to stop
But you can catch it in this way:
g = odd(3)
while True:
try:
x = next(g)
print(x)
except StopIteration as e:
print("g return value:", e.value)
break
Does Python have anything in the fashion of a "redo" statement that exists in some languages?
(The "redo" statement is a statement that (just like "break" or "continue") affects looping behaviour - it jumps at the beginning of innermost loop and starts executing it again.)
No, Python doesn't have direct support for redo. One option would something faintly terrible involving nested loops like:
for x in mylist:
while True:
...
if shouldredo:
continue # continue becomes equivalent to redo
...
if shouldcontinue:
break # break now equivalent to continue on outer "real" loop
...
break # Terminate inner loop any time we don't redo
but this mean that breaking the outer loop is impossible within the "redo-able" block without resorting to exceptions, flag variables, or packaging the whole thing up as a function.
Alternatively, you use a straight while loop that replicates what for loops do for you, explicitly creating and advancing the iterator. It has its own issues (continue is effectively redo by default, you have to explicitly advance the iterator for a "real" continue), but they're not terrible (as long as you comment uses of continue to make it clear you intend redo vs. continue, to avoid confusing maintainers). To allow redo and the other loop operations, you'd do something like:
# Create guaranteed unique sentinel (can't use None since iterator might produce None)
sentinel = object()
iterobj = iter(mylist) # Explicitly get iterator from iterable (for does this implicitly)
x = next(iterobj, sentinel) # Get next object or sentinel
while x is not sentinel: # Keep going until we exhaust iterator
...
if shouldredo:
continue
...
if shouldcontinue:
x = next(iterobj, sentinel) # Explicitly advance loop for continue case
continue
...
if shouldbreak:
break
...
# Advance loop
x = next(iterobj, sentinel)
The above could also be done with a try/except StopIteration: instead of two-arg next with a sentinel, but wrapping the whole loop with it risks other sources of StopIteration being caught, and doing it at a limited scope properly for both inner and outer next calls would be extremely ugly (much worse than the sentinel based approach).
No, it doesn't. I would suggest using a while loop and resetting your check variable to the initial value.
count = 0
reset = 0
while count < 9:
print 'The count is:', count
if not someResetCondition:
count = count + 1
This is my solution using iterators:
class redo_iter(object):
def __init__(self, iterable):
self.__iterator = iter(iterable)
self.__started = False
self.__redo = False
self.__last = None
self.__redone = 0
def __iter__(self):
return self
def redo(self):
self.__redo = True
#property
def redone(self):
return self.__redone
def __next__(self):
if not (self.__started and self.__redo):
self.__started = True
self.__redone = 0
self.__last = next(self.__iterator)
else:
self.__redone += 1
self.__redo = False
return self.__last
# Display numbers 0-9.
# Display 0,3,6,9 doubled.
# After a series of equal numbers print --
iterator = redo_iter(range(10))
for i in iterator:
print(i)
if not iterator.redone and i % 3 == 0:
iterator.redo()
continue
print('---')
Needs explicit continue
redone is an extra feature
For Python2 use def next(self) instead of def __next__(self)
requires iterator to be defined before the loop
I just meet the same question when I study perl,and I find this page.
follow the book of perl:
my #words = qw(fred barney pebbles dino wilma betty);
my $error = 0;
my #words = qw(fred barney pebbles dino wilma betty);
my $error = 0;
foreach (#words){
print "Type the word '$_':";
chomp(my $try = <STDIN>);
if ($try ne $_){
print "Sorry - That's not right.\n\n";
$error++;
redo;
}
}
and how to achieve it on Python ??
follow the code:
tape_list=['a','b','c','d','e']
def check_tape(origin_tape):
errors=0
while True:
tape=raw_input("input %s:"%origin_tape)
if tape == origin_tape:
return errors
else:
print "your tape %s,you should tape %s"%(tape,origin_tape)
errors += 1
pass
all_error=0
for char in tape_list:
all_error += check_tape(char)
print "you input wrong time is:%s"%all_error
Python has not the "redo" syntax,but we can make a 'while' loop in some function until get what we want when we iter the list.
Not very sophiscated but easy to read, using a while and an increment at the end of the loop. So any continue in between will have the effect of a redo. Sample to redo every multiple of 3:
redo = True # To ends redo condition in this sample only
i = 0
while i<10:
print(i, end='')
if redo and i % 3 == 0:
redo = False # To not loop indifinively in this sample
continue # Redo
redo = True
i += 1
Result: 00123345667899
There is no redo in python.
A very understandable solution is as follow:
for x in mylist:
redo = True
while redo:
redo = False
If should_redo:
redo = True
It's clear enough to do not add comments
Continue will work as if it was in the for loop
But break is not useable, this solution make break useable but the code is less clear.
Here is a solution for python 3.8+ since now we have the := operator:
for key in mandatory_attributes: # example with a dictionary
while not (value := input(f"{key} (mandatory): ")):
print("You must enter a value")
mandatory_attributes[key] = value
Have a look at the code:
def main():
for p in test1(): print(p)
def test1():
s = set()
s.update(range(5))
for p in s: yield p
return s
Why I got only 0,1,2,3,4? The output should be: 0,1,2,3,4 two times (1 for 'yield' and 1 for 'return')
PS: Python-3.4
A return statement in a Python 3.3+ generator doesn't do what you think it does. The value is not returned to the caller like in a normal function, but added as an attribute on the StopIteration exception the generator raises to signal that it is done iterating. The behavior you're seeing in your loop is unrelated.
First, lets understand the loop behavior. This comes down to a simple fact: The loop variable (e.g. i) doesn't go out of scope when a for loop ends:
for i in range(5): # this loop will print 0 through 4
print(i)
print(i) # this line will print 4 again, since 4 it was the last value assigned to i
Your code is doing exactly this. The else clause you're using does nothing special, since you never break out of the loop. (Neftas's answer explains what an else attached to a loop is for.)
As for where the return value is going, you can find it if you iterate over your generator manually:
gen = test1()
print(next(gen)) # prints 0
print(next(gen)) # prints 1
print(next(gen)) # prints 2
print(next(gen)) # prints 3
print(next(gen)) # prints 4
print(next(gen)) # prints set([0,1,2,3,4]) from the last yield statement
try:
next(gen)
except StopIteration as e:
print(e.value) # prints set([0,1,2,3,4]) from the return statement
This isn't a very common usage. The usual way of getting at the returned value is by using the result of a yield from expression in another generator:
def test3():
print(yield from test1())
This is a generator that yields all the same values as test1, but it also prints out the value that test1 returns.
I don't think the return idiom is terribly useful in most situations. yield from can be very useful in recursive or otherwise complex generators, but I've never found a need to return a value from one.
If you want more information about the yield from expression and the return value from generators, read PEP 380, which describes the new features that were added in Python 3.3.
For a discussion of return in a generator, see Blckknght's answer, this is about the else clause in a for loop.
I read a nice article about else clauses in for loops if you don't like the documentation.
The gist of it is that an else clause in a for loop is all about completion, rather than about conditionals. Compare:
if 1:
print "True"
else:
print "False"
Here, the else clause is executed when the comparison falls through.
But:
for i in xrange(5):
if i == 123:
print "Found it!"
break
else:
print "Value not in list"
# output: "Value not in list"
Here, the else clause gets executed unless the flow of execution hits the break statement:
for i in xrange(5):
if i == 4:
print "Found it!"
break
else:
print "Value not in list"
# output: "Found it!"
If your remove the break, both strings will be printed. In your code the flow of execution will always reach the else statement, so the code there is run.
What exactly happens, when yield and return are used in the same function in Python, like this?
def find_all(a_str, sub):
start = 0
while True:
start = a_str.find(sub, start)
if start == -1: return
yield start
start += len(sub) # use start += 1 to find overlapping matches
Is it still a generator?
Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.
PEP 255 spells it out:
Specification: Return
A generator function can also contain return statements of the form:
"return"
Note that an expression_list is not allowed on return statements in
the body of a generator (although, of course, they may appear in the
bodies of non-generator functions nested within the generator).
When a return statement is encountered, control proceeds as in any
function return, executing the appropriate finally clauses (if any
exist). Then a StopIteration exception is raised, signalling that the
iterator is exhausted. A StopIteration exception is also raised if
control flows off the end of the generator without an explict return.
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
Note that return isn't always equivalent to raising StopIteration:
the difference lies in how enclosing try/except constructs are
treated. For example,
>>> def f1():
... try:
... return
... except:
... yield 1
>>> print list(f1())
[]
because, as in any function, return simply exits, but
>>> def f2():
... try:
... raise StopIteration
... except:
... yield 42
>>> print list(f2())
[42]
because StopIteration is captured by a bare "except", as is any
exception.
Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see #NPE's answer for details).
Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.
As pointed out by #BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.
From PEP 380:
In a generator, the statement
return value
is semantically equivalent to
raise StopIteration(value)
There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.
It probably is not as clean as you would want but it does do what you expect.
Here's an example:
def six(how_many=None):
if how_many is None or how_many < 1:
return None # returns value
if how_many == 1:
return 6 # returns value
def iter_func():
for count in range(how_many):
yield 6
return iter_func() # returns generator
Note: you don't get StopIteration exception with the example below.
def odd(max):
n = 0
while n < max:
yield n
n = n + 1
return 'done'
for x in odd(3):
print(x)
The for loop catches it. That's its signal to stop
But you can catch it in this way:
g = odd(3)
while True:
try:
x = next(g)
print(x)
except StopIteration as e:
print("g return value:", e.value)
break
I'm calling a bunch of methods that return a list. The list may be empty. If the list is non-empty, I want to return the first item; otherwise, I want to return None. This code works:
def main():
my_list = get_list()
if len(my_list) > 0:
return my_list[0]
return None
but it seems to me that there should be a simple one-line idiom for doing this. Is there?
Python 2.6+
next(iter(your_list), None)
If your_list can be None:
next(iter(your_list or []), None)
Python 2.4
def get_first(iterable, default=None):
if iterable:
for item in iterable:
return item
return default
Example:
x = get_first(get_first_list())
if x:
...
y = get_first(get_second_list())
if y:
...
Another option is to inline the above function:
for x in get_first_list() or []:
# process x
break # process at most one item
for y in get_second_list() or []:
# process y
break
To avoid break you could write:
for x in yield_first(get_first_list()):
x # process x
for y in yield_first(get_second_list()):
y # process y
Where:
def yield_first(iterable):
for item in iterable or []:
yield item
return
The best way is this:
a = get_list()
return a[0] if a else None
You could also do it in one line, but it's much harder for the programmer to read:
return (get_list()[:1] or [None])[0]
(get_list() or [None])[0]
That should work.
BTW I didn't use the variable list, because that overwrites the builtin list() function.
The most python idiomatic way is to use the next() on a iterator since list is iterable. just like what #J.F.Sebastian put in the comment on Dec 13, 2011.
next(iter(the_list), None) This returns None if the_list is empty. see next() Python 2.6+
or if you know for sure the_list is not empty:
iter(the_list).next() see iterator.next() Python 2.2+
If you find yourself trying to pluck the first thing (or None) from a list comprehension you can switch to a generator to do it like:
next((x for x in blah if cond), None)
Pro: works if blah isn't indexable Con: it's unfamiliar syntax. It's useful while hacking around and filtering stuff in ipython though.
The OP's solution is nearly there, there are just a few things to make it more Pythonic.
For one, there's no need to get the length of the list. Empty lists in Python evaluate to False in an if check. Just simply say
if list:
Additionally, it's a very Bad Idea to assign to variables that overlap with reserved words. "list" is a reserved word in Python.
So let's change that to
some_list = get_list()
if some_list:
A really important point that a lot of solutions here miss is that all Python functions/methods return None by default. Try the following below.
def does_nothing():
pass
foo = does_nothing()
print foo
Unless you need to return None to terminate a function early, it's unnecessary to explicitly return None. Quite succinctly, just return the first entry, should it exist.
some_list = get_list()
if some_list:
return list[0]
And finally, perhaps this was implied, but just to be explicit (because explicit is better than implicit), you should not have your function get the list from another function; just pass it in as a parameter. So, the final result would be
def get_first_item(some_list):
if some_list:
return list[0]
my_list = get_list()
first_item = get_first_item(my_list)
As I said, the OP was nearly there, and just a few touches give it the Python flavor you're looking for.
Python idiom to return first item or None?
The most Pythonic approach is what the most upvoted answer demonstrated, and it was the first thing to come to my mind when I read the question. Here's how to use it, first if the possibly empty list is passed into a function:
def get_first(l):
return l[0] if l else None
And if the list is returned from a get_list function:
l = get_list()
return l[0] if l else None
New in Python 3.8, Assignment Expressions
Assignment expressions use the in-place assignment operator (informally called the walrus operator), :=, new in Python 3.8, allows us to do the check and assignment in-place, allowing the one-liner:
return l[0] if (l := get_list()) else None
As a long-time Python user, this feels like we're trying to do too much on one line - I feel it would be better style to do the presumptively equally performant:
if l := get_list():
return l[0]
return None
In support of this formulation is Tim Peter's essay in the PEP proposing this change to the language. He didn't address the first formulation, but based on the other formulations he did like, I don't think he would mind.
Other ways demonstrated to do this here, with explanations
for
When I began trying to think of clever ways to do this, this is the second thing I thought of:
for item in get_list():
return item
This presumes the function ends here, implicitly returning None if get_list returns an empty list. The below explicit code is exactly equivalent:
for item in get_list():
return item
return None
if some_list
The following was also proposed (I corrected the incorrect variable name) which also uses the implicit None. This would be preferable to the above, as it uses the logical check instead of an iteration that may not happen. This should be easier to understand immediately what is happening. But if we're writing for readability and maintainability, we should also add the explicit return None at the end:
some_list = get_list()
if some_list:
return some_list[0]
slice or [None] and select zeroth index
This one is also in the most up-voted answer:
return (get_list()[:1] or [None])[0]
The slice is unnecessary, and creates an extra one-item list in memory. The following should be more performant. To explain, or returns the second element if the first is False in a boolean context, so if get_list returns an empty list, the expression contained in the parentheses will return a list with 'None', which will then be accessed by the 0 index:
return (get_list() or [None])[0]
The next one uses the fact that and returns the second item if the first is True in a boolean context, and since it references my_list twice, it is no better than the ternary expression (and technically not a one-liner):
my_list = get_list()
return (my_list and my_list[0]) or None
next
Then we have the following clever use of the builtin next and iter
return next(iter(get_list()), None)
To explain, iter returns an iterator with a .next method. (.__next__ in Python 3.) Then the builtin next calls that .next method, and if the iterator is exhausted, returns the default we give, None.
redundant ternary expression (a if b else c) and circling back
The below was proposed, but the inverse would be preferable, as logic is usually better understood in the positive instead of the negative. Since get_list is called twice, unless the result is memoized in some way, this would perform poorly:
return None if not get_list() else get_list()[0]
The better inverse:
return get_list()[0] if get_list() else None
Even better, use a local variable so that get_list is only called one time, and you have the recommended Pythonic solution first discussed:
l = get_list()
return l[0] if l else None
Regarding idioms, there is an itertools recipe called nth.
From itertools recipes:
def nth(iterable, n, default=None):
"Returns the nth item or a default value"
return next(islice(iterable, n, None), default)
If you want one-liners, consider installing a library that implements this recipe for you, e.g. more_itertools:
import more_itertools as mit
mit.nth([3, 2, 1], 0)
# 3
mit.nth([], 0) # default is `None`
# None
Another tool is available that only returns the first item, called more_itertools.first.
mit.first([3, 2, 1])
# 3
mit.first([], default=None)
# None
These itertools scale generically for any iterable, not only for lists.
for item in get_list():
return item
Frankly speaking, I do not think there is a better idiom: your is clear and terse - no need for anything "better". Maybe, but this is really a matter of taste, you could change if len(list) > 0: with if list: - an empty list will always evaluate to False.
On a related note, Python is not Perl (no pun intended!), you do not have to get the coolest code possible.
Actually, the worst code I have seen in Python, was also very cool :-) and completely unmaintainable.
By the way, most of the solution I have seen here do not take into consideration when list[0] evaluates to False (e.g. empty string, or zero) - in this case, they all return None and not the correct element.
my_list[0] if len(my_list) else None
Not sure how pythonic this is but until there is a first function in the library I include this in the source:
first = lambda l, default=None: next(iter(l or []), default)
It's just one line (conforms to black) and avoids dependencies.
Out of curiosity, I ran timings on two of the solutions. The solution which uses a return statement to prematurely end a for loop is slightly more costly on my machine with Python 2.5.1, I suspect this has to do with setting up the iterable.
import random
import timeit
def index_first_item(some_list):
if some_list:
return some_list[0]
def return_first_item(some_list):
for item in some_list:
return item
empty_lists = []
for i in range(10000):
empty_lists.append([])
assert empty_lists[0] is not empty_lists[1]
full_lists = []
for i in range(10000):
full_lists.append(list([random.random() for i in range(10)]))
mixed_lists = empty_lists[:50000] + full_lists[:50000]
random.shuffle(mixed_lists)
if __name__ == '__main__':
ENV = 'import firstitem'
test_data = ('empty_lists', 'full_lists', 'mixed_lists')
funcs = ('index_first_item', 'return_first_item')
for data in test_data:
print "%s:" % data
for func in funcs:
t = timeit.Timer('firstitem.%s(firstitem.%s)' % (
func, data), ENV)
times = t.repeat()
avg_time = sum(times) / len(times)
print " %s:" % func
for time in times:
print " %f seconds" % time
print " %f seconds avg." % avg_time
These are the timings I got:
empty_lists:
index_first_item:
0.748353 seconds
0.741086 seconds
0.741191 seconds
0.743543 seconds avg.
return_first_item:
0.785511 seconds
0.822178 seconds
0.782846 seconds
0.796845 seconds avg.
full_lists:
index_first_item:
0.762618 seconds
0.788040 seconds
0.786849 seconds
0.779169 seconds avg.
return_first_item:
0.802735 seconds
0.878706 seconds
0.808781 seconds
0.830074 seconds avg.
mixed_lists:
index_first_item:
0.791129 seconds
0.743526 seconds
0.744441 seconds
0.759699 seconds avg.
return_first_item:
0.784801 seconds
0.785146 seconds
0.840193 seconds
0.803380 seconds avg.
try:
return a[0]
except IndexError:
return None
def head(iterable):
try:
return iter(iterable).next()
except StopIteration:
return None
print head(xrange(42, 1000) # 42
print head([]) # None
BTW: I'd rework your general program flow into something like this:
lists = [
["first", "list"],
["second", "list"],
["third", "list"]
]
def do_something(element):
if not element:
return
else:
# do something
pass
for li in lists:
do_something(head(li))
(Avoiding repetition whenever possible)
Borrowing more_itertools.first_true code yields something decently readable:
def first_true(iterable, default=None, pred=None):
return next(filter(pred, iterable), default)
def get_first_non_default(items_list, default=None):
return first_true(items_list, default, pred=lambda x: x!=default)
Following code covers several scenarios by using lambda:
l1 = [1,2,3]
l2 = []
l3 = None
first_elem = lambda x: x[0] if x else None
print(first_elem(l1))
print(first_elem(l2))
print(first_elem(l3))
Using the and-or trick:
a = get_list()
return a and a[0] or None
Probably not the fastest solution, but nobody mentioned this option:
dict(enumerate(get_list())).get(0)
if get_list() can return None you can use:
dict(enumerate(get_list() or [])).get(0)
Advantages:
-one line
-you just call get_list() once
-easy to understand
My use case was only to set the value of a local variable.
Personally I found the try and except style cleaner to read
items = [10, 20]
try: first_item = items[0]
except IndexError: first_item = None
print first_item
than slicing a list.
items = [10, 20]
first_item = (items[:1] or [None, ])[0]
print first_item
How about this:
(my_list and my_list[0]) or None
Note: This should work fine for lists of objects but it might return incorrect answer in case of number or string list per the comments below.
You could use Extract Method. In other words extract that code into a method which you'd then call.
I wouldn't try to compress it much more, the one liners seem harder to read than the verbose version. And if you use Extract Method, it's a one liner ;)
Several people have suggested doing something like this:
list = get_list()
return list and list[0] or None
That works in many cases, but it will only work if list[0] is not equal to 0, False, or an empty string. If list[0] is 0, False, or an empty string, the method will incorrectly return None.
I've created this bug in my own code one too many times !
isn't the idiomatic python equivalent to C-style ternary operators
cond and true_expr or false_expr
ie.
list = get_list()
return list and list[0] or None
if mylist != []:
print(mylist[0])
else:
print(None)