I write the following code in Python:
def fib(n):
a, b = 0, 1
while b < n:
print(b, end=' ')
a, b = b, a + b
print(fib, fib(10))
I think the correct output should be:
<function fib at 0x000001DF4BB13E18> 1 1 2 3 5 8
But The output is:
1 1 2 3 5 8 <function fib at 0x000001C60F823E18> None
The code prints None and its behavior is odd.
Why behavior of print function is odd?
The behavior is correct. It's just that fib(10) is executed before you call print. That's because the arguments for the function call have to be executed before they are passed to the function.
So in fact you do calculate fib(10) (including all the prints inside the function) and then print fib and the result from the fib(10) call (which is None because your fib function contains no explicit return).
You can also invoke dis.dis to see the order of evaluation:
def f():
print(fib, fib(10))
import dis
dis.dis(f)
Result:
10 0 LOAD_GLOBAL 0 (print)
2 LOAD_GLOBAL 1 (fib)
4 LOAD_GLOBAL 1 (fib)
6 LOAD_CONST 1 (10)
8 CALL_FUNCTION 1 ----> the fib(10) call
10 CALL_FUNCTION 2 ----> the print call
^------- number of arguments!
12 POP_TOP
14 LOAD_CONST 0 (None)
16 RETURN_VALUE
Both arguments to print() are fully evaluated before print() is actually called. Since the second argument to your main print() is fib(10), fib is called with argument 10 during this process. And since it prints things, these get printed during the evaluation process. The function itself doesn't contain a return statement, so it returns None, which is why that's printed.
Let's break this down.
First we'll look at the ordering:
print(fib, fib(10))
The call to fib will be evaluated first and pass its returned value into the print function.
fib(10) will do some printing of it's own and then exit. In this case it doesn't explicitly return a value and so gets treated as None
The above call to print can therefore be seen as
print(fib, None)
Which will result in printing the function address and then None
Related
Given the following example
try:
a, b = 0, 0
for _ in range(100):
a, b = (a+1, b+1)
except KeyboardInterrupt:
assert a == b
could an AssertionError be thrown? If so, is there a way to prevent it, i.e. to ensure that either both of a and b or none is updated on every iteration?
Possibly related to Is Python unpacking thread safe?
Within, the following example and corresponding opcode are given:
>>> def t(self): a,b=20,20
...
>>> dis.dis(t)
1 0 LOAD_CONST 2 ((20, 20))
3 UNPACK_SEQUENCE 2
6 STORE_FAST 1 (a)
9 STORE_FAST 2 (b)
12 LOAD_CONST 0 (None)
15 RETURN_VALUE
Since there are two separate instructions for storing a and b, I would expect that no guarantees can be given whether both or none of the two instructions is executed prior to a KeyboardInterrupt.
Your intuition is correct: while Python will handle the interrupt internally and re-expose it separately (so interrupts are not quite as fraught as they are in C), as noted in e.g. PEP 343:
Even if you write bug-free code, a KeyboardInterrupt exception can still cause it to exit between any two virtual machine opcodes.
I am researching the lambda function, and I came across this code sample:
def myfunc(n):
return lambda a : a * n
tripler = myfunc(3)
print(tripler(10)) # Output: 30
I understand that when tripler is assigned to the return value from the myfunc function, the value of 3 is passed as an argument and from within the function, it is represented as 'n'. But lambda uses a bound variable labeled 'a'. So at this point, ignoring the final print line, I'm not sure how 'a' is factored into this. When I run this code, I receive no errors.
I would be interested in understanding how this code is processed and how it comes up with its conclusion without generating errors.
The return value of myfunc is a closure, which retains a reference to the argument passed to myfunc.
>>> x = myfunc(3)
>>> x.__closure__[0].cell_contents
3
When you call x, the value is used.
>>> x(10) # (lambda a: a * n)(10), with n == 3
30
You can see this in the byte code generated for x:
>>> import dis
>>> dis.dis(x)
2 0 LOAD_FAST 0 (a)
2 LOAD_DEREF 0 (n)
4 BINARY_MULTIPLY
6 RETURN_VALUE
LOAD_FAST references the local variables (in this case, the only local variable is the parameter a), while LOAD_DEREF references the closure. The argument 0 is an index into the tuple stored in x.__closure__.
def fib(n):
if n <= 1:
return n
else:
return fib(n - 2) + fib(n - 1)
def count(f):
def counted(*args):
counted.call_count += 1
return f(*args)
counted.call_count = 0
return counted
>>> fib = count(fib)
>>> fib(5)
5
>>> fib.call_count
15
>>> counted.call_count
NameError
I understand that fib is now actually counted. However, I cannot figure out why, when I want to call counted.call_count, I have to call fib.call_count.
I think that counted is an instance of the class Function. But what does counted.call_count exactly mean?
More specifically, I do not understand an attribute defined outside the code class ****:. In that case defining an attribute is like self.call_count is clearer, because I can understand self to refer to the instance. In my example, it seems that counted in counted.call_count is just like self in self.call_count, but is it?
The issue here is that the counted function isn't called counted in your main program code, it's called fib. (count returns counted, which you bind to fib with an assignment), so that's the name you should use.
Your reliance on the decorated function name is a little tenuous. You will observe that you rely on that name. If you change the calling code to read
fibfun = count(fib)
fibfun(5)
fibfun.call_count
you will see it only counts the outer call.
You are in fact writing a decorator, though you may not know it. A more conventional way to write this might be
def count(f):
def counted(*args):
counted.call_count += 1
return f(*args)
counted.call_count = 0
return counted
#count
def fib(n):
if n <= 1:
return n
else:
return fib(n - 2) + fib(n - 1)
There's then no need to wrap the function manually (the decorator syntax simply does the call for you), and you can just write
>>> fib(5)
5
>>> fib.call_count
15
The problem is that you can only call this decorated function once, unless you take into account that the count continues to accumulate as further calls are made:
>>> fib(5)
5
>>> fib.call_count
30
This may be acceptable. Otherwise, further complexity may be required.
Thinking about how the python interpreter works under the hood may help you understand what's going on here. When source code is passed through the lexer, and turned into bytecode we can see references to "load" and "store" commands indicating that the interpreter works primarily as a stack based machine:
>>> dis.dis(count)
2 0 LOAD_CLOSURE 0 (counted)
2 LOAD_CLOSURE 1 (f)
4 BUILD_TUPLE 2
6 LOAD_CONST 1 (<code object counted at 0x00000193D034C8A0, file "<ipython-input-2-a31b910d61de>", line 2>)
8 LOAD_CONST 2 ('count.<locals>.counted')
10 MAKE_FUNCTION 8
12 STORE_DEREF 0 (counted)
5 14 LOAD_CONST 3 (0)
16 LOAD_DEREF 0 (counted)
18 STORE_ATTR 0 (call_count)
6 20 LOAD_DEREF 0 (counted)
22 RETURN_VALUE
We can also see that the call_count attribute is bound to the new code object that is created, but the name: counted is explicitly de-referenced as it is no longer in scope and only the code object is returned.
Here's a simple file depicting some inconsistent Python (3.6) behavior. Why is it possible that Case 1 and Case 2 run but Case 3 fails, even though Case 3 is just a merger of the first two cases?
I have provided the dis output of the first two cases.
import dis # Python bytecode disassembler
class A(object):
def __init__(self):
self.x # In case 2 (and 3), getting x results in a function call (because they are #properties), which fails when instantiating A because y is undefined. Case 1 evaluates the reference to a function without calling it and so it does not raise an exception.
# CASE 1: Legal
def x(self):
y
pass
'''
# CASE 2: Legal
#property
def x(self):
pass
'''
'''
# CASE 3: Illegal:
#property
def x(self):
y
pass
'''
if __name__ == '__main__':
a = A()
dis.dis(A)
Case 1 bytecode:
Disassembly of __init__:
5 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (x)
4 POP_TOP
6 LOAD_CONST 0 (None)
8 RETURN_VALUE
Disassembly of x:
9 0 LOAD_GLOBAL 0 (y)
2 POP_TOP
10 4 LOAD_CONST 0 (None)
6 RETURN_VALUE
Case 2 bytecode:
Disassembly of __init__:
5 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (x)
4 POP_TOP
6 LOAD_CONST 0 (None)
8 RETURN_VALUE
There is no inconsistency here.
When you instantiate a = A(), __init__ is called, which calls self.x, which will execute the body of x. At that point, there is no y inscope, so you get an exception.
Thanks to #chepner's comment:
In case 1, you aren't calling anything; self.x is a function reference
that isn't used. In case 3, self.x actually calls the defined getter
for x, which presumably is then trying to access an undefined global
name.
The behavior caused by the line self.x in case 3 is fundamentally different from case 1 because case 1 doesn't call anything -- it just evaluates a reference to a function.
On the other hand, self.x in case 3 executes the body of the x method, resulting in the undefined y error.
In order to confirm #chepner's comment, I ran a.x() with case 1 and got the same error as in case 3.
A generator function can be defined by putting the yield keyword in the function’s body:
def gen():
for i in range(10):
yield i
How to define an empty generator function?
The following code doesn’t work, since Python cannot know that it is supposed to be a generator function instead of a normal function:
def empty():
pass
I could do something like this:
def empty():
if False:
yield
But that would be very ugly. Is there a nicer way?
You can use return once in a generator; it stops iteration without yielding anything, and thus provides an explicit alternative to letting the function run out of scope. So use yield to turn the function into a generator, but precede it with return to terminate the generator before yielding anything.
>>> def f():
... return
... yield
...
>>> list(f())
[]
I'm not sure it's that much better than what you have -- it just replaces a no-op if statement with a no-op yield statement. But it is more idiomatic. Note that just using yield doesn't work.
>>> def f():
... yield
...
>>> list(f())
[None]
Why not just use iter(())?
This question asks specifically about an empty generator function. For that reason, I take it to be a question about the internal consistency of Python's syntax, rather than a question about the best way to create an empty iterator in general.
If question is actually about the best way to create an empty iterator, then you might agree with Zectbumo about using iter(()) instead. However, it's important to observe that iter(()) doesn't return a function! It directly returns an empty iterable. Suppose you're working with an API that expects a callable that returns an iterable each time it's called, just like an ordinary generator function. You'll have to do something like this:
def empty():
return iter(())
(Credit should go to Unutbu for giving the first correct version of this answer.)
Now, you may find the above clearer, but I can imagine situations in which it would be less clear. Consider this example of a long list of (contrived) generator function definitions:
def zeros():
while True:
yield 0
def ones():
while True:
yield 1
...
At the end of that long list, I'd rather see something with a yield in it, like this:
def empty():
return
yield
or, in Python 3.3 and above (as suggested by DSM), this:
def empty():
yield from ()
The presence of the yield keyword makes it clear at the briefest glance that this is just another generator function, exactly like all the others. It takes a bit more time to see that the iter(()) version is doing the same thing.
It's a subtle difference, but I honestly think the yield-based functions are more readable and maintainable.
See also this great answer from user3840170 that uses dis to show another reason why this approach is preferable: it emits the fewest instructions when compiled.
iter(())
You don't require a generator. C'mon guys!
Python 3.3 (because I'm on a yield from kick, and because #senderle stole my first thought):
>>> def f():
... yield from ()
...
>>> list(f())
[]
But I have to admit, I'm having a hard time coming up with a use case for this for which iter([]) or (x)range(0) wouldn't work equally well.
Another option is:
(_ for _ in ())
Like #senderle said, use this:
def empty():
return
yield
I’m writing this answer mostly to share another justification for it.
One reason for choosing this solution above the others is that it is optimal as far as the interpreter is concerned.
>>> import dis
>>> def empty_yield_from():
... yield from ()
...
>>> def empty_iter():
... return iter(())
...
>>> def empty_return():
... return
... yield
...
>>> def noop():
... pass
...
>>> dis.dis(empty_yield_from)
2 0 LOAD_CONST 1 (())
2 GET_YIELD_FROM_ITER
4 LOAD_CONST 0 (None)
6 YIELD_FROM
8 POP_TOP
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
>>> dis.dis(empty_iter)
2 0 LOAD_GLOBAL 0 (iter)
2 LOAD_CONST 1 (())
4 CALL_FUNCTION 1
6 RETURN_VALUE
>>> dis.dis(empty_return)
2 0 LOAD_CONST 0 (None)
2 RETURN_VALUE
>>> dis.dis(noop)
2 0 LOAD_CONST 0 (None)
2 RETURN_VALUE
As we can see, the empty_return has exactly the same bytecode as a regular empty function; the rest perform a number of other operations that don’t change the behaviour anyway. The only difference between empty_return and noop is that the former has the generator flag set:
>>> dis.show_code(noop)
Name: noop
Filename: <stdin>
Argument count: 0
Positional-only arguments: 0
Kw-only arguments: 0
Number of locals: 0
Stack size: 1
Flags: OPTIMIZED, NEWLOCALS, NOFREE
Constants:
0: None
>>> dis.show_code(empty_return)
Name: empty_return
Filename: <stdin>
Argument count: 0
Positional-only arguments: 0
Kw-only arguments: 0
Number of locals: 0
Stack size: 1
Flags: OPTIMIZED, NEWLOCALS, GENERATOR, NOFREE
Constants:
0: None
The above disassembly is outdated as of CPython 3.11, but empty_return still comes out on top, with only two more opcodes (four bytes) than a no-op function:
>>> dis.dis(empty_yield_from)
1 0 RETURN_GENERATOR
2 POP_TOP
4 RESUME 0
2 6 LOAD_CONST 1 (())
8 GET_YIELD_FROM_ITER
10 LOAD_CONST 0 (None)
>> 12 SEND 3 (to 20)
14 YIELD_VALUE
16 RESUME 2
18 JUMP_BACKWARD_NO_INTERRUPT 4 (to 12)
>> 20 POP_TOP
22 LOAD_CONST 0 (None)
24 RETURN_VALUE
>>> dis.dis(empty_iter)
1 0 RESUME 0
2 2 LOAD_GLOBAL 1 (NULL + iter)
14 LOAD_CONST 1 (())
16 PRECALL 1
20 CALL 1
30 RETURN_VALUE
>>> dis.dis(empty_return)
1 0 RETURN_GENERATOR
2 POP_TOP
4 RESUME 0
2 6 LOAD_CONST 0 (None)
8 RETURN_VALUE
>>> dis.dis(noop)
1 0 RESUME 0
2 2 LOAD_CONST 0 (None)
4 RETURN_VALUE
Of course, the strength of this argument is very dependent on the particular implementation of Python in use; a sufficiently smart alternative interpreter may notice that the other operations amount to nothing useful and optimise them out. However, even if such optimisations are present, they require the interpreter to spend time performing them and to safeguard against optimisation assumptions being broken, like the iter identifier at global scope being rebound to something else (even though that would most likely indicate a bug if it actually happened). In the case of empty_return there is simply nothing to optimise, as bytecode generation stops after a return statement, so even the relatively naïve CPython will not waste time on any spurious operations.
Must it be a generator function? If not, how about
def f():
return iter(())
The "standard" way to make an empty iterator appears to be iter([]).
I suggested to make [] the default argument to iter(); this was rejected with good arguments, see http://bugs.python.org/issue25215
- Jurjen
I want to give a class based example since we haven't had any suggested yet. This is a callable iterator that generates no items. I believe this is a straightforward and descriptive way to solve the issue.
class EmptyGenerator:
def __iter__(self):
return self
def __next__(self):
raise StopIteration
>>> list(EmptyGenerator())
[]
generator = (item for item in [])
Nobody has mentioned it yet, but calling the built-in function zip with no arguments returns an empty iterator:
>>> it = zip()
>>> next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration