Consider this code
class A:
class B:
foo = 1
bar = {x: B for x in range(5)}
This will produce an error saying that B is not defined. However, when stopping with the debugger at the corresponding line, on the REPL, I can run that code just fine. This appears to be some nesting / scoping problem, but I don't understand it.
Can someone explain what's going on and maybe how to work around this?
A dict comprehension implicitly defines an anonymous function. Within the body of that function, B is a free variable. The variable lookup rules state that the value of B is taken from the closest enclosing scope that defines B, but a class statement doesn't define a scope (though it does define a temporary namespace that is similar to a proper scope). The next scope up from the anonymous function's local scope is the global scope, where B is not defined.
To avoid the binding issue, define a function which takes B as an argument, and have that function execute the dict comprehension:
bar = (lambda cls: {x: cls for x in range(5)})(B)
Now, B is can be found in the current namespace of the class statement (scoping rules are avoided), and cls, being defined in the scope of the anonymous function defined by the lambda expression, can now be found in the stack of scopes used by the dict comprehension. Put another way, the anonymous function created by the comprehension is a closure over the scope of the lambda expression's function.
That said, nested classes aren't really common in Python; you could define B at the global scope to solve the problem in this case.
Related
I'm trying to wrap my head around the scope of Lambda functions. I noticed that I can create a lambda function in one module A and pass it to a function in another module B, but able to call functions from module A .
Is this bad practice to pass lambda functions around like this, or is there a more preferred (Best Practice) method for handling this?
target.py
class TestLambda():
def __init__(self,name):
self.name = name
def call(self,func):
func(self.name)
source.py
from target import TestLambda
def sayHello(name):
print("Hello {}".format(name))
func = lambda n: sayHello(n)
l = TestLambda("John")
l.call(func)
output
➜ lambda-test python3 source.py
Hello John
The key here is that every function object keeps a reference to the scope in which the function was defined.
>>> func.__globals__['sayHello']
<function sayHello at 0x1085f2680>
This is what lets func still call sayHello even when called from a function with a different global scope. Name lookups are static; they depend only on the lexical (static) context in which the names appear, not the runtime (dynamic) context. There are languages (Lisp, shell, etc) that use dynamic scoping; lexical or static scoping is widely regarded as easier to reason about, though.
I don't think there are any issues with creating and passing lambda functions around, as opposed to using
def func(name):
but I don't see the point in defining a function as a lambda expression if you are going to use it in a different module.
The result is the same, the only difference is that you're not consistent with your function definitions.
The Python docs specifically discourage this:
Always use a def statement instead of an assignment statement that
binds a lambda expression directly to an identifier.
Yes:
def f(x): return 2*x
No:
f = lambda x: 2*x
The first form means that the name of the resulting
function object is specifically 'f' instead of the generic ''.
This is more useful for tracebacks and string representations in
general. The use of the assignment statement eliminates the sole
benefit a lambda expression can offer over an explicit def statement
(i.e. that it can be embedded inside a larger expression)
def decorator(func):
def returning():
var = 1
func()
print(var)
return(returning)
#decorator
def function():
nonlocal var
var = 5
function()
var is declared inside the returning() function before calling func(), yet I get a binding error.
I don't understand why this happens.
Python determines scopes at compile time, making the scope model static, not dynamic. The nonlocal and global statements tell the compiler to alter the scope where a name is set. nonlocal tells the compiler that a given name is to be assigned to as a closure, living in an enclosing scope. See the Naming and binding section of the Python execution model documentation:
If a name is bound in a block, it is a local variable of that block, unless declared as nonlocal or global.
and
Each assignment or import statement occurs within a block defined by a class or function definition or at the module level (the top-level code block).
and
A scope defines the visibility of a name within a block. If a local variable is defined in a block, its scope includes that block. If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name.
So only function definitions, class definitions and the module level are blocks where assignments take place. nonlocal can only act on names in a nested scope:
The nonlocal statement causes corresponding names to refer to previously bound variables in the nearest enclosing function scope.
function() is not a nested block, there is no enclosing function scope.
Decorators are a runtime feature, and do not produce a new encloding function scope. You didn't nest function() inside the decorator, you only passed in a reference to the function object. The function has already been created, compilation is done and the scope of the names is set in stone.
The only way to do what you want would require re-compilation or bytecode manipulation, both subjects that are very much on the very advanced side of Python hacking.
For example, with access to the source code (usually the case), you could use inspect and ast to merge the abstract syntax tree of function into that of returning to create a nested scope, then compile that new tree into bytecode that would do what you want. Or you'd have to do something similar with the bytecode of both functions to make returning produce a closure, and function() take that closure for the value of var. This would require an intimate knowledge of how Python closures work, and what bytecode the compiler produces to handle closures.
All this means that it would be much easier to find yourself a different approach to your problem. Perhaps use a class with state to alter the state in different contexts, etc.
Here is the code:
def caller(callee):
callee()
def wrapper():
def a():
print v0
for i in range(5):
v0 = i*i
caller(a)
wrapper()
The above code outputs:
0
1
4
9
16
But I don't understand how a resolves the variable v0. I can not find the related python docs regarding this language feature.
The scope of variables defined in a function includes all the nested functions within it. So variables defined in wrapper() are accessible within a(), because this function is nested in it. This is known as lexical scoping, and it's often used in creating closures.
This is explained in the Python Resolution of Names documentation:
A scope defines the visibility of a name within a block. If a local variable is defined in a block, its scope includes that block. If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name.
The function a is defined per invocation of wrapper. When a given instance of a is called, it looks up v0 within the context of its definition, which is the specific call to wrapper that defined it. By the time a has been called, v0 has been defined within wrapper, and a uses that definition of v0.
In this example, the invocation of wrapper that defines a is still active when a is called, but it need not be. In other words, wrapper could return, and a reference to a from that context could still be called. In this case it would be a closure. But that does not occur in this example.
Here's an example that does use closures:
def foo(x):
def bar():
return x
return bar
f1 = foo(123)
f2 = foo(456)
print(f1())
print(f2())
The output is:
123
456
When foo is called, it returns an instance of bar which evaluates x in the context in which foo was called. In the first invocation x is 123, and in the second it is 456. Those bindings persist even after the calls to foo have returned.
Remember that all code in a function is not run until the function is called. By the time a() is called, v0 has been defined in an outer scope, so it can recognize the variable and evaluate it.
I am using a list on which some functions works in my program. This is a shared list actually and all of my functions can edit it. Is it really necessary to define it as "global" in all the functions?
I mean putting the global keyword behind it in each function that uses it, or defining it outside of all the functions is enough without using the global word behind its definition?
When you assign a variable (x = ...), you are creating a variable in the current scope (e.g. local to the current function). If it happens to shadow a variable fron an outer (e.g. global) scope, well too bad - Python doesn't care (and that's a good thing). So you can't do this:
x = 0
def f():
x = 1
f()
print x #=>0
and expect 1. Instead, you need do declare that you intend to use the global x:
x = 0
def f():
global x
x = 1
f()
print x #=>1
But note that assignment of a variable is very different from method calls. You can always call methods on anything in scope - e.g. on variables that come from an outer (e.g. the global) scope because nothing local shadows them.
Also very important: Member assignment (x.name = ...), item assignment (collection[key] = ...), slice assignment (sliceable[start:end] = ...) and propably more are all method calls as well! And therefore you don't need global to change a global's members or call it methods (even when they mutate the object).
Yes, you need to use global foo if you are going to write to it.
foo = []
def bar():
global foo
...
foo = [1]
No, you can specify the list as a keyword argument to your function.
alist = []
def fn(alist=alist):
alist.append(1)
fn()
print alist # [1]
I'd say it's bad practice though. Kind of too hackish. If you really need to use a globally available singleton-like data structure, I'd use the module level variable approach, i.e.
put 'alist' in a module and then in your other modules import that variable:
In file foomodule.py:
alist = []
In file barmodule.py:
import foomodule
def fn():
foomodule.alist.append(1)
print foomodule.alist # [1]
LISTL = []
VAR1 = 0
def foo():
... VAR1 += 1
... return VAR1
...
On calling foo(), I get this error:
UnboundLocalError: local variable 'VAR1' referenced before assignment
However, consider the list LISTL
>>> def foo(x):
... LISTL.append(x)
... return LISTL
...
>>> foo(5)
[5]
This works as expected. The question is why the append on a list works but I can't change the int?
Also, is this the right way to declare a global in Python? (Right after the import statements)
The reason for this difference has to do with how Python namespaces the names. If you're inside a function definition (def foo():), and you ACCESS a name (VAR1 or LISTL), it will first search your local namespace, where it will find nothing, and then it will search the namespace of the module the function was defined in, all the way up to the global namespace until it finds a match, or fails.
However, ACCESSING a name, and ASSIGNING a name, are two different concepts. If you're again within your function definition, and you say VAR1 = 2, you're declaring a new variable with the new local name VAR1 inside the function. This makes sense if you consider that otherwise you would encounter all sorts of naming collisions if there was no such namespacing at work.
When you append to a list, you are merely ACCESSING the list, and then calling a method on it which happens to change its conceptual value. When you use do +=, you're actually ASSIGNING a value to a name.
If you would like to be able to assign values to names defined outside of the current namespace, you can use the global keyword. In that case, within your function, you would first say global VAR1, and from there the name VAR1 would be the name in the outer namespace, and any assignments to it would take effect outside of the function.
If you assign to a variable within a function, that variable is assumed to be local unless you declare it global.