This question already has answers here:
masking the built-in variable with its magic behavior?
(2 answers)
Closed 6 years ago.
In the python tutorial:
In interactive mode, the last printed expression is assigned to the variable _. This means that when you are using Python as a desk calculator, it is somewhat easier to continue calculations, for example:
>>> tax = 12.5 / 100
>>> price = 100.50
>>> price * tax
12.5625
>>> price + _
113.0625
>>> round(_, 2)
113.06
This variable should be treated as read-only by the user. Don’t explicitly assign a value to it — you would create an independent local variable with the same name masking the built-in variable with its magic behavior.
What is masking here?
In general what is meant by masking or mask variable in python?
THanks
A variable is masked if some other variable with the same name exists, which is preferred. An example is having a global variable x. If you're in a function which also has a local variable x, then the global won't be accessible, without special measures the local variable is preferred whenever you use identifier x inside the function.
It's as if you have two kids in your classroom both called Mary, one of which is closer, and will hear your voice earlier, so always gives the quickest answer. How to give the other girl a chance?
Overwriting/reassigning a previously named identifier in a new scope
let's look at this code for example
def f():
return 3
def g():
f = 5
f()
try:
print(g())
except TypeError:
print(f())
within the scope of g, the identifier f has been reassigned from a function that returns an integer, to just an integer. so when you try calling f inside of g you should get a type error because you can't call an int. however outside of g, f is still a function that returns 3
the interpreter has its own scope so when you assign _ to another value, it forgets it's previous functionality in replace of the new value, until you restart the interpreter, or in this special case delete it with the del keyword
The _ is built-in to the interpreter, it has special meaning. It only acts like a read-only variable. When you read from it, the interpreter replaces it with the last calculated value. If you do something like
_ = 1 + 1
You are actually creating a new variable named _ in the current namespace (try running locals()function before and after creating the variable). Since there is now a variable with the same name as the interpreter built-in _ that variable is accessed first when you do subsequent reads, thus hiding the meaning of the built-in, or masking it.
You can also do what you're doing with Python built-ins as well, such as defining a function parameter namedint. In the the local function namespace, if you tried to call theint() function on a value, it would attempt to use your parameterint as a callable because there is a local variable with that name masking the built-in function int. Some IDEs will warn you when you try to do something like this because in practice it is generally a bad idea to mask the name of a built-in like that because you're eliminating the behavior of that what that symbol is intended for.
Where making comes in its if you do something like
val = 20
def do_stuff(val):
# parameter masks the value defined in outer scope
print('Value of a inside function: {}'.f format(val))
do_stuff(10) # prints 10
This assigns a variable val a value of 20, but when the function do_stuff is called it had it's own local variable namedval. Accessingval from within the function will give you the value passed in, 10 in this case, not 20 from the outer variable because thea within the function masks the one in the outer scope.
Related
This question already has answers here:
Short description of the scoping rules?
(9 answers)
Closed 5 months ago.
I am confused about scope.
What, if anything, is the difference between assigning a variable 'within a function' and assigning one within an
indented block? I have read many places that if and try blocks do not create, or have, their own scope, but I have also
read that the scope of a variable is the innermost block where it is defined.
I googled but was not able to find an example of nonlocal inside an if or try block.
def sixdig2iso(pathfrom):
os.chdir(pathfrom)
filelist = os.listdir()
nonlocal xfrs
xfrs = 0
PyCharm says nonlocal variable 'xfrs' must be bound in an outer function scope
Isn't this the outermost part of this function? Then what's the problem?
Is the outermost part of this function != an outer function? Even if the scopes of each are different from the inner parts of those same functions?!
if xfrs == 0:
restofit = frs[1:]
try:
convert = datetime.strptime(mm, '%m%d%y')
except ValueError as e:
logger.info(f"For {filename}, mm is: {mm} - and the error is: {e}")
count_err += 1
ender = ' '.join(restofit)
fronter = str(convert.date())
PyCharm says the 2nd convert 'might' be used before assignment
I tried making an inner function
def sixdig2iso(pathfrom):
"""Converts six digit dates into proper datetime format in place."""
os.chdir(pathfrom)
filelist = os.listdir()
nonlocal xfrs
xfrs = 0
def blockscope():
But PyCharm gives me the same "nonlocal variable 'xfrs' must be bound in an outer function scope" warning.
UPDATE
My response is too long for a comment.
“We have to guess because you didn't provide a complete example”
I can never seem to get the balance between ‘not enough’ and ‘too much’ information for these questions. I probably didn’t think the part you say is missing was relevant, which goes to my understanding of the problem in the first place.
“ it doesn't matter how many nestings of function you create inside this function, nonlocal only looks outward.“
See, I didn’t know that. And I infer, then, it only looks ‘upward’, too, right?
“As .strptime() might fail with an exception, convert may end up not having been assigned a value, as you did not initialise convert outside the try block. “
Ok, good, that is very helpful. I didn’t realize (and PyCharm does not explain as you just did) this is what PyCharm was talking about. But this is also why I was confused about the ‘scope’ of a try block.
In the first example you give, xfrs is only defined in the function you've provided as an example. We have to guess because you didn't provide a complete example, but the message from PyCharm suggests that you did not define this function inside another function that already had such an identifier defined in it and there's no xfrs in the global scope either.
In the second example, you assign to convert in the try section of a try .. except block. As .strptime() might fail with an exception, convert may end up not having been assigned a value, as you did not initialise convert outside the try block. So, PyCharm is correct (again, we have to assume as you did not provide a complete example.
Finally, in the third example, you start defining a function inside the function, but still apply nonlocal to a variable in the sixdig2iso() scope - it doesn't matter how many nestings of function you create inside this function, nonlocal only looks outward.
A typical use of nonlocal is as #Carcigenicate provides in their link (some modifications here):
x = 0
def outer():
x = 1
def inner():
nonlocal x
x += 1
return x
return x, inner
v, f = outer()
print(x, v, f())
The function returned by outer() produces 2 when called, as its non-local variable x of inner() is the same as the local variable x of outer(), starting at 1 and having 1 added as the function is called.
You can tell that is what happens, as the value returned by outer() is 1, but calling the returned function f() returns 2. All the time, the global x is untouched.
Try changing nonlocal to global and you'll find the the result changes from:
0 1 2
To:
0 1 1
I hope that helps explain what is going on with nonlocal.
Does UnboundLocalError occur only when we try to use assignment operators (e.g. x += 5) on a non-local variable inside a function or there other cases? I tried using methods and functions (e.g. print) and it worked. I also tried to define a local variable (y) using a global variable (x)
(y = x + 5) and it worked too.
Yes - the presence of the assignment operator is what creates a new variable (of the same name) in the current scope. Calling mutating methods on the old object are not a problem, nor is doing something with that old value, since there's no question (if only a single assignment was ever used) which value you're talking about.
The concern here is not the modification of a value. The concern is the ambiguity of the variable used. This can also be solved using the global keyword, which specifically tells Python to use the global version, eliminating the ambiguity.
Remember also that Python variables (or globals) are sort of hoisted, like in JavaScript. Any variable used inside a specific scope is a variable in that scope from the beginning of that scope. That means a variable used inside a function is a variable in that scope from the start of the function, regardless of if it shows up half way through.
A really good reference for this is here. Some more specifics here.
This question already has answers here:
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
According to my programming language class, in a language that uses lexical scoping
The body of a function is evaluated in the environment where the
function is defined, not the environment where the function is called.
For example, SML follows this behavior:
val x = 1
fun myfun () =
x
val x = 10
val res = myfun() (* res is 1 since x = 1 when myfun is defined *)
On the other hand, Python does not follow this behavior:
x = 1
def myfun():
return x
x = 10
myfun() # 10 since x = 10 when myfun is called
So why is Python described as using lexical scoping?
Your Python myfun is using the x variable from the environment where it was defined, but that x variable now holds a new value. Lexical scoping means functions remember variables from where they were defined, but it doesn't mean they have to take a snapshot of the values of those variables at the time of function definition.
Your Standard ML code has two x variables. myfun is using the first variable.
In Python, just as in SML, or (modern) Lisp, the body of a function is evaluated in the environment where it was defined. So, all three languages are lexically scoped.
In Python and Lisp, environments are mutable. That is, you can assign a new value to an existing variable, and that mutates the environment the variable is part of. Any functions defined within that environment will be evaluated in that environment—which means they will see the new value of the variable.
In SML, environments are not mutable; the environment can't change, there is no new value, so there's no question of whether the function will see that new value.
The syntax can be a bit misleading. In ML, val x = 1 and val x = 10 both define a brand new variable. In Python, x = 1 and x = 10 are assignment statements—they reassign to an existing variable, only defining a new one if there wasn't one of that name yet. (You don't see this in Lisp, where, e.g., let and setq are pretty hard to confuse.)
By the way, a closure with mutable variables is functionally equivalent to a mutable object (in the OO sense), so this feature of Lisp (and Python) has traditionally been pretty important.
As a side note, Python actually has slightly special rules for the global namespace (and the builtins one above it), so you could argue that the code in your example technically isn't relying on lexical scoping. But if you put the whole thing inside a function and call that function, then it definitely is an example of lexical scoping, so the global issue really isn't that important here.
In addition to the responses of #abarnert and #user2357112, it may help you to consider an SML equivalent of the Python code:
val x = ref 1
fun myfun () = !x
val () = x := 10
val res = myfun ()
The first line declares a variable x that references an integer and sets the referenced cell to 1. The function body dereferences x to return the value in the referenced cell. The third line sets the referenced cell to 10. The function call in the fourth line now returns 10.
I used the awkward val () = _ syntax to fix an ordering. The declaration is added solely for its side effect on x. It is also possible to write:
val x = ref 1
fun myfun () = !x;
x := 10;
val res = myfun ()
The environment is immutable—notably x always points to the same memory cell—but some data structures (reference cells and arrays) are mutable.
Let me start with a rationale that helped me understand the matter (and write this answer).
There are two concepts that overlap each other in the evaluation of variables in Python that may create some confusion:
One is the (memory) lookup strategy Python deploys to evaluate a variable, the famous BGEL order (Built-in < Global < Enclosing < Local scopes).
The other is the scope in which an object (function, variable) is defined and/or evaluated.
As the OP notes, Python doesn't look lexically scoped when we can switch the value of our variable ("x"), or from the fact that we can define a function (using "x") without even declaring the variables therein a priori; E.g.,
> def f():
print(x)
> x = 1
> f()
1
What is lexical scope
So, what is lexical scope after all, and how does it work in Python?
Lexical scope means that an object's (memory) scope will that where it (i.e., the object) was defined.
Let's take a function "f()", for instance. A function has an inner scope and an outer scope. In a lexically scoped language (eg, Python), the outer scope of "f()" will be the scope where the function was defined. Always. Even if you evaluate your function (f()) anywhere else in your code, the outer scope of f() will always be that where it was defined.
So, when Python applies its BGEL symbols evaluation strategy, the "E" is where lexical scope comes into action: the scope where f() was defined.
Lexical scope at work
> def f():
print(x)
> def g(foo):
x = 99
foo()
print(x)
> x = 1
> g(f)
1
99
We then can see the recursive (BGEL) scope search strategy on evaluating variables is respecting the scopes local-and-above where the function 'f()' was defined.
I am new to Python so I am unsure about the difference in variable assignment before or after the function definition.
Specifically, the first example was adopted from Lutz's book.
def tester(start):
print("inside tester")
def nested(label):
print("inside nested")
print(label,nested.state)
nested.state += 1
print("done with nested")
nested.state = start
print("done with tester")
return nested
F = tester(0)
F('spam')
F.state
F.state
The objective of the code is to store the state information without using nonlocal.
I am unsure what nested.state means here. I am unsure because nested.state is used inside nested() function (i.e. nested.state +=1) and outside nested() function (i.e. nested.state = start).
I modified the code above to see whether Python accepts assigning variable after function declaration for nested() and to see whether there is any concept I am missing relating to function.variable call (i.e. nested.state call).
def tester(start):
def nested(label):
print(label, state)
state += 1 #replaced 'nested.state' with 'state' here
state = start #replaced 'nested.state' with 'state' here
return nested
F=tester(0)
F('spam')
F('ham')
Unfortunately, above code generates error local variable 'state' referenced before assignment. This tells me that I am missing some concept about function.variable (i.e. nested.state).
Can someone please help me understand three things:
I. why it is that the code with nested.state doesn't generate any error but state does?
II. what does nested.state mean? If nested.state is a mechanism to access function's variables, why is it that the call inside nested() function also uses nested.state and not state?
III. If nested.state is a mechanism to access variable inside function, then why is it that PyCharm fails to show state under dropdown when I type nested.?
I'd appreciate any help. I research SO, and couldn't find any explanation on such problems.
The reason the first code example worked is because it was assigning and referencing an attribute of the nested function object. The key concept to understand here, is that Python allows you to assign new, arbitrary attributes to objects - including functions:
>>> def func(a, b):
return a + b
>>> func(1, 2)
3
>>> func.attr = 5
>>> func.attr
5
The first code example takes advantage of this fact by using the nested function object to store the necessary state. This is the same concept as using any other object to store the state. It's particularly convenient, however, to use a function since it's readily available.
In the second example, a normal variable is used. Because of this, normal scoping rules apply which means simply that the state variable defined in tester is not the state variable being referenced in nested. Thus, an error is raised.
Actually, I think you're asking a question about scope in Python, ignoring your code, check this:
def scope_level_1():
variable = 'Scope_level_1'
def scope_level_2():
variable = 'Scope_level_2'
def core():
nonlocal variable
variable += '_TOUCHED_AND_MODIFIED_BY_CORE'
print(variable)
return core()
return scope_level_2()
scope_level_1()
# 'Scope_level_2_TOUCHED_AND_MODIFIED_BY_CORE'
Don't worry about the keyword nonlocal, treat it just as a declaring to make code more readable.
First, remember a += b is the same as a = a + b. So a must exist before getting to the +=.
Simply put, in the first example the function nested has an attribute called state (accessed by nested.state). It is an attribute, which means that once you tell nested that it has an attribute called state (you are doing this in line 9 when nested.state = start) it keep that attribute. So, in the first example nested.state exists when you get to the +=.
In the second example, you are declaring a variable called state in tester, and another variable called state in nested. The one in nested could be called potato for all that matters, because it is not the same variable. Therefore when you arrive to the +=, the variable state does not exist!
This question already has answers here:
Short description of the scoping rules?
(9 answers)
Closed 8 years ago.
>>> a = 10
>>> def f(x):
return x + a
>>> a = 3
>>> f(1)
According to my experience on Java, the definition of f contains a local variable a, but how could the global binding a been visible on the function f call stack environment?
I did a research on the python syntax and found that's true, could anybody offer some background on why python dealing variable scope this way? thanks.
Your function call is in the last line.
When the function gets called, python first looks up for local variables with name a,
if not found, it goes into global scope, and in global scope, the last assigned value of a is 3 ( just before the function was called)
What you may find even stranger is that this will also work:
>>> def f(x):
return x + a
>>> a = 3
>>> f(1)
Note that a hasn't even been defined before the function f. It still works because your call to f is after a is defined and placed in the global namespace. At that point, since f does not have a in its local namespace, it will fetch it from the global namespace.
You can fetch the contents of the global namespace and check for yourself with globals(), and the local namespace with locals(). There's also some neat tricks you can do by manipulating the namespaces directly, but that is in most cases considered bad coding practice in Python, unless you really have a compelling reason and know what you're doing.
It would return 4 because you declare a and f(x) a function then you give valuea=3 and then you give x=1 so the function would return 3+1 which is 4
Python decide variable scope in function based on where they have been assigned. As you didn't assigned variable 'a' inside function so it starts looking out and consider the global value.
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a new value anywhere within the function’s body, it’s assumed to be a local. If a variable is ever assigned a new value inside the function, the variable is implicitly local, and you need to explicitly declare it as global.
Ref : http://effbot.org/pyfaq/what-are-the-rules-for-local-and-global-variables-in-python.htm
Java is a purely object-oriented language, while Python is not. Python supports both structural as well as object-oriented paradigms.
Global variables are part of the structural programming paradigm. So global variables will be available in the scope of a function, unless another variable with exactly the same name exists in the local scope of that function.