Why mutable objects like array or list can be visited and changed in function directly while immutable objects like number can be only visited but not changed? What is the mechnism. Below is some simple test code.
import numpy as np
a = [1,2,3]
b = np.array([1,2,3])
c = 3
def func():
a.append(1)
b[0] = 2
c += 1
print(c)
func()
The difference is whether you assign or mutate. When you mutate data, like with a.append(1), the object reference (a) is not changed: it is still the same list reference. When you assign, the variable really gets a different reference, and the object that was previously referenced does not get affected.
Without global, globals can be mutated (when they are mutable), but not assigned.
This has little to do with mutable or not, as even a = [] would not be allowed without the corresponding global statement. Even though lists are mutable, this a = [] is not attempting to mutate anything. It assigns. And that requires global.
Well, you're kind of answering your own question.
Mutable objects are internally mutable, while immutable objects, well, aren't.
The global c (or nonlocal c in an inner function) statement tells Python that the name c refers to the name c in the global (or outer) scope.
Since integers aren't internally mutable (and don't implement __iadd__, the magic method that could back +=), c += 1 does c = c + 1, i.e. assigns the value of the expression c + 1 to the global name c.
If you attempt c += 1 in a function without the global statement, you'll get an UnboundLocalError, since there is no name c to access and increment before that assignment itself.
This stands for any name; trying to do
>>> a = []
>>> def f():
... a = a + ["foo"]
...
>>> f()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f
UnboundLocalError: local variable 'a' referenced before assignment
also fails, since a has not been assigned within the function.
Related
Consider the following cases:
Case 1:
def fun(arg):
arg += 1
my_var = 1
fun(my_var)
print(my_var)
>> 1
Case 2:
def fun(arg):
arg += [4]
my_var = [1,2,3]
fun(my_var)
print(my_var)
>> [1, 2, 3, 4]
Case 3:
def fun(arg):
arg = arg + [4]
my_var = [1,2,3]
fun(my_var)
print(my_var)
>> [1, 2, 3]
Case 4:
def fun(arg):
print(arg)
fun("Hi")
print(arg)
>> Hi
Traceback (most recent call last):
File "<string>", line 8, in <module>
NameError: name 'arg' is not defined
Case 4 demonstrates that the scope of the argument variable lies within the function. Case 1 and 3 support that as changes to the arg variable within the function are not reflected in the parameters globally.
But why does Case 2 happen at all? I noticed the same thing when I used append instead of the +=. Shouldn't the changes that happen to arg not have any effect on the variables the function was called with from a different scope?
Any help appreciated. Thanks in advance.
The answer is simple: Function arguments are local variables!
Some more observations:
= is the assignment operator and rebinds a variable. This will never mutate objects (ecxeption slice assignment).
+= is its own operator and is implemented as a mutation for mutable types. You can implement its behaviour for your own types by implementing the __iadd__ dunder method.
Hence, in general, a = a + b is not equivalent to a += b!
For lists, a += b corresponds to a.extend(b). a + b on the other hand, creates a new list object, and a = a + b rebinds variable a to that new object.
Case 2 is the only one where you use a mutating method on an instance. This affects the parameter passed.
The others do nothing or just reassign the argument. Assignment in python only affects the variable being assigned and not other variables that still refer to the previous instance.
Mandatory link to Ned Batchelder
Python lets you create variables simply by assigning a value to the variable, without the need to declare the variable upfront. The value assigned to a variable determines the variable type.
if you print the variable type on each function you will see it.
#case 2
def fun2(arg2):
arg2 += [4]
my_var2 = [1,2,3]
#fun(my_var2)
print(type(my_var2))
print(type(my_var2[0]))
#case 4 where no type assigned
def fun4(arg44):
print(arg)
#fun(myvar4)
print(arg4)
Watch this code in action with a visual explanation.
Any variables declared inside a function are local, but lists and dict behave a bit differently. You can append elements to a list even if you dont write global list_name
inside the function nor pass the lists as arguments to the functon. Only lists and dict behave this way. However the difference in case 2 and 3 seems to be a bug. It seems that when you write
a = [1,2,3]
def fun(a_list):
a_list = a_list + [4]
print(a_list)
fun(a)
>>>[1,2,3,4]
you get [1,2,3,4], so it becomes local variable, but when you write arg += [4] , it behaves as global variable
here is a simple (useless) function:
def f(x):
b = [x]
def g(a):
b[0] -= 1
return a - b[0]
return g
it works fine. let's change it a tiny bit:
def f(x):
b = x
def g(a):
b -= 1
return a - b
return g
Now it gives an error saying that b is undefined! Sure, it can be solved using nonlocal, but I'd like to know why does this happen in the first place? Why are mutables accessable and immutables aren't?
Technically, it has nothing to do with mutable or immutable (the language does not know whether a type is "mutable" or not). The difference here is because you are assigning to the variable in one case and just reading from it in the other case.
In your second example, the line b -= 1 is the same as b = b - 1. The fact you are assigning to the variable b makes a local variable named b, separate from the outside variable named b. Since the local variable b has not been assigned to when evaluating the right side of the assignment, reading from the local b there is an error.
In your first example, the line b[0] -= 1 is the same as b[0] = b[0] - 1. But that is not a variable assignment. That is just a list element access. It is just syntactic sugar for b.__setitem__(0, b.__getitem__(0) - 1). You are not assigning to the variable b here -- all you're doing is reading from the variable b two times (in order to call methods on the object it points to). Since you are only reading from b, it uses the external variable b.
If in the first example, with your mutable list, you did a variable assignment like you did in the second example, it would equally create a local variable, and the reading of that local variable before assignment would also not work:
def f(x):
b = [x]
def g(a):
b = [b[0] - 1]
return a - b[0]
return g
I have found that the scope of a variable referencing a List is different than the variable referencing a Tuple or Integer or String. Why does it happen ?
1) When I am using an Integer, String or Tuple:-
>>> def foo(anInt, aStr, aTup):
anInt += 5
aStr += ' hi'
aTup += (12,)
print (anInt,aStr,aTup)
>>> anInt, aStr, aTup = 5, 'Fox', (11,)
>>> foo(anInt, aStr, aTup)
10 Fox hi (11, 12)
>>> print (anInt, aStr, aTup)
5 Fox (11,)
2) When i am using an List:-
>>> def foo(aList):
aList.append(2)
print (aList)
>>> aList = [1]
>>> foo(aList)
[1, 2]
>>> print (aList)
[1, 2]
In the first case changes in the values of anInt, aStr, aTup is limited to the scope of function while in case of a List scope changes.
'
It is not a question of scope. The scope does not differ between types. It can not differ, in fact: a variable is just a named reference to a value, completely type-agnostic.
But some types in Python are mutable (once created, a value can be changed), and list is one of them. Some are immutable, and changing a value of these types requires creating a new object. += works in two different ways for mutable and immutable types. For immutable types a += b is equivalent to a = a + b. A new object is created and the a variable now refers to it. But the objects of mutable types are modified "in place", quite as your list. You may want to read this.
Now let's have a look at scopes. Here you have global objects that are passed to a function. The function uses them as its parameters, not as global objects (yes, aList inside the function is not the same variable as aList outside it, here is more information on scopes). It uses other references to the same objects, and it can not modify the variables, it can just modify objects the variables refer to. But only the mutable objects.
You may be surprised if you compare the results of two following code samples.
>>> a = 1; a1 = a; a1 += 1; a
1
>>> a = [1]; a1 = a; a1 += [1]; a
[1, 1]
What is the difference? The only difference is that int is an immutable type, and the list is a mutable one. After assigning a to a1 they always refer a single object. But the += operator creates a new object in case of int.
Here is a good example of what looks like a difference in scope, when using a variable referring to an int, vs a list.
Note the 'prev' variable:
1 class Solution:
2 def convertBST(self, root: TreeNode) -> TreeNode:
3
4 if root == None:
5 return root
6
7 prev = 0
8 def traverse(node):
9 if node.right:
10 traverse(node.right)
11 node.val += prev
12 prev = node.val
13 if node.left:
14 traverse(node.left)
15
16 traverse(root)
17 return root
This errors with the following message:
UnboundLocalError: local variable 'prev' referenced before assignment
node.val += prev
There is no error, if instead I replaced these lines with this code:
line 7: prev = [0]
line 11: node.val += prev[0]
line 12: prev[0]= node.val
making one believe that prev = 0 was invisible within the traverse function, while prev = [0] is visible!
But in fact, prev = 0 is also visible and can be used within the traverse function, if no assignment to it takes place i.e. UnboundLocalError occurs on line 11, only if line 12 is present.
Because of the immutability of an int, line 12 causes the variable prev within the traverse function to point to a new memory location, and this then "hides" the variable prev = 0 defined outside, causing the error.
But when a new int is assigned to prev[0], an element in a mutable list, the pointer for the first element of the list can be updated to point to the new int (both inside and outside the traverse function); no separate scope (ie local variable) is created, the original prev variable remains visible within the traverse function, and prev[0] can be used on line 11 before the assignment on line 12.
The difference is that the immutable types int, string and tuple are passed by value. When the function updates the value, it is updating a local copy.
The list is passed by reference. When you use append, it is updating the original list.
First of all, I understand that I can use global statement to access global variables. But somehow I was able to modify a global list without global like below:
def func1(nums):
nums = [4,5,6]
nums = [1,2,3]
func1(nums)
print nums # print [1,2,3]
def func2(nums):
nums[0] = 4
nums[1] = 5
nums[2] = 6
nums = [1,2,3]
func2(nums)
print nums # print [4,5,6]
After trying func2, I realized that I can always access global list in a function if I specify the index:
def func3(nums):
nums[:] = [4,5,6]
nums = [1,2,3]
func3(nums)
print nums # print [4,5,6]
Is it because Python automatically goes trying to match a global variable if a function variable is used before definition?
I understand that I can use global statement to access global variables
Your understanding is wrong. You can always access a global variable as long as you don't have a local variable of the same name. You only need the global statement when you are going to change what object a variable name refers to. In your func2, you are not doing that; you are only changing the contents of the object. nums still refers to the same list.
It is of concept based on mutable and immutable objects in Python. In your case, for example:
a=[1,2]
def myfn():
a=[3,4]
print id(a)
>>>id(a)
3065250924L
>>>myfn()
3055359596
It is clear both are different objects. Now:
a=[1,2]
def myfn():
a[:] =[3,4]
print id(a)
>>>id(a)
3055358572
>>>myfn()
3055358572
That means it is same variable using in local and global scope.
In this specific case it is because lists are mutable.
As a result having them in the global namespace, or even passed through a function, means that they will be changed as Python holds the reference to the mutable object, not a copy of it.
If you try the same thing with tuples it will not work, as they are immutable.
The way to avoid this is to provide a copy of the list to the function, not the list itself:
func2(list[:])
At the same time you can do this with default arguments, where you can specify a default argument to be [], and if you then .append() something to it, that default argument will forever hold that item within it for all future calls (unless you remove it in some way).
2 variables nums are different and they point to a same object or 2 different objects, though they have same name.
when you call func1(nums), mean that you pass a reference. Now the 2 variable nums point to same object. (2 variables, 1 object)
when you assign in func1, the inside variable nums will point to a new object, the outside is still unchanged (2 variables, 2 object)
and when you call print nums then this nums is the outside variable,
There are two reasons for this result:
Variables are simply names that refer to objects
List is mutable
In func1,nums refer to a new object because a new list is created. Therefore global nums is not affected.
In func2, the modification is applied to the object passed in. Thus global nums is changed. A new object is not created because list is mutable.
ref: https://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python
from random import randint
shifts = [4, 4.2, 5, 6, 7]
days_names = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday']
workers_names = ['Itai', 'Or', 'Reut', 'Kuka', 'Aviel']
counter = 1
def shift_arrange(worker):
for day in days.values():
counter+=1
global avilable_shifts
avilable_shifts = check_avilable_shifts(day)
if not random_shifte_selector(worker,day): soft_reset(worker)
I set counter as a global variable, and when i try to run this code i get the local variable error:
Traceback (most recent call last):
File "C:\Or\mypy\shift creator\shift cretor.py", line 144, in <module>
for w in workers.values(): shift_arrange(w)
File "C:\Or\mypy\shift creator\shift cretor.py", line 105, in shift_arrange
counter+=1
UnboundLocalError: local variable 'counter' referenced before assignmen
I saw some guy ask this question here, he deleted his pyc file or something(i don't know what is it) and its work fine. Why this is happen? Its not happen to other variables in the program.
Thanks, Or
You need to declare a global variable
def shift_arrange(worker):
global counter
for day in days.values():
counter+=1
...
Since you modify counter in that scope, python treats it as a local variable, unless you declare it as global. If you only need to read it, that isn't necessary.
Consider the following:
This works:
c = 0
def f():
print c
f()
While this does not:
c = 0
def f():
print c
c = 1
f()
While this does:
c = 0
def f():
global c
print c
c = 1
f()
print c # prints 1, f() modified the global value
There's a lot to look up here, but in general python has names, and things that are assigned to them. The name could be either in the local or global scope.
For reads, python looks through local scope, then through global scope, and uses the first one it finds (or error if it doesn't).
For writes... python needs to know where to put it. Normally, what it would do is look in the local scope, and if it's not there, create a variable there and assign the value. This would hide the global variable. You could have it also look in globals and use that one if it exists - but that could be undesirable & unexpected. So, you need a way to tell python to use a global variable instead if it exists (and then, it will create if it doesn't).
This leads to some odd behaviour sometimes as well. In addition to the earlier answer...
c = 0
# Passes. We are just looking up the global variable.
def f1(x):
return x + c
# Passes, but may not be as expected. Sets a local variable c to a value, does not
# modify the global one.
def f2(x):
c = x
# Correct way to do the above; now it sets the global variable.
def f3(x):
global c
c = x
# What if you mix them?
def f4(x):
c = c + x
# This fails. What happens is that first it sees that it's writing to c, and it's
# not marked global, so it assigns c to the local space. Then it tries to dereference
# it. Since we've marked it local, it masks the global one, and since it doesn't
# have a value, it throws an error. c += x works the same way and also fails.
# However, this works, though is just as equally wrong:
def f5(x):
d = c
c = d + x
# This one works, because we copy the value of the global c into the local d.
# Then, the second line assigns a local c to addition of local d and x.
# But does not update the global.
# Each line is separate though:
def f6(x):
d = c
c = c + 1
# You might think that d=c already made c local for the whole function. But it doesn't
# work like that. The first line just looks up the value for c, which it finds
# globally, but then forgets about it - it cares about the object c is the name of,
# not the name c itself. The second line still fails.