First of all, I understand that I can use global statement to access global variables. But somehow I was able to modify a global list without global like below:
def func1(nums):
nums = [4,5,6]
nums = [1,2,3]
func1(nums)
print nums # print [1,2,3]
def func2(nums):
nums[0] = 4
nums[1] = 5
nums[2] = 6
nums = [1,2,3]
func2(nums)
print nums # print [4,5,6]
After trying func2, I realized that I can always access global list in a function if I specify the index:
def func3(nums):
nums[:] = [4,5,6]
nums = [1,2,3]
func3(nums)
print nums # print [4,5,6]
Is it because Python automatically goes trying to match a global variable if a function variable is used before definition?
I understand that I can use global statement to access global variables
Your understanding is wrong. You can always access a global variable as long as you don't have a local variable of the same name. You only need the global statement when you are going to change what object a variable name refers to. In your func2, you are not doing that; you are only changing the contents of the object. nums still refers to the same list.
It is of concept based on mutable and immutable objects in Python. In your case, for example:
a=[1,2]
def myfn():
a=[3,4]
print id(a)
>>>id(a)
3065250924L
>>>myfn()
3055359596
It is clear both are different objects. Now:
a=[1,2]
def myfn():
a[:] =[3,4]
print id(a)
>>>id(a)
3055358572
>>>myfn()
3055358572
That means it is same variable using in local and global scope.
In this specific case it is because lists are mutable.
As a result having them in the global namespace, or even passed through a function, means that they will be changed as Python holds the reference to the mutable object, not a copy of it.
If you try the same thing with tuples it will not work, as they are immutable.
The way to avoid this is to provide a copy of the list to the function, not the list itself:
func2(list[:])
At the same time you can do this with default arguments, where you can specify a default argument to be [], and if you then .append() something to it, that default argument will forever hold that item within it for all future calls (unless you remove it in some way).
2 variables nums are different and they point to a same object or 2 different objects, though they have same name.
when you call func1(nums), mean that you pass a reference. Now the 2 variable nums point to same object. (2 variables, 1 object)
when you assign in func1, the inside variable nums will point to a new object, the outside is still unchanged (2 variables, 2 object)
and when you call print nums then this nums is the outside variable,
There are two reasons for this result:
Variables are simply names that refer to objects
List is mutable
In func1,nums refer to a new object because a new list is created. Therefore global nums is not affected.
In func2, the modification is applied to the object passed in. Thus global nums is changed. A new object is not created because list is mutable.
ref: https://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python
Related
This question already has answers here:
Why can a function modify some arguments as perceived by the caller, but not others?
(13 answers)
Closed 6 months ago.
I was studying merge sort and I wrote this code
def mergesort(a): #sorting algo
#code is alright just confused about scope of variable
if(len(a)>1):
mid = len(a)//2
l = a[:mid]
r = a[mid:]
mergesort(l)
mergesort(r)
i=j=k=0
while(i<len(l) and j<len(r)):
if(l[i]>r[j]):
a[k] = r[j]
j+=1
else:
a[k] = l[i]
i+=1
k+=1
while(i<len(l)):
a[k] = l[i]
i+=1
k+=1
while(j<len(r)):
a[k] = r[j]
j+=1
k+=1
a = [30,2,4,5,-1,3,7,3,9,2,5]
mergesort(a)
print(a)
here a is a global variable, but when the function parameter name is also a doesn't the a in function scope override the a in the global scope? how the global variable is getting edited inside mergesort function?
I tried another code with same approach but here the global variable doesn't get edited(as expected),how is this different from the above code?
def foo(a):
a=20
a = 10
foo(a)
print(a)
The problem is not that a is global (because it's not). The a is local to the function, but it references the same list as the global a (because variables are passed by reference in python).
a is a list (which is mutable) in the first example, while a is an integer (which is immutable) in the second example. So changing values inside of the list (like a[k] = l[i]), changes the global list as well, while changing the reference to an integer (like a = 20) only replaces the local reference.
If you don't want the function to change the global list, you can fix it by creating a copy of the list:
def mergesort(a):
a = a[:] # or a = a.copy(), whichever you prefer
...
Note that in this case, the function has no effect, so you'd probably want to return the resulting list.
Lists are mutable but int is not. So when you change int, you get an entirely new int. When you make changes within a list, you don't get a new list. You just have a modified list.
So when you pass the list to a function, it does not create a new list unless you do something like this: a.copy() or a[:] which creates a new list. In your case, you are using the same list as outside the function.
For more info read about mutability of data types or pass by reference and pass by value.
The first block of code works (lines 15 - 21).
The error is occurring in the second block (lines 24 - 30).
Here is my code:
# converting numerical input into list and tuple <-- line 15
data = input("provide numbers separated by ',': ")
list = data.split(",")
tuple = tuple(list)
print("list:", list, "tuple:", tuple)
def convert(): # <-- line 24
data = input("provide numbers separated by ',': ")
list = data.split(',')
tuple = tuple(list)
print("list:", list, "tuple:", tuple)
convert()
I have read solutions that suggest declaring the variable in the global namespace, but the solutions do not seem to be working for my function. And a solution still eludes me.
The function has the same code as the first block, its just wrapped in a function. I don't understand why the code works at the top level, but not inside a function.
Does anyone know what's happening here?
If you re-bind a non-global variable name (such as a = b) anywhere within a function(1), it is made a local variable for that entire function (even before the modification).
That means that tuple = tuple(list) is assigning to a local tuple, and your use of tuple(list) is using that variable, rather than the actual built-in tuple() function. Since that variable name is not bound at this point, you get the "use before set" error.
This is why it's a bad idea to use built-in function names as variable names, which is something you appear to have done with abandon, both tuple and list :-)
So I would suggest two things:
rename your variables to prevent clashes (use my_list and my_tuple for example).
make my_tuple global within the function (better would be to avoid globals altogether).
Avoiding globals can be as simple as:
def convert():
data = input("provide numbers separated by ',': ")
my_list = data.split(",")
my_tuple = tuple(my_list)
print("list:", my_list, "tuple:", my_tuple)
return my_tuple
outer_tuple = convert()
(1) This does not include simply modifying the variable's value such as changing one item in a list:
x = [1, 2, 3]
x[1] = 42 # Mutation of x, not re-binding.
x = [7, 8, 9] # Re-binding x to new object.
And note that for immutable types (like int for example), x = 7 is re-binding x to a different object 7, not changing the value of the object "behind" x. This particular aspect of Python (the fact that all things are objects and variable names are simply bound to those objects) was one of my greatest epiphanies when learning the language.
Consider the following cases:
Case 1:
def fun(arg):
arg += 1
my_var = 1
fun(my_var)
print(my_var)
>> 1
Case 2:
def fun(arg):
arg += [4]
my_var = [1,2,3]
fun(my_var)
print(my_var)
>> [1, 2, 3, 4]
Case 3:
def fun(arg):
arg = arg + [4]
my_var = [1,2,3]
fun(my_var)
print(my_var)
>> [1, 2, 3]
Case 4:
def fun(arg):
print(arg)
fun("Hi")
print(arg)
>> Hi
Traceback (most recent call last):
File "<string>", line 8, in <module>
NameError: name 'arg' is not defined
Case 4 demonstrates that the scope of the argument variable lies within the function. Case 1 and 3 support that as changes to the arg variable within the function are not reflected in the parameters globally.
But why does Case 2 happen at all? I noticed the same thing when I used append instead of the +=. Shouldn't the changes that happen to arg not have any effect on the variables the function was called with from a different scope?
Any help appreciated. Thanks in advance.
The answer is simple: Function arguments are local variables!
Some more observations:
= is the assignment operator and rebinds a variable. This will never mutate objects (ecxeption slice assignment).
+= is its own operator and is implemented as a mutation for mutable types. You can implement its behaviour for your own types by implementing the __iadd__ dunder method.
Hence, in general, a = a + b is not equivalent to a += b!
For lists, a += b corresponds to a.extend(b). a + b on the other hand, creates a new list object, and a = a + b rebinds variable a to that new object.
Case 2 is the only one where you use a mutating method on an instance. This affects the parameter passed.
The others do nothing or just reassign the argument. Assignment in python only affects the variable being assigned and not other variables that still refer to the previous instance.
Mandatory link to Ned Batchelder
Python lets you create variables simply by assigning a value to the variable, without the need to declare the variable upfront. The value assigned to a variable determines the variable type.
if you print the variable type on each function you will see it.
#case 2
def fun2(arg2):
arg2 += [4]
my_var2 = [1,2,3]
#fun(my_var2)
print(type(my_var2))
print(type(my_var2[0]))
#case 4 where no type assigned
def fun4(arg44):
print(arg)
#fun(myvar4)
print(arg4)
Watch this code in action with a visual explanation.
Any variables declared inside a function are local, but lists and dict behave a bit differently. You can append elements to a list even if you dont write global list_name
inside the function nor pass the lists as arguments to the functon. Only lists and dict behave this way. However the difference in case 2 and 3 seems to be a bug. It seems that when you write
a = [1,2,3]
def fun(a_list):
a_list = a_list + [4]
print(a_list)
fun(a)
>>>[1,2,3,4]
you get [1,2,3,4], so it becomes local variable, but when you write arg += [4] , it behaves as global variable
The intended objective of function foo is to add the number provided as argument to the list and, in case it is 0, reset the list. First I wrote this program:
def foo(n, bar = []):
if n == 0:
bar = []
print("list empty")
else:
bar.append(n)
for y in bar:
print(y, end=', ')
print()
foo(5)
foo(3)
foo(0)
foo(6)
output:
5,
5, 3,
list empty
5, 3, 6,
but it looks like bar = [] is ignored. Then I changed bar = [] with bar.clear() and it works as I thought:
def foo(n, bar = []):
if n == 0:
bar.clear()
print("list empty")
else:
bar.append(n)
for y in bar:
print(y, end=', ')
print()
foo(5)
foo(3)
foo(0)
foo(6)
output:
5,
5, 3,
list empty
6,
I haven't understood why bar.clear() works differntly from bar = [] since clear() should
Remove all elements from the set.
so the same thing bar = [] does.
Edit: I don't think my question is a duplicate of “Least Astonishment” and the Mutable Default Argument, I'm aware that
The default value is evaluated only once.
(from the official tutorial) but what I am asking is, why bar = [] doesn't edit (in this case clear) the list, while append and clear does?
bar = [] rebinds bar; it creates a brand new empty list, and binds the name bar to it. It doesn't matter what bar was before; it could have been an int, a tuple, or dict, and it's irrelevant now, because it's now bound to a brand new list. The old binding no longer exists in the current method call, but the original list remains in existence as the value of the default argument to foo (note: Mutable defaults are considered surprising and ugly because of this confusion).
bar.clear() calls a method on the existing list, which tells that list to clear itself (and has no effect on what bar refers to; it better be something with a clear method, but other than that, it's not really required that it be a list). Since it's the same bar you started with (referencing the same mutable default value cached in the function defaults), that affects your future calls (because bar continues to refer to the list you provided as the mutable default).
The thing you should really understand is how variables work in Python.
The variables are always just references to the actual content. Nothing is stored in the variable itself it just points to the unnamed content.
Now, when you call foo() what you have is a list in memory and the local bar variable is pointing to that. If you call bar.clear(), that list is actually cleared and that's what you want. However, if you say bar = [] you just bind the local bar variable to a new, empty list and the other list (that already contains 5 and 3) remains intact and it will be used the next time you call foo().
Here in you code bar is the reference to some list located in memory.
When you use bar = [], the variable bar becomes associated with a new (empty) list. When you use bar.clear(), you modify current list.
Speaking generally, this means that lists in Python are passed by reference and not by value. You can read more about it here.
When you do a list.clear(), the list gets cleared :-)
When you do a list = [], the list gets overwritten by a new empty list.
The difference is that if something was referencing the list before if was overwritten, it will still be referencing the values you think now has been cleared.
This is an excellent way of introducing extremely hard to debug bugs.
Let's say you have a list containing some values, and you assign it to a variable named foo.
foo = [0, 1]
This variable now points to this list in memory.
Let's say you then create a new variable named bar and assign to foo.
bar = foo
These two variables now point to the same list object in memory. If you then change foo and have it equal [], what happens to bar?
foo = []
bar == [0, 1] # True
Now let's do the same with foo.clear().
foo.clear()
bar == [] # True
This is because using the clear method on a list clears that list object, which affects everything that references this object, while assigning a variable to an empty list only changes what that variables, and doesn't change the original list whatsoever.
this has desired property:
def foo(n, bar = []):
if n == 0:
bar[:] = []
print("list empty")
else:
bar.append(n)
for y in bar:
print(y, end=', ')
print()
I have a hard time to understand the python reference model
def changer(a,b):
a = 2
b[0] = 'spam'
X = 1
L = [1,2]
changer(X,L)
X,L
(1,['spam',2])
here comes my question, for assignment b[0] = 'spam' : I want to know how python modify the mutable object in this way(instead of create a new string objects and link the variable b to that object which will not affect the original object pointed by L)
thanks
a and b are both references to objects.
When you reassign a you change which object a is referencing.
When you reassign b[0] you are reassigning another reference contained within b. b itself still references the same list object that it did originally, which is also the list that was passed into the changer function.
Variables name are pointers to a special memory address ,so when you pass L and X to function the function does not create a new address with a,b just changed the labels !, so any changes on those actually change the value of that part of memory that X,L point to. So for refuse that you can use copy module :
>>> from copy import copy
>>> def changer(a,b):
... i = copy(a)
... j = copy(b)
... i = 2
... j[0] = 'spam'
...
>>> X = 1
>>> L = [1,2]
>>> changer(X,L)
>>> X,L
(1, [1, 2])
In Python, lists are mutable, and integers are immutable. This means that Python will never actually change the value of an integer stored in memory, it creates a new integer and points the variable at the new one.
The reason for this is to make Python's dynamic typing work. Unlike most languages, you can create a variable and store an int in it, then immediately store a string in it, or a float, etc.
MyVariable = 10 # This creates an integer and points MyVariable at it.
MyVariable = "hi" # Created a new string and MyVariable now points to that.
MyVariable = 30 # Created a new integer, and updated the pointer
So this is what happens in your code:
MyVar = 1 # An integer is created and MyVar points to it.
def Increase(num):
num = num + 1 #A new integer is created, the temp variable num points at it.
Increase(MyVar)
print(MyVar) # MyVar still points to the original integer
This is a 'feature' of dynamically typed languages ;)