I have a hard time to understand the python reference model
def changer(a,b):
a = 2
b[0] = 'spam'
X = 1
L = [1,2]
changer(X,L)
X,L
(1,['spam',2])
here comes my question, for assignment b[0] = 'spam' : I want to know how python modify the mutable object in this way(instead of create a new string objects and link the variable b to that object which will not affect the original object pointed by L)
thanks
a and b are both references to objects.
When you reassign a you change which object a is referencing.
When you reassign b[0] you are reassigning another reference contained within b. b itself still references the same list object that it did originally, which is also the list that was passed into the changer function.
Variables name are pointers to a special memory address ,so when you pass L and X to function the function does not create a new address with a,b just changed the labels !, so any changes on those actually change the value of that part of memory that X,L point to. So for refuse that you can use copy module :
>>> from copy import copy
>>> def changer(a,b):
... i = copy(a)
... j = copy(b)
... i = 2
... j[0] = 'spam'
...
>>> X = 1
>>> L = [1,2]
>>> changer(X,L)
>>> X,L
(1, [1, 2])
In Python, lists are mutable, and integers are immutable. This means that Python will never actually change the value of an integer stored in memory, it creates a new integer and points the variable at the new one.
The reason for this is to make Python's dynamic typing work. Unlike most languages, you can create a variable and store an int in it, then immediately store a string in it, or a float, etc.
MyVariable = 10 # This creates an integer and points MyVariable at it.
MyVariable = "hi" # Created a new string and MyVariable now points to that.
MyVariable = 30 # Created a new integer, and updated the pointer
So this is what happens in your code:
MyVar = 1 # An integer is created and MyVar points to it.
def Increase(num):
num = num + 1 #A new integer is created, the temp variable num points at it.
Increase(MyVar)
print(MyVar) # MyVar still points to the original integer
This is a 'feature' of dynamically typed languages ;)
Related
I have found that the scope of a variable referencing a List is different than the variable referencing a Tuple or Integer or String. Why does it happen ?
1) When I am using an Integer, String or Tuple:-
>>> def foo(anInt, aStr, aTup):
anInt += 5
aStr += ' hi'
aTup += (12,)
print (anInt,aStr,aTup)
>>> anInt, aStr, aTup = 5, 'Fox', (11,)
>>> foo(anInt, aStr, aTup)
10 Fox hi (11, 12)
>>> print (anInt, aStr, aTup)
5 Fox (11,)
2) When i am using an List:-
>>> def foo(aList):
aList.append(2)
print (aList)
>>> aList = [1]
>>> foo(aList)
[1, 2]
>>> print (aList)
[1, 2]
In the first case changes in the values of anInt, aStr, aTup is limited to the scope of function while in case of a List scope changes.
'
It is not a question of scope. The scope does not differ between types. It can not differ, in fact: a variable is just a named reference to a value, completely type-agnostic.
But some types in Python are mutable (once created, a value can be changed), and list is one of them. Some are immutable, and changing a value of these types requires creating a new object. += works in two different ways for mutable and immutable types. For immutable types a += b is equivalent to a = a + b. A new object is created and the a variable now refers to it. But the objects of mutable types are modified "in place", quite as your list. You may want to read this.
Now let's have a look at scopes. Here you have global objects that are passed to a function. The function uses them as its parameters, not as global objects (yes, aList inside the function is not the same variable as aList outside it, here is more information on scopes). It uses other references to the same objects, and it can not modify the variables, it can just modify objects the variables refer to. But only the mutable objects.
You may be surprised if you compare the results of two following code samples.
>>> a = 1; a1 = a; a1 += 1; a
1
>>> a = [1]; a1 = a; a1 += [1]; a
[1, 1]
What is the difference? The only difference is that int is an immutable type, and the list is a mutable one. After assigning a to a1 they always refer a single object. But the += operator creates a new object in case of int.
Here is a good example of what looks like a difference in scope, when using a variable referring to an int, vs a list.
Note the 'prev' variable:
1 class Solution:
2 def convertBST(self, root: TreeNode) -> TreeNode:
3
4 if root == None:
5 return root
6
7 prev = 0
8 def traverse(node):
9 if node.right:
10 traverse(node.right)
11 node.val += prev
12 prev = node.val
13 if node.left:
14 traverse(node.left)
15
16 traverse(root)
17 return root
This errors with the following message:
UnboundLocalError: local variable 'prev' referenced before assignment
node.val += prev
There is no error, if instead I replaced these lines with this code:
line 7: prev = [0]
line 11: node.val += prev[0]
line 12: prev[0]= node.val
making one believe that prev = 0 was invisible within the traverse function, while prev = [0] is visible!
But in fact, prev = 0 is also visible and can be used within the traverse function, if no assignment to it takes place i.e. UnboundLocalError occurs on line 11, only if line 12 is present.
Because of the immutability of an int, line 12 causes the variable prev within the traverse function to point to a new memory location, and this then "hides" the variable prev = 0 defined outside, causing the error.
But when a new int is assigned to prev[0], an element in a mutable list, the pointer for the first element of the list can be updated to point to the new int (both inside and outside the traverse function); no separate scope (ie local variable) is created, the original prev variable remains visible within the traverse function, and prev[0] can be used on line 11 before the assignment on line 12.
The difference is that the immutable types int, string and tuple are passed by value. When the function updates the value, it is updating a local copy.
The list is passed by reference. When you use append, it is updating the original list.
From Chapter "Classes" of the official Python tutorial:
[...] if a function modifies an object passed as an argument, the caller will see the change — this eliminates the need for two different argument passing mechanisms as in Pascal.
What would be an example of how exactly the caller will see a change? Or how could it be (not in Python but in general) that the caller doesn't see the change?
It basically means that if a mutable object is changed, it will change everywhere.
For an example of passing by reference (which is what Python does):
x = []
def foo_adder(y):
y.append('foo')
foo_addr(x)
print(x) # ['foo']
vs something like Pascal, where you can pass copies of an object as a parameter, instead of the object itself:
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x)
print(x) # []
You can get the behavior of the second example in Python if you pass a copy of the object. For lists, you use [:].
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x[:])
print(x) # []
For your second question about how the caller might not see the change, let's take that same foo_adder function and change it a little so that it doesn't modify the object, but instead replaces it.
x = []
def foo_adder(y):
y = y + ['foo']
foo_adder(x)
print(x) # []
What would be an example of how exactly the caller will see a change?
>>> def modify(x):
... x.append(1)
...
>>> seq = []
>>> print(seq)
[]
>>> modify(seq)
>>> print(seq)
[1]
Or how could it be (not in Python but in general) that the caller doesn't see the change?
Hypothetically, a language could exist where a deep copy of seq is created and assigned to x, and any change made to x has no effect on seq, in which case print(seq) would display [] both times. But this isn't what happens in Python.
Edit: note that assigning a new value to an old variable name typically doesn't count as "modification".
>>> def f(x):
... x = x + 1
...
>>> y = 23
>>> f(y)
>>> print(y)
23
First of all, I understand that I can use global statement to access global variables. But somehow I was able to modify a global list without global like below:
def func1(nums):
nums = [4,5,6]
nums = [1,2,3]
func1(nums)
print nums # print [1,2,3]
def func2(nums):
nums[0] = 4
nums[1] = 5
nums[2] = 6
nums = [1,2,3]
func2(nums)
print nums # print [4,5,6]
After trying func2, I realized that I can always access global list in a function if I specify the index:
def func3(nums):
nums[:] = [4,5,6]
nums = [1,2,3]
func3(nums)
print nums # print [4,5,6]
Is it because Python automatically goes trying to match a global variable if a function variable is used before definition?
I understand that I can use global statement to access global variables
Your understanding is wrong. You can always access a global variable as long as you don't have a local variable of the same name. You only need the global statement when you are going to change what object a variable name refers to. In your func2, you are not doing that; you are only changing the contents of the object. nums still refers to the same list.
It is of concept based on mutable and immutable objects in Python. In your case, for example:
a=[1,2]
def myfn():
a=[3,4]
print id(a)
>>>id(a)
3065250924L
>>>myfn()
3055359596
It is clear both are different objects. Now:
a=[1,2]
def myfn():
a[:] =[3,4]
print id(a)
>>>id(a)
3055358572
>>>myfn()
3055358572
That means it is same variable using in local and global scope.
In this specific case it is because lists are mutable.
As a result having them in the global namespace, or even passed through a function, means that they will be changed as Python holds the reference to the mutable object, not a copy of it.
If you try the same thing with tuples it will not work, as they are immutable.
The way to avoid this is to provide a copy of the list to the function, not the list itself:
func2(list[:])
At the same time you can do this with default arguments, where you can specify a default argument to be [], and if you then .append() something to it, that default argument will forever hold that item within it for all future calls (unless you remove it in some way).
2 variables nums are different and they point to a same object or 2 different objects, though they have same name.
when you call func1(nums), mean that you pass a reference. Now the 2 variable nums point to same object. (2 variables, 1 object)
when you assign in func1, the inside variable nums will point to a new object, the outside is still unchanged (2 variables, 2 object)
and when you call print nums then this nums is the outside variable,
There are two reasons for this result:
Variables are simply names that refer to objects
List is mutable
In func1,nums refer to a new object because a new list is created. Therefore global nums is not affected.
In func2, the modification is applied to the object passed in. Thus global nums is changed. A new object is not created because list is mutable.
ref: https://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python
In this case, I want that the program print "X = changed"
class Clase:
def __init__(self,variable):
self.var = variable
def set_var(self):
self.var = 'changed'
X = 'unchanged'
V = Clase(X)
V.set_var()
print "X = ",X
All values are objects and are passed by reference in Python, and assignment changes the reference.
def myfunc(y):
y = 13
x = 42 # x now points at the integer object, 42
myfunc(y) # inside myfunc, y initially points to 42,
# but myfunc changes its y to point to a
# different object, 13
print(x) # prints 42, since changing y inside myfunc
# does not change any other variable
It's important to note here that there are no "simple types" as there are in other languages. In Python, integers are objects. Floats are objects. Bools are objects. And assignment is always changing a pointer to refer to a different object, whatever the type of that object.
Thus, it's not possible to "assign through" a reference and change someone else's variable. You can, however, simulate this by passing a mutable container (e.g. a list or a dictionary) and changing the contents of the container, as others have shown.
This kind of mutation of arguments through pointers is common in C/C++ and is generally used to work around the fact that a function can only have a single return value. Python will happily create tuples for you in the return statement and unpack them to multiple variables on the other side, making it easy to return multiple values, so this isn't an issue. Just return all the values you want to return. Here is a trivial example:
def myfunc(x, y, z):
return x * 2, y + 5, z - 3
On the other side:
a, b, c = myFunc(4, 5, 6)
In practice, then, there is rarely any reason to need to do what you're trying to do in Python.
In python list and dict types are global and are passed around by reference. So if you change the type of your variable X to one of those you will get the desired results.
[EDIT: Added use case that op needed]
class Clase:
def __init__(self,variable):
self.var = variable
def set_var(self):
self.var.test = 'changed'
class ComplicatedClass():
def __init__(self, test):
self.test = test
X = ComplicatedClass('unchanged')
print('Before:', X.test)
V = Clase(X)
V.set_var()
print("After:",X.test)
>>> Before: unchanged
>>> After: changed
strings are immutable so you could not change X in this way
... an alternative might be reassigning X in the global space... this obviously will fail in many many senarios (ie it is not a global)
class Clase:
def __init__(self,variable):
self.var = variable
def set_var(self):
globals()[self.var] = 'changed'
X = 'unchanged'
V = Clase('X')
V.set_var()
print "X = ",X
the other alternative is to use a mutable data type as suggested by Ashwin
or the best option is that this is probably not a good idea and you should likely not do it...
I am breaking my old question to parts because it is very messy beast here. This question is related to this answer and this answer. I try to understand pointers, not sure even whether they exist in Python.
# Won't change x with y=4
>>> x = 0; y = x; y = 4; x
0
# Won't change y
>>> x = 0; y = x; x = 2; y
0
#so how can I change pointers? Do they even exist in Python?
x = 0
y.pointerDestination = x.pointerDestination #How? By which command?
x = 2
# y should be 0, how?
[Update 2: Solved]
Perhaps, contradictive statements about the existence There are no pointers in Python. and Python does not have the concept of a "pointer" to a simple scalar value.. Does the last one infer that there are pointers to something else, nullifying the first statement?
Scalar objects in Python are immutable. If you use a non-scalar object, such as a list, you can do this:
>>> x = [0]
>>> y = x
>>> y[0] = 4
>>> y
[4]
>>> x
[4]
>>> x is y
True
Python does not have the concept of a "pointer" to a simple scalar value.
Don't confuse pointers to references. They are not the same thing. A pointer is simply an address to an object. You don't really have access to the address of an object in python, only references to them.
When you assign an object to a variable, you are assigning a reference to some object to the variable.
x = 0
# x is a reference to an object `0`
y = [0]
# y is a reference to an object `[0]`
Some objects in python are mutable, meaning you can change properties of the object. Others are immutable, meaning you cannot change the properties of the object.
int (a scalar object) is immutable. There isn't a property of an int that you could change (aka mutating).
# suppose ints had a `value` property which stores the value
x.value = 20 # does not work
list (a non-scalar object) on the other hand is mutable. You can change individual elements of the list to refer to something else.
y[0] = 20 # changes the 0th element of the list to `20`
In the examples you've shown:
>>> x = [0]
>>> y = [x]
you're not dealing with pointers, you're dealing with references to lists with certain values. x is a list that contains a single integer 0. y is a list that contains a reference to whatever x refers to (in this case, the list [0]).
You can change the contents of x like so:
>>> print(x)
[0]
>>> x[0] = 2
>>> print(x)
[2]
You can change the contents of the list referenced by x through y's reference:
>>> print(x)
[2]
>>> print(y)
[[2]]
>>> y[0][0] = 5
>>> print(x)
[5]
>>> print(y)
[[5]]
You can change the contents of y to reference something else:
>>> print(y)
[[5]]
>>> y[0] = 12345
>>> print(x)
[5]
>>> print(y)
[12345]
It's basically the same semantics of a language such as Java or C#. You don't use pointers to objects directly (though you do indirectly since the implementations use pointers behind the scenes), but references to objects.
There are no pointers in Python. There are things called references (which, like C++ references, happen to be commonly implemented in pointers - but unlike C++ references don't imply pass-by-reference). Every variable stores a reference to an object allocated somewhere else (on the heap). Every collection stores references to objects allocated somewhere else. Every member of an object stores a reference to an object allocated somewhere else.
The simple expression x evaluates to the reference stored in x - whoever uses it has no way to determine that is came from a variable. There's no way to get a link to a variable (as opposed to the contents of it) that could be used to track changes of that variable. Item (x[y] = ...) and member (x.y = ...) assignments are different in one regard: They invoke methods and mutate existing objects instead of overwriting a local variable. This difference is mainly important when dealing with scoping, but you can use either of those to emulate mutability for immutable types (as shown by #Greg Hewgill) and to share state changes across function boundaries (def f(x): x = 0 doesn't change anything, but def g(x): x.x = 0 does). It's not fully up to emulating pass by reference though - unless you replace every variable by a wrapper object whose sole purpose is to hold a mutable val property. This would be the equivalent to emulating pass-by-reference through pointers in C, but much more cumbersome.