I am breaking my old question to parts because it is very messy beast here. This question is related to this answer and this answer. I try to understand pointers, not sure even whether they exist in Python.
# Won't change x with y=4
>>> x = 0; y = x; y = 4; x
0
# Won't change y
>>> x = 0; y = x; x = 2; y
0
#so how can I change pointers? Do they even exist in Python?
x = 0
y.pointerDestination = x.pointerDestination #How? By which command?
x = 2
# y should be 0, how?
[Update 2: Solved]
Perhaps, contradictive statements about the existence There are no pointers in Python. and Python does not have the concept of a "pointer" to a simple scalar value.. Does the last one infer that there are pointers to something else, nullifying the first statement?
Scalar objects in Python are immutable. If you use a non-scalar object, such as a list, you can do this:
>>> x = [0]
>>> y = x
>>> y[0] = 4
>>> y
[4]
>>> x
[4]
>>> x is y
True
Python does not have the concept of a "pointer" to a simple scalar value.
Don't confuse pointers to references. They are not the same thing. A pointer is simply an address to an object. You don't really have access to the address of an object in python, only references to them.
When you assign an object to a variable, you are assigning a reference to some object to the variable.
x = 0
# x is a reference to an object `0`
y = [0]
# y is a reference to an object `[0]`
Some objects in python are mutable, meaning you can change properties of the object. Others are immutable, meaning you cannot change the properties of the object.
int (a scalar object) is immutable. There isn't a property of an int that you could change (aka mutating).
# suppose ints had a `value` property which stores the value
x.value = 20 # does not work
list (a non-scalar object) on the other hand is mutable. You can change individual elements of the list to refer to something else.
y[0] = 20 # changes the 0th element of the list to `20`
In the examples you've shown:
>>> x = [0]
>>> y = [x]
you're not dealing with pointers, you're dealing with references to lists with certain values. x is a list that contains a single integer 0. y is a list that contains a reference to whatever x refers to (in this case, the list [0]).
You can change the contents of x like so:
>>> print(x)
[0]
>>> x[0] = 2
>>> print(x)
[2]
You can change the contents of the list referenced by x through y's reference:
>>> print(x)
[2]
>>> print(y)
[[2]]
>>> y[0][0] = 5
>>> print(x)
[5]
>>> print(y)
[[5]]
You can change the contents of y to reference something else:
>>> print(y)
[[5]]
>>> y[0] = 12345
>>> print(x)
[5]
>>> print(y)
[12345]
It's basically the same semantics of a language such as Java or C#. You don't use pointers to objects directly (though you do indirectly since the implementations use pointers behind the scenes), but references to objects.
There are no pointers in Python. There are things called references (which, like C++ references, happen to be commonly implemented in pointers - but unlike C++ references don't imply pass-by-reference). Every variable stores a reference to an object allocated somewhere else (on the heap). Every collection stores references to objects allocated somewhere else. Every member of an object stores a reference to an object allocated somewhere else.
The simple expression x evaluates to the reference stored in x - whoever uses it has no way to determine that is came from a variable. There's no way to get a link to a variable (as opposed to the contents of it) that could be used to track changes of that variable. Item (x[y] = ...) and member (x.y = ...) assignments are different in one regard: They invoke methods and mutate existing objects instead of overwriting a local variable. This difference is mainly important when dealing with scoping, but you can use either of those to emulate mutability for immutable types (as shown by #Greg Hewgill) and to share state changes across function boundaries (def f(x): x = 0 doesn't change anything, but def g(x): x.x = 0 does). It's not fully up to emulating pass by reference though - unless you replace every variable by a wrapper object whose sole purpose is to hold a mutable val property. This would be the equivalent to emulating pass-by-reference through pointers in C, but much more cumbersome.
Related
This question already has answers here:
Why can a function modify some arguments as perceived by the caller, but not others?
(13 answers)
Closed 3 years ago.
Let's take a simple code:
y = [1,2,3]
def plusOne(y):
for x in range(len(y)):
y[x] += 1
return y
print plusOne(y), y
a = 2
def plusOne2(a):
a += 1
return a
print plusOne2(a), a
Values of 'y' change but value 'a' stays the same. I have already learned that it's because one is mutable and the other is not. But how to change the code so that the function doesn't change the list?
For example to do something like that (in pseudocode for simplicity):
a = [1,2,3,...,n]
function doSomething(x):
do stuff with x
return x
b = doSomething(a)
if someOperation(a) > someOperation(b):
do stuff
EDIT: Sorry, but I have another question on nested lists. See this code:
def change(y):
yN = y[:]
for i in range(len(yN)):
if yN[i][0] == 1:
yN[i][0] = 0
else:
yN[i][0] = 1
return yN
data1 = [[1],[1],[0],[0]]
data2 = change(data1)
Here it doesn't work. Why? Again: how to avoid this problem? I understand why it is not working: yN = y[:] copies values of y to yN, but the values are also lists, so the operation would have to be doubled for every list in list. How to do this operation with nested lists?
Python variables contain pointers, or references, to objects. All values (even integers) are objects, and assignment changes the variable to point to a different object. It does not store a new value in the variable, it changes the variable to refer or point to a different object. For this reason many people say that Python doesn't have "variables," it has "names," and the = operation doesn't "assign a value to a variable," but rather "binds a name to an object."
In plusOne you are modifying (or "mutating") the contents of y but never change what y itself refers to. It stays pointing to the same list, the one you passed in to the function. The global variable y and the local variable y refer to the same list, so the changes are visible using either variable. Since you changed the contents of the object that was passed in, there is actually no reason to return y (in fact, returning None is what Python itself does for operations like this that modify a list "in place" -- values are returned by operations that create new objects rather than mutating existing ones).
In plusOne2 you are changing the local variable a to refer to a different integer object, 3. ("Binding the name a to the object 3.") The global variable a is not changed by this and continues to point to 2.
If you don't want to change a list passed in, make a copy of it and change that. Then your function should return the new list since it's one of those operations that creates a new object, and the new object will be lost if you don't return it. You can do this as the first line of the function: x = x[:] for example (as others have pointed out). Or, if it might be useful to have the function called either way, you can have the caller pass in x[:] if he wants a copy made.
Create a copy of the list. Using testList = inputList[:]. See the code
>>> def plusOne(y):
newY = y[:]
for x in range(len(newY)):
newY[x] += 1
return newY
>>> y = [1, 2, 3]
>>> print plusOne(y), y
[2, 3, 4] [1, 2, 3]
Or, you can create a new list in the function
>>> def plusOne(y):
newList = []
for elem in y:
newList.append(elem+1)
return newList
You can also use a comprehension as others have pointed out.
>>> def plusOne(y):
return [elem+1 for elem in y]
You can pass a copy of your list, using slice notation:
print plusOne(y[:]), y
Or the better way would be to create the copy of list in the function itself, so that the caller don't have to worry about the possible modification:
def plusOne(y):
y_copy = y[:]
and work on y_copy instead.
Or as pointed out by #abarnet in comments, you can modify the function to use list comprehension, which will create a new list altogether:
return [x + 1 for x in y]
Just create a new list with the values you want in it and return that instead.
def plus_one(sequence):
return [el + 1 for el in sequence]
As others have pointed out, you should use newlist = original[:] or newlist = list(original) to copy the list if you do not want to modify the original.
def plusOne(y):
y2 = list(y) # copy the list over, y2 = y[:] also works
for i, _ in enumerate(y2):
y2[i] += 1
return y2
However, you can acheive your desired output with a list comprehension
def plusOne(y):
return [i+1 for i in y]
This will iterate over the values in y and create a new list by adding one to each of them
To answer your edited question:
Copying nested data structures is called deep copying. To do this in Python, use deepcopy() within the copy module.
You can do that by make a function and call this function by map function ,
map function will call the add function and give it the value after that it will print the new value like that:
def add(x):
return x+x
print(list(map(add,[1,2,3])))
Or you can use (*range) function it is very easy to do that like that example :
print ([i+i for i in [*range (1,10)]])
I moved from using Matlab to Python and the variable assignment while using functions is confusing me.
I have a code as follows:
a = [1,1,1]
def keeps(x):
y = x[:]
y[1] = 2
return y
def changes(x):
y = x
y[1] = 2
return y
aout = keeps(a)
print(a, aout)
aout = changes(a)
print(a, aout)
The first print statement gives [1, 1, 1] [1, 2, 1], while
the second one gives [1, 2, 1] [1, 2, 1].
I had a understanding (coming from Matlab) that the operations on a variable within a function are local. But here, if I don't make a copy of the variable inside a function, the values change outside the function as well. It's almost as if the variable is defined as global.
It will be very helpful if someone can explain how the variables are allocated differently in both the methods and what are the best practices if one wants to send a variable to the function without affecting it's value outside the function? Thanks.
Argument passing is done by assignment. In changes, the first thing that happens implicitly is
x = a when you call changes(a). Since assingment NEVER copies data you mutate a.
In keeps you are not mutating the argument list because x[:] is creating a (shallow) copy which then the name y is assigned to.
I highly recommend watching Facts and Myths about Python names and values.
Let's look at your code, but first, we will mode the function declarations to the top, so that the order of execution becomes clearer.
def keeps(x):
y = x[:] #Here you are creating a modifiable copy of the original x list and referencing it with y
y[1] = 2
return y
def changes(x):
y = x # Here you are just referencing x itself with a new name y
y[1] = 2
return y
a = [1,1,1]
aout = keeps(a)
print(a, aout)
aout = changes(a)
print(a, aout)
Basically if you just assign another variable name to a list, you are giving two names to the same object, so any changes in the contents may affect both "lists". When you use y = x[:]you are in fact creating a new copy of the x list in memory, through list slicing, and assigning the new variable name y to that new copy of the list.
From Chapter "Classes" of the official Python tutorial:
[...] if a function modifies an object passed as an argument, the caller will see the change — this eliminates the need for two different argument passing mechanisms as in Pascal.
What would be an example of how exactly the caller will see a change? Or how could it be (not in Python but in general) that the caller doesn't see the change?
It basically means that if a mutable object is changed, it will change everywhere.
For an example of passing by reference (which is what Python does):
x = []
def foo_adder(y):
y.append('foo')
foo_addr(x)
print(x) # ['foo']
vs something like Pascal, where you can pass copies of an object as a parameter, instead of the object itself:
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x)
print(x) # []
You can get the behavior of the second example in Python if you pass a copy of the object. For lists, you use [:].
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x[:])
print(x) # []
For your second question about how the caller might not see the change, let's take that same foo_adder function and change it a little so that it doesn't modify the object, but instead replaces it.
x = []
def foo_adder(y):
y = y + ['foo']
foo_adder(x)
print(x) # []
What would be an example of how exactly the caller will see a change?
>>> def modify(x):
... x.append(1)
...
>>> seq = []
>>> print(seq)
[]
>>> modify(seq)
>>> print(seq)
[1]
Or how could it be (not in Python but in general) that the caller doesn't see the change?
Hypothetically, a language could exist where a deep copy of seq is created and assigned to x, and any change made to x has no effect on seq, in which case print(seq) would display [] both times. But this isn't what happens in Python.
Edit: note that assigning a new value to an old variable name typically doesn't count as "modification".
>>> def f(x):
... x = x + 1
...
>>> y = 23
>>> f(y)
>>> print(y)
23
I have a hard time to understand the python reference model
def changer(a,b):
a = 2
b[0] = 'spam'
X = 1
L = [1,2]
changer(X,L)
X,L
(1,['spam',2])
here comes my question, for assignment b[0] = 'spam' : I want to know how python modify the mutable object in this way(instead of create a new string objects and link the variable b to that object which will not affect the original object pointed by L)
thanks
a and b are both references to objects.
When you reassign a you change which object a is referencing.
When you reassign b[0] you are reassigning another reference contained within b. b itself still references the same list object that it did originally, which is also the list that was passed into the changer function.
Variables name are pointers to a special memory address ,so when you pass L and X to function the function does not create a new address with a,b just changed the labels !, so any changes on those actually change the value of that part of memory that X,L point to. So for refuse that you can use copy module :
>>> from copy import copy
>>> def changer(a,b):
... i = copy(a)
... j = copy(b)
... i = 2
... j[0] = 'spam'
...
>>> X = 1
>>> L = [1,2]
>>> changer(X,L)
>>> X,L
(1, [1, 2])
In Python, lists are mutable, and integers are immutable. This means that Python will never actually change the value of an integer stored in memory, it creates a new integer and points the variable at the new one.
The reason for this is to make Python's dynamic typing work. Unlike most languages, you can create a variable and store an int in it, then immediately store a string in it, or a float, etc.
MyVariable = 10 # This creates an integer and points MyVariable at it.
MyVariable = "hi" # Created a new string and MyVariable now points to that.
MyVariable = 30 # Created a new integer, and updated the pointer
So this is what happens in your code:
MyVar = 1 # An integer is created and MyVar points to it.
def Increase(num):
num = num + 1 #A new integer is created, the temp variable num points at it.
Increase(MyVar)
print(MyVar) # MyVar still points to the original integer
This is a 'feature' of dynamically typed languages ;)
This behavior has me puzzled:
import code
class foo():
def __init__(self):
self.x = 1
def interact(self):
v = globals()
v.update(vars(self))
code.interact(local=v)
c = foo()
c.interact()
Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
(InteractiveConsole)
>>> id(x)
29082424
>>> id(c.x)
29082424
>>> x
1
>>> c.x
1
>>> x=2
>>> c.x
1
Why doesn't 'c.x' behave like an alias for 'x'? If I understand the id() function correctly, they are both at the same memory address.
Small integers from from -5 to 256 are cached in python, i.e their id() is always going to be same.
From the docs:
The current implementation keeps an array of integer objects for all
integers between -5 and 256, when you create an int in that range you
actually just get back a reference to the existing object.
>>> x = 1
>>> y = 1 #same id() here as integer 1 is cached by python.
>>> x is y
True
Update:
If two identifiers return same value of id() then it doesn't mean they can act as alias of
each other, it totally depends on the type of the object they are pointing to.
For immutable object you cannot create alias in python. Modifying one of the reference to an immutable object will simple make it point to a new object, while other references to that older object will still remain intact.
>>> x = y = 300
>>> x is y # x and y point to the same object
True
>>> x += 1 # modify x
>>> x # x now points to a different object
301
>>> y #y still points to the old object
300
A mutable object can be modified from any of it's references, but those modifications must be in-place modifications.
>>> x = y = []
>>> x is y
True
>>> x.append(1) # list.extend is an in-place operation
>>> y.append(2) # in-place operation
>>> x
[1, 2]
>>> y #works fine so far
[1, 2]
>>> x = x + [1] #not an in-place operation
>>> x
[1, 2, 1] #assigns a new object to x
>>> y #y still points to the same old object
[1, 2]
code.interact simply did (effectively) x=c.x for you. So when you checked their ids, they were pointing to the exact same object. But x=2 creates a new binding for the variable x. It is not an alias. Python does not have aliases, as far as I am aware.
Yes, in CPython id(x) is the memory address of the object x points to. It is not the memory address of the variable x itself (which is, after all, just a key in a dictionary).
If I understand the id() function correctly, they are both at the same memory address.
You don't understand it correctly. id returns an integer in respect of which the following identity is guaranteed: if id(x) == id(y) then x is y is guaranteed (and vice versa).
Accordingly, id tells you about the objects (values) that variables point to, not about the variables themselves.
Any relationship to memory addresses is purely an implementation detail. Python, unlike, e.g. C, does not assume any particular relationship to the underlying machine (whether physical or virtual). Variables in python are both opaque, and not language accessible (i.e. not first class).