We Keep Coding

Python confusion -- convention, name and value

I am a beginner and have a confusion when I am learning python. If I have the following python code:
import numpy as np
X = np.array([1,0,0])
Y = X
X[0] = 2
print Y
Y will be shown to be array([2, 0, 0])
However, if I do the following:
import numpy as np
X = np.array([1,0,0])
Y = X
X = 2*X
print Y
Y is still array([1,0,0])
What is going on?

think of it this way:
the equals sign in python assigns references.
Y = X makes Y point to the same address X points to
X[0] = 2 makes x[0] point to 2
X = 2*X makes X point to a new thing, but Y is still pointing to the address of the original X, so Y is unchanged
this isn't exactly true, but its close enough to understand the principle

That's because X and Y are references to the same object np.array([1,0,0]) this means that regardless whether a call is done through X or Y, the result will be the same, but changing the reference of one, has no effect.
If you write:
X = np.array([1,0,0])
Y = X
basically what happens is that there are two local variables X and Y that refer to the same object. So the memory looks like:
+--------+
Y -> |np.array| <- X
+--------+
|[1,0,0] |
+--------+
Now if you do X[0] = 2 that is basically short for:
X.__setitem__(0,2)
so you call a method on the object. So now the memory looks like:
+--------+
Y -> |np.array| <- X
+--------+
|[2,0,0] |
+--------+
If you however write:
X = 2*X
first 2*X is evaluated. Now 2*X is short for:
X.__rmul__(2)
(Python first looks if 2 supports __mul__ for X, but since 2 will raise a NotImplementedException), Python will fallback to X.__rmul__). Now X.__rmul__ does not change X: it leaves X intact, but constructs a new array and returns that. X catches by that new array that now references to that array).
which creates an new array object: array([4, 0, 0]) and then X references to that new object. So now the memory looks like:
+--------+ +--------+
Y -> |np.array| X ->|np.array|
+--------+ +--------+
|[2,0,0] | |[4,0,0] |
+--------+ +--------+
But as you can see, Y still references to the old object.

This is more about convention and names than reference and value.
When you assign:
Y = X
Then the name Y refers to the object that the name X points to. In some way the pointer X and Y point to the same object:
X is Y # True
The is checks if the names point to the same object!
Then it get's tricky: You do some operations on the arrays.
X[0] = 2
That's called "item assignment" and calls
X.__setitem__(0, 2)
What __setitem__ should do (convention) is to update some value in the container X. So X should still point to the same object afterwards.
However X * 2 is "multiplication" and the convention states that this should create a new object (again convention, you can change that behaviour by overwriting X.__mul__). So when you do
X = X * 2
The name X now refers to the new object that X * 2 created:
X is Y # False
Normally common libraries follow these conventions but it's important to highlight that you can completly change this!

When you say X = np.array([1, 0, 0]), you create an object that has some methods and some internal buffers that contain the actual data and other information in it.
Doing Y = X sets Y to refer to the same actual object. This is called binding to a name in Python. You have bound the same object that was bound to X to the name Y as well.
Doing X[0] = 2 calls the object's __setitem__ method, which does some stuff to the underlying buffers. If modifies the object in place. Now when you print the values of either X or Y, the numbers that come out of that object's buffers are 2, 0, 0.
Doing X = 2 * X translates to X.__rmul__(2). This method does not modify X in place. It creates and returns a new array object, each of whose elements is twice the corresponding element of X. Then you bind the new object to the name X. However, the name Y is still bound to the original array because you have not done anything to change that. As an aside, X.__rmul__ is used because 2.__mul__(X) does not work. Numpy arrays naturally define multiplication to be commutative, so X.__mul__ and X.__rmul__ should to the same thing.
It is interesting to note that you can also do X *= 2, which will propagate the changes to Y. This is because the *= operator translates to the __imul__ method, which does modify the input in place.

How exactly does the caller see a change in the object?

From Chapter "Classes" of the official Python tutorial:
[...] if a function modifies an object passed as an argument, the caller will see the change — this eliminates the need for two different argument passing mechanisms as in Pascal.
What would be an example of how exactly the caller will see a change? Or how could it be (not in Python but in general) that the caller doesn't see the change?

It basically means that if a mutable object is changed, it will change everywhere.
For an example of passing by reference (which is what Python does):
x = []
def foo_adder(y):
y.append('foo')
foo_addr(x)
print(x) # ['foo']
vs something like Pascal, where you can pass copies of an object as a parameter, instead of the object itself:
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x)
print(x) # []
You can get the behavior of the second example in Python if you pass a copy of the object. For lists, you use [:].
# Pretend this is Pascal code.
x = []
def foo_adder(y):
y.append('foo')
foo_adder(x[:])
print(x) # []
For your second question about how the caller might not see the change, let's take that same foo_adder function and change it a little so that it doesn't modify the object, but instead replaces it.
x = []
def foo_adder(y):
y = y + ['foo']
foo_adder(x)
print(x) # []

What would be an example of how exactly the caller will see a change?
>>> def modify(x):
... x.append(1)
...
>>> seq = []
>>> print(seq)
[]
>>> modify(seq)
>>> print(seq)
[1]
Or how could it be (not in Python but in general) that the caller doesn't see the change?
Hypothetically, a language could exist where a deep copy of seq is created and assigned to x, and any change made to x has no effect on seq, in which case print(seq) would display [] both times. But this isn't what happens in Python.
Edit: note that assigning a new value to an old variable name typically doesn't count as "modification".
>>> def f(x):
... x = x + 1
...
>>> y = 23
>>> f(y)
>>> print(y)
23

Different Python Variables with Same Value Pointing to Same Object [duplicate]

This question already has answers here:
"is" operator behaves unexpectedly with integers
(11 answers)
Closed 7 years ago.
In python, I have declared two variables with same value. Strangely, they are pointing to same object. I need to understand how this objects and their corresponding values are assigned.
#!/usr/bin/python
a = 100
b = 100
print id(a)
print id(b)
--------------------
Output :
157375428
157375428
-------------------
I assume, a and b are two different variables with same value. Then why the same object is pointing to both of them ?

By calling id(a) you actually get same result as when calling id(100), a and b share the same instance of 100. I know this is quite confusing, almost every other programming language behaves differently. Maybe you shouldn't think a and b as variables but instead "named references" to objects.

Technically a and b are two different variables.
In Python a variable is a just a name. Values are somewhere else and a variable refers to a value.
From the Python documentation
For immutable types, operations that compute new values may actually
return a reference to any existing object with the same type and
value, while for mutable objects this is not allowed. E.g., after a = 1; b = 1, a and b may or may not refer to the same object with the
value one, depending on the implementation.

Python pre-allocates a number of integers (see http://blog.lerner.co.il/why-you-should-almost-never-use-is-in-python/). For instance, on my computer I have:
>>> x = 100
>>> y = 100
>>> x is y
True
But:
>>> x = 10**1000
>>> y = 10**1000
>>> x is y
False
In fact, we can see that only the first 256 positive integers are pre-allocated:
>>> x = 0
>>> y = 0
>>> while True:
... if not x is y:
... print x
... break
... x += 1
... y += 1
...
257

Python reference model

I have a hard time to understand the python reference model
def changer(a,b):
a = 2
b[0] = 'spam'
X = 1
L = [1,2]
changer(X,L)
X,L
(1,['spam',2])
here comes my question, for assignment b[0] = 'spam' : I want to know how python modify the mutable object in this way(instead of create a new string objects and link the variable b to that object which will not affect the original object pointed by L)
thanks

a and b are both references to objects.
When you reassign a you change which object a is referencing.
When you reassign b[0] you are reassigning another reference contained within b. b itself still references the same list object that it did originally, which is also the list that was passed into the changer function.

Variables name are pointers to a special memory address ,so when you pass L and X to function the function does not create a new address with a,b just changed the labels !, so any changes on those actually change the value of that part of memory that X,L point to. So for refuse that you can use copy module :
>>> from copy import copy
>>> def changer(a,b):
... i = copy(a)
... j = copy(b)
... i = 2
... j[0] = 'spam'
...
>>> X = 1
>>> L = [1,2]
>>> changer(X,L)
>>> X,L
(1, [1, 2])

In Python, lists are mutable, and integers are immutable. This means that Python will never actually change the value of an integer stored in memory, it creates a new integer and points the variable at the new one.
The reason for this is to make Python's dynamic typing work. Unlike most languages, you can create a variable and store an int in it, then immediately store a string in it, or a float, etc.
MyVariable = 10 # This creates an integer and points MyVariable at it.
MyVariable = "hi" # Created a new string and MyVariable now points to that.
MyVariable = 30 # Created a new integer, and updated the pointer
So this is what happens in your code:
MyVar = 1 # An integer is created and MyVar points to it.
def Increase(num):
num = num + 1 #A new integer is created, the temp variable num points at it.
Increase(MyVar)
print(MyVar) # MyVar still points to the original integer
This is a 'feature' of dynamically typed languages ;)

Pointers in Python? ` x.pointerDest = y.pointerDest`?

I am breaking my old question to parts because it is very messy beast here. This question is related to this answer and this answer. I try to understand pointers, not sure even whether they exist in Python.
# Won't change x with y=4
>>> x = 0; y = x; y = 4; x
0
# Won't change y
>>> x = 0; y = x; x = 2; y
0
#so how can I change pointers? Do they even exist in Python?
x = 0
y.pointerDestination = x.pointerDestination #How? By which command?
x = 2
# y should be 0, how?
[Update 2: Solved]
Perhaps, contradictive statements about the existence There are no pointers in Python. and Python does not have the concept of a "pointer" to a simple scalar value.. Does the last one infer that there are pointers to something else, nullifying the first statement?

Scalar objects in Python are immutable. If you use a non-scalar object, such as a list, you can do this:
>>> x = [0]
>>> y = x
>>> y[0] = 4
>>> y
[4]
>>> x
[4]
>>> x is y
True
Python does not have the concept of a "pointer" to a simple scalar value.

Don't confuse pointers to references. They are not the same thing. A pointer is simply an address to an object. You don't really have access to the address of an object in python, only references to them.
When you assign an object to a variable, you are assigning a reference to some object to the variable.
x = 0
# x is a reference to an object `0`
y = [0]
# y is a reference to an object `[0]`
Some objects in python are mutable, meaning you can change properties of the object. Others are immutable, meaning you cannot change the properties of the object.
int (a scalar object) is immutable. There isn't a property of an int that you could change (aka mutating).
# suppose ints had a `value` property which stores the value
x.value = 20 # does not work
list (a non-scalar object) on the other hand is mutable. You can change individual elements of the list to refer to something else.
y[0] = 20 # changes the 0th element of the list to `20`
In the examples you've shown:
>>> x = [0]
>>> y = [x]
you're not dealing with pointers, you're dealing with references to lists with certain values. x is a list that contains a single integer 0. y is a list that contains a reference to whatever x refers to (in this case, the list [0]).
You can change the contents of x like so:
>>> print(x)
[0]
>>> x[0] = 2
>>> print(x)
[2]
You can change the contents of the list referenced by x through y's reference:
>>> print(x)
[2]
>>> print(y)
[[2]]
>>> y[0][0] = 5
>>> print(x)
[5]
>>> print(y)
[[5]]
You can change the contents of y to reference something else:
>>> print(y)
[[5]]
>>> y[0] = 12345
>>> print(x)
[5]
>>> print(y)
[12345]
It's basically the same semantics of a language such as Java or C#. You don't use pointers to objects directly (though you do indirectly since the implementations use pointers behind the scenes), but references to objects.

There are no pointers in Python. There are things called references (which, like C++ references, happen to be commonly implemented in pointers - but unlike C++ references don't imply pass-by-reference). Every variable stores a reference to an object allocated somewhere else (on the heap). Every collection stores references to objects allocated somewhere else. Every member of an object stores a reference to an object allocated somewhere else.
The simple expression x evaluates to the reference stored in x - whoever uses it has no way to determine that is came from a variable. There's no way to get a link to a variable (as opposed to the contents of it) that could be used to track changes of that variable. Item (x[y] = ...) and member (x.y = ...) assignments are different in one regard: They invoke methods and mutate existing objects instead of overwriting a local variable. This difference is mainly important when dealing with scoping, but you can use either of those to emulate mutability for immutable types (as shown by #Greg Hewgill) and to share state changes across function boundaries (def f(x): x = 0 doesn't change anything, but def g(x): x.x = 0 does). It's not fully up to emulating pass by reference though - unless you replace every variable by a wrapper object whose sole purpose is to hold a mutable val property. This would be the equivalent to emulating pass-by-reference through pointers in C, but much more cumbersome.