I am trying to understand chain assignment in Python.
If I run x = x[1] = [1, 2], I get an infinite list [1, [...]].
But if I run x = x[1:] = [1, 2], I will get a normal list [1, 1, 2].
How does it work in the background to make these two different results?
First, understand that in a chained assignment, the right-most expression is evaluated to an object. A reference to that object is then assigned to each target in sequence, from left to right. x = y = z is effectively the same as
t = z # A new temporary variable "t" to hold the result of evaluating z
x = t
y = t
del t
Second, you need to understand the difference between assigning t to the subscription x[1] and to the slicing x[1:]. In the former, element 1 of x is replaced by t. In the latter, elements x[1], x[2], etc are replaced by t[0], t[1], etc, extending or shrinking x as necessary.
The first example is equivalent to
t = [1,2]
x = t
x[1] = t
del t
x[1] becomes a reference to x itself; list.__repr__ can detect cycles like this and represents them with .... x, x[1], x[1][1], etc are all references to the original list [1,2].
The second example is equivalent to
t = [1,2]
x = t
x[1:] = t
del t
In this case, no cycles are created. x[1] is replaced by a reference to 1, not x/t itself, and x is extended to make x[2] a reference to 2. (If it helps, think of x[1:] = t as producing the same result as x = [x[0]]; x.extend(t).)
x = x[1] = [1, 2]
In this case, x and x[1] are the same object. Therefore x contains itself, which is represented by ... in your output.
x = x[1:] = [1, 2]
In this case, x is a slice, and slices are copies. so they are not the same object.
Explaining the outputs:
x = x[1] = [1, 2] # Output: [1, [...]]
The reason is that the list is not copied, but referenced.
x[1] is a reference to the list [1, 2].
When you assign x[1] to x, you are assigning a reference to the list [1, 2] to x.
So x and x[1] are both references to the same list.
When you change the list through one of the references, the change appears in the other reference as well. This is called aliasing.
x = x[1:] = [1, 2] # Output: [1, 1, 2]
The reason is that the list is copied, not referenced.
x[1:] is slice of the list [1, 2].
The slice is a copy of the list, not a reference to the list.
When you assign x[1:] to x, you are assigning a copy of the list [1, 2] to x.
So x and x[1:] are both copies of the same list.
When you change the list through one of the references, the change does not appear in the other reference.
Related
I am quite confused, which case is correct, first case or second case or both the cases are correct in Python?
Why in first case the list x remain unchanged?
why in second case the list x also got modified?
Case 1:
x = [0, 1, 2]
def cleanIt(y):
y = []
return y
print(cleanIt(x)) #[]
print(x) #[0, 1, 2]
Case 2:
x = [0, 1, 2]
def appendOne(z):
z.append(4)
return z
print(appendOne(x)) #[0, 1, 2, 4]
print(x) #[0, 1, 2, 4]
Python behaves like pass-by-object-reference. Therefore what you really pass into the functions are actually references to the objects you give. In the first example, you pass a reference that points to the memory address in which x is present. However when you do the assignment operation y = [], you simply change the memory address your reference points to. You do not change the content of x. In the second example, you call the method append(). In that case, you call this method for the object that is present in the memory address pointed to by your reference.
These may come confusing at the beginning, but you should remember that assigning your variable directly to something else doesn't actually change your variable, it changes where your reference points to. However using a method, modifying your variable updates it.
I have used id() to give you a clear idea of the references. id() returns the memory address of the argument passed to it.
Case-1
x = [0, 1, 2]
print(id(x))
def cleanIt(y):
print(id(y))
y = []
print(id(y))
return y
print(cleanIt(x))
print(x)
Output:
139677941202304
139677941202304
139677940574528
[]
[0, 1, 2]
You pass the reference of x to cleanIt.
y = [] - Creates a new list altogether. This has nothing to do with x. See the references above.
return y - You return the newly created list i.e, y = []
Since x is unchanged, it will retain its initial values.
Case-2
x = [0, 1, 2]
print(id(x))
def appendOne(z):
print(id(z))
z.append(4)
print(id(z))
return z
print(appendOne(x))
print(x)
Output
139677940574912
139677940574912
139677940574912
[0, 1, 2, 4]
[0, 1, 2, 4]
You pass the reference of x to appendOne(). x and z point to the same list.
z.append(4) - Appends a 4 to list referenced by z.
Since z and x refer to the same list, changes made in z reflects in x too. See the references above.
In Python values are sent to functions by means of object reference.
Please read this to know more about this concept.
Edit:
If I want to have x unchanged while z should be [0,1,2,4], then how should I modify the python code?
This is how you can do.
x = [0, 1, 2]
print(id(x))
def appendOne(z):
print(id(z))
# Creates a new list with contents of x.
z = list(z)
z.append(4)
print(id(z))
return z
print(appendOne(x))
print(x)
Output
140368975642112
140368975642112
140368975011392
[0, 1, 2, 4]
[0, 1, 2]
I can't seem to understand following behavior in python:
x = [0, [1,2,3,4,5],[6]]
y = list(x)
y[0] = 10
y[2][0] = 7
print x
print y
It Outputs:
[0, [1, 2, 3, 4, 5], [7]]
[10, [1, 2, 3, 4, 5], [7]]
Why is second index of x and y updated and only the first index of y?
This happens because list(x) creates a shallow copy of the list x. Some of the elements in x are lists themselves. No copies are created for them; they are instead passed as references. In this way x and y end up having a reference to the same list as an element.
If you want to create a deep copy of x (i.e. to also copy the sublists) use:
import copy
y = copy.deepcopy(x)
In Python,sequence are divided into mutable sequence,which can be changed after they are created, and immutable sequence.For immutable sequences(string,Unicode,Tuples),Python make a copy for them.For mutable sequences(Lists,Byte Arrays), Python make a reference for them.
So if you change x ,y will also be changed as as they have a reference to the same list.
The standard type hierarchy
I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]
Please don't be harsh if my question is very simple or obvious.
Am a Python newbie, so just started out.
Actually this was a piece of code I came across on this Stack Overflow only but could not find an answer for why this is happening, so decided to ask it myself.
I wrote the below two programs :
1)
x=[1,2,3]
y=x
print x
y=y+[3,2,1]
print x
Output:
[1,2,3]
[1,2,3]
2)
x=[1,2,3]
y=x
print x
y+=[3,2,1]
print x
Output:
[1,2,3]
[1,2,3,3,2,1]
I can't understand why the two outputs are different in this case ?
is y=y+(something) not the same as y+=(something)
what is it that I am missing here ?
Thanks a lot for helping me out on this
This has nothing to do with mutability. It has to do with list objects and references to it.
When you do:
y=x
You are essentially making y refer to the same list that x is referring to.
Now,
y=y+[3,2,1]
^-------^ - Create a **new** List is that is equal to concatenation of `y` and [1,2,3]
^---^ - Bind variable `y` to this **new** list.
- The original list which `x` refered to -- is still *intact*
Again,
y+=[3,2,1]
^-------^ - Append [1,2,3] **in-place**.
- Since even `x` is pointing to the same list, it gets modified.
y = y+something will replace the contents of y, while y+=something will modify the list in-place.
TL;DR;
Code explanation:
>>> x = [1, 2]
>>> y = x
>>> id(y) == id(x)
True
>>> y += [3]
>>> id(y) == id(x)
True
>>> y = y + [3]
>>> id(y) == id(x)
False
Here's a little explanation of what's going on in your code.\
First code
You declared x:
x=[1,2,3]
Then you point y to the value of x:
y=x
After that, here's the tricky part:
y=y+[3,2,1]
This creates a new y variable, replacing the old one. y is no longer related to x. Now, x has it's own place in memory, and y has a separate one
Second code
Declare x:
x=[1,2,3]
Points y to the value of x:
y=x
And lastly:
y+=[3,2,1]
This modifies the list pointed by y in-place. In other words, you're modifying x as well:
If this isn't clear enough, just comment:)
Hope this helps!
After the line
y = x
Both x and y point to the same object. In way that operators are overloadable in Python:
y += [3, 2, 1]
Is calling function
y.__iadd__([3, 2, 1])
Which in turn adds the elements of [3, 2, 1] to a list pointed by y - which happens to be the same list pointed by x.
On the other hand:
y = y + [3, 2, 1]
Is the same as:
y = y.__add__([3, 2, 1])
Which, in turn, creates a new list containing combination of both and assign it to y. After it y and x points to different objects and the one pointed by x is not modified.
No, it's not the same. In fact this was one of the reasons why it took a long time for Python to get +=, in the beginning people thought it would be too confusing to add.
y = y + [3, 2, 1]
Rebinds y to a new value, the one that is the result of the expression y + [3, 2, 1].
But
y += [3, 2, 1]
changes y, if it currently refers to something that can be changed. And lists can.
Since the list that y refers to is changed, and x refers to the same list, printing x also shows the new value.
If y were immutable (like a tuple), then it doesn't work that way:
x = y = (1, 2, 3)
y += (3, 2, 1)
print(x) # prints (1, 2, 3)
print(y) # prints (1, 2, 3, 3, 2, 1)
The reason is that in the case of mutable objects, it's more likely that changing the object is what you actually want, so they made it work that way. Immutable object can't work that way, so they don't.
In python every time you use the operator = you "delete" the old version and create a new one.
Thus y = y + something will create a new variable of y that isn't the same as the old y.
While y+=something will add something to the current value in y.
I'm having trouble understanding the odd behaviour in python functions if i pass in a list.
I made the following functions:
def func(x):
y = [4, 5, 6]
x = y
def funcsecond(x):
y = [4, 5, 6]
x[1] = y[1]
x = [1, 2, 3]
When i call func(x) and then print out x , it prints out [1, 2, 3], just the way x was before, it doesnt assign the list y to x. However, if i call funcsecond(x), it assigns 5 to the second position of x. Why is that so? When i assign the whole list, it doesn't do anything, but when i assign only one element, it changes the list i originally called it with.
Thank you very much and i hope you understand what im intending to say , i'm having a hard time expressing myself in English.
The former rebinds the name, the latter mutates the object. Changes to the name only exist in local scope, whereas a mutated object remains mutated after the scope is exited.
this is happening beacuse x points to a object which is mutable.
def func(x): # here x is a local variable which refers to the object[1,2,3]
y = [4, 5, 6]
x = y #now the local variable x refers to the object [4,5,6]
def funcsecond(x): # here x is a local variable which refers to the object[1,2,3]
y = [4, 5, 6]
x[1] = y[1] # it means [1,2,3][1]=5 , means you changed the object x was pointing to
x = [1, 2, 3]