This question already has answers here:
Why does += behave unexpectedly on lists?
(9 answers)
Closed last month.
I had learned that n = n + v and n += v are the same. Until this;
def assign_value(n, v):
n += v
print(n)
l1 = [1, 2, 3]
l2 = [4, 5, 6]
assign_value(l1, l2)
print(l1)
The output will be:
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]
Now when I use the expanded version:
def assign_value(n, v):
n = n + v
print(n)
l1 = [1, 2, 3]
l2 = [4, 5, 6]
assign_value(l1, l2)
print(l1)
The output will be:
[1, 2, 3, 4, 5, 6]
[1, 2, 3]
Using the += has a different result with the fully expanded operation. What is causing this?
Thats because in the first implementation you are editing the list n itself (and therefore the changes still apply when leaving the function), while on the other implementation you are creating a new temporary list with the same name, so when you leave the function the new list disappears and the variable n is linked to the original list.
the += operator works similarly to x=x+y for immutable objects (since they always create new objects), but for mutable objects such as lists they work differently. x=x+y creats a new object x while x+=y edits the current object.
It may seem counter-intuitive, but they are not always the same. In fact,
a = a + b means a = a.__add__(b), creating a new object
a += b means a = a.__iadd__(b), mutating the object
__iadd__, if absent, defaults to the __add__, but it also can (and it does, in the case of lists) mutate the original object in-place.
This works on how python treats objects and passes variables into functions.
Basically - in first example (with += )
You are passing n and v into function by "pass-by-assignment"
So n gets modified and it will be also modified out of function scope.
In second example - n is reassigned inside of the function to a new list. Which is not seen outside of the function.
In your 1st code. You changes list n itself see the below image..!
In your 2nd code. you just created a temporary list which is cleared when function call ends.. see the below images..!
In the next step when function ends the temporary list clear!!
Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
I've just started working with Python, and I just came across some behavior that I don't understand. I've searched the site for an explanation, but I haven't been able to find it. Perhaps I don't know the right keywords to search.
I (think I)'m trying to define a list recursively, but in stead of repeatedly appending a new item to the list, all items in the list get replaced by this new item. See the snippet below. The code is supposed to generate a list I containing all (ordered) sublists of [0,...,n] of length d.
n = 5
d = 2
def next(S):
m = S.index(min([s for s in S if s+1 not in S]))
for i in range(m):
S[i] = i
S[m] += 1
return S
I = [[i for i in range(d)]]
while I[-1][0] <= n-d:
I += [next(I[-1])]
print(I)
I expected this to return the following list:
[[0,1],[0,2],[1,2],[0,3],[1,3],[2,3],[0,4],[1,4],[2,4],[3,4],[0,5],[1,5],[2,5],[3,5],[4,5]]
But in stead it returns the following list:
[[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5],[4,5]]
Can anyone point me to an explanation as to why this code does not do what I expect it to? Thanks in advance.
Each call to next should get its own copy of the list; for example
I += [next(I[-1][:])]
Your code was building a list with multiple references to the same list. By sending each call to next its own copy, all of the elements in the result are distinct.
This could also have been accomplished by changing next to build a new list from scratch instead of modifying the list it is passed.
Your function next() alters the list it is given in-place, instead of creating a new copy. This is why all items in I are actually just the same list.
A statement like S = list(S) in next() would be sufficient to replace S with a (shallow) copy and ensure that changes to it are not applied to the original list.
You're passing a reference to an existing list rather than a copy of the list, so the list is modified as you iterate. The following might help explain this behavior:
def modify(li, i=3):
print(li)
li += [i] # This adds to the passed list
if i > 0:
modify(li, i-1)
def modify_copy(li, i=3):
print(li)
li = li + [i] # This overwrites "li" on every call
if i > 0:
modify_copy(li, i-1)
one = [4]
modify(one)
print("Result:", one)
two = [4]
modify_copy(two)
print("Result:", two)
Output
[4]
[4, 3]
[4, 3, 2]
[4, 3, 2, 1]
Result: [4, 3, 2, 1, 0]
[4]
[4, 3]
[4, 3, 2]
[4, 3, 2, 1]
Result: [4]
This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 2 years ago.
As far as I know, python passes parameter as reference. I have following code
def func(arr):
print(arr)
if arr == [] :
return
for i in range(len(arr)):
arr[i] *= 2
func(arr[1:])
r = [1,1,1,1]
func(r)
print(r)
I would expect the output to be [2,4,8,16].
Why it outputs [2,2,2,2] as if the reference only works for one level of recursion?
maybe 'arr[1:]' always creates a new object? If that is the case, is there any way to make arr[1:] work?
You've asked a couple different questions, let's go through them one by one.
As far as I know, python passes parameter as reference.
The correct term for how python passes it's arguments is "pass py assignment". It means that parameters inside the function behave similar to how they would if they had been directly assigned with an = sign. (So, mutations will reflect across all references to the object. So far so good)
Sidenote (skip if this is confusing): For all intents and purposes, the distinction of "pass by assignment" is important because it abstracts away pass by value vs pass by reference, concepts that are not exposed in python directly. If you wish to know how the underlying mechanism works, it's actually a pass by value, but every value itself is a reference to an object (equivalent to a first level pointer in C speak). We can see why it's easier and important initially not to worry about this particular abstraction, and use "pass by assignment" as the more intuitive explanation.
Next,
maybe 'arr[1:]' always creates a new object?
Correct, slicing always creates a shallow copy of the list. docs
If that is the case, is there any way to make arr[1:] work?
Not directly, but we can use indexes instead to build a solution that works and gives us the output you desire. Just keep track of a starting index while doing the recursion, and increment it as you continue recursing.
def func(arr, start=0):
print(arr)
if arr[start:] == [] :
return
for i in range(start, len(arr)):
arr[i] *= 2
func(arr, start + 1)
r = [1,1,1,1]
func(r)
print(r)
Output:
[1, 1, 1, 1]
[2, 2, 2, 2]
[2, 4, 4, 4]
[2, 4, 8, 8]
[2, 4, 8, 16]
[2, 4, 8, 16]
You do slice arr[1:] and as the result it creates new list. That's why you got such result, in future I would not recommend you do such thing it's implicit and hard to debug. Try to return new value instead of changing it by reference when you work with functions
For example like this:
def multiplier(arr):
return [
value * (2 ** idx)
for idx, value in enumerate(arr, start=1)
]
result = multiplier([1, 1, 1, 1])
print(result) # [2, 4, 8, 16]
Slicing a list will create a new object (as you speculated), which would explain why the original list isn't updated after the first call.
Yes, arr[1:] create a new object. You can pass the index to indicate the starting index.
def func(arr, start_idx):
print(arr)
if arr == [] :
return
for i in range(start_idx, len(arr)):
arr[i] *= 2
func(arr, start_idx + 1)
r = [1,1,1,1]
func(r, 0)
print(r)
You can use this. The variable s represents start and e represents end, of the array.
def func(arr,s,e):
print(arr) #comment this line if u dont want the output steps
if s>=e:
return
for i in range(s,e):
arr[i] *= 2
func(arr,s+1,e)
r = [1,1,1,1]
func(r,0,len(r))
print(r)
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
Simple example:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print myList
prints
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]
while:
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
prints
[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]
Conclusion:
Lists can be modified in place using global list access Lists can
List items can NOT be modified in place using the iterator object
All example code I can find uses the global list accessors to modify the list inplace.
Is it so evil to modify a list iterator?
The reason obj += 1 does not do what you expect is that this statement does not modify obj in-place. Instead, it computes the new value, and rebinds the variable obj to point to the new value. This means that the contents of the list remain unchanged.
In general it is possible to modify the list while iterating over it using for obj in myList. For example:
myList = [[1], [2], [3], [4], [5]]
for obj in myList:
obj[0] += 1
print(myList)
This prints out:
[[2], [3], [4], [5], [6]]
The difference between this and your first example is that here, the list contains mutable objects, and the code modifies those objects in-place.
Note that one could also write the loop using a list comprehension:
myList = [val+1 for val in myList]
I think you've misunderstood what an "iterator object" is. A for loop is not an iterator object. For all intents and purposes, a for loop like this:
myList = [0, 1, 2, 3, 4]
for x in myList:
print x
does this (but more efficiently and less verbosely):
i = 0
while i < len(myList)
x = myList[i]
print x
i += 1
So you see, any changes made to x are lost as soon as the next loop starts, because the value of x is overwritten by the value of the next item in the list.
As others have observed, it is possible to alter the value of a list while iterating over it. (But don't change its length! That's where you get into trouble.) One elegant way to do so is as follows:
for i, x in enumerate(myList):
myList[i] = some_func(x)
Update: It's also important to understand that no copying goes on in a for loop. In the above example, i and x -- like all variables in Python -- are more like pointers in C/C++. As the for loop progresses, obj points at myList[0], myList[1], etc, in turn. And like a C/C++ pointer, the properties of the object pointed to are not changed when the pointer is changed. But also like a C pointer, you can directly modify the thing pointed at, because it's not a copy. In C, this is done by dereferencing the pointer; in Python, this is done by using a mutable object. That's why NPE's answer works. If i and x were even shallow copies, it wouldn't be possible to do what he does.
The reason you can't directly change ints the way you can change lists (as in NPE's answer), is that ints aren't mutable. Once a 5 object is created, nothing can change its value. That's why passing around a pointer to 5 is safe in Python -- no side-effects can occur, because the thing pointed to is immutable.
in for obj in myList:, in every iteration, obj is a (shallow) copy of the element in myList. So the change on the obj does nothing to myList's elements.
It's different with the Perl for my $obj (#myList) {}
You are confused. Consider your first snippet:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print a
obj is not some kind of magical pointer into the list. It is a variable which holds a reference to an object which happens to also be in myList. obj += 1 has the effect of increasing the value stored in obj. Your code then does nothing with that value.
To be clear: There are no copies in this code example. obj is a variable, which holds an object in the list. That is all.
In the first example the integer is copied into obj which is increased by 1.
The list is not changed.
If you would use a class instance and perform operations on it, it would be changed.
Modification in list is allowed. Your code examples arbove are pretty garbled...
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
This works.