Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
I have a long list where each element is a list of length 2. The first element of each is a list is a string and the second element of each list is an integer corresponding to the string.
I want to loop through the long "parent" list and delete any "child" lists where the integer is less than three. This is my code.
for i in range(len(fontsizenum) / 2):
if int(fontsizenum[i][1]) < 3:
del fontsizenum[i]
However, it is not working as when I print the list afterwards, it still contains values with numbers less than three.
Say this is the list that I am altering.
fontsizenum = [[cereal, 1], [dog, 4], [cat, 2], [water, 5]]
The expected output is [[dog, 4], [water, 5]].
However, the actual output for me right now is still the original, unchanged list.
The resulting list still contains unexpected values because you are modifying the list while iterating over it. In almost all cases, you should avoid modifying an iterable while iterating over it.
If you change your code to the following, the resulting list should be what you expect.
new_lst = []
for idx, value in enumerate(lst):
if value[1] >= 3:
new_lst.append(value)
As noted in the comments by #AntonvBR, the above snippet can be simplified to the following list comprehension
[i for i in lst if i[1] > 3]
Example
If lst is set to
[
['forbidden', 1],
['hath', 1],
['causes', 2],
['whose', 3],
]
then the resulting list will be
[['whose', 3]]
Explanation
My code snippet creates a new list that contains elements from the old list. Notably, this allows us to avoid modifying the old list.
I changed the condition in the if-statement to check whether to include, rather than to exclude, an element. Finally, if that check is satisfied, then I append the value to the new list.
You want to use a simple list comprehension for this:
expectedoutput = [i for i in fontsizenum if i[1] >= 3]
I was trying to modify the values in lists via slices and for-loops, and ran into some pretty interesting behavior. I would appreciate if someone could explain what's happening internally here.
>>> x = [1,2,3,4,5]
>>> x[:2] = [6,7] #slices can be modified
>>> x
[6, 7, 3, 4, 5]
>>> x[:2][0] = 8 #indices of slices cannot be modified
>>> x
[6, 7, 3, 4, 5]
>>> x[:2][:1] = [8] #slices of slices cannot be modified
>>> x
[6, 7, 3, 4, 5]
>>> for z in x: #this version of a for-loop cannot modify lists
... z += 1
...
>>> x
[6, 7, 3, 4, 5]
>>> for i in range(len(x)): #this version of a for-loop can modify lists
... x[i] += 1
...
>>> x
[7, 8, 4, 5, 6]
>>> y = x[:2] #if I assign a slice to a var, it can be modified...
>>> y[0] = 1
>>> y
[1, 8]
>>> x #...but it has no impact on the original list
[7, 8, 4, 5, 6]
Let's break down your comments 1 by 1:
1.) x[:2] = [6, 7] slices can be modified:
See these answers here. It's calling the __setitem__ method from the list object and assigning the slice to it. Each time you reference x[:2] a new slice object is created (you can simple do id(x[:2]) and it's apparent, not once will it be the same id).
2.) indices of slices cannot be modified:
That's not true. It couldn't be modified because you're performing the assignment on the slice instance, not the list, so it doesn't trigger the __setitem__ to be performed on the list. Also, int are immutable so it cannot be changed either way.
3.) slices of slices cannot be modified:
See above. Same reason - you are assigning to an instance of the slice and not modifying the list directly.
4.) this version of a for-loop cannot modify lists:
z being referenced here is the actual objects in the elements of x. If you ran the for loop with id(z) you'll note that they're identical to id(6), id(7), id(3), id(4), id(5). Even though list contains all 5 identical references, when you do z = ... you are only assigning the new value to the object z, not the object that is stored in list. If you want to modify the list, you'll need to assign it by index, for the same reason you can't expect 1 = 6 will turn x into [6, 2, 3, 4, 5].
5.) this version of a for-loop can modify lists:
See my answer above. Now you are directly performing item assignment on the list instead of its representation.
6.) if I assign a slice to a var, it can be modified:
If you've been following so far, you'll realize now you are assigning the instance of x[:2] to the object y, which is now a list. The story follows - you perform an item assignment by index on y, of course it will be updated.
7.) ...but it has no impact on the original list:
Of course. x and y are two different objects. id(x) != id(y), therefore any operation performed on x will not affect y whatsoever. if you however assigned y = x and then made a change to y, then yes, x will be affected as well.
To expand a bit on for z in x:, say you have a class foo() and assigned two instances of such to the list f:
f1 = foo()
f2 = foo()
f = [f1, f2]
f
# [<__main__.foo at 0x15f4b898>, <__main__.foo at 0x15f4d3c8>]
Note that the reference in question is the actual foo instance, not the object f1 and f2. So even if I did the following:
f1 = 'hello'
f
# [<__main__.foo at 0x15f4b898>, <__main__.foo at 0x15f4d3c8>]
f still remains unchanged since the foo instances remains the same even though object f1 now is assigned to a different value. For the same reason, whenever you make changes to z in for z in x:, you are only affecting the object z, but nothing in the list is changed until you update x by index.
If however the object have attribute or is mutable, you can directly update the referenced object in the loop:
x = ['foo']
y = ['foo']
lst = [x,y]
lst
# [['foo'], ['foo']]
for z in lst:
z.append('bar')
lst
# [['foo', 'bar'], ['foo', 'bar']]
x.append('something')
lst
# [['foo', 'bar', 'something'], ['foo', 'bar']]
That is because you are directly updating the object in reference instead of assigning to object z. If you however assigned x or y to a new object, lst will not be affected.
There is nothing odd happening here. Any slice that you obtain from a list is a new object containing copies of your original list. The same is true for tuples.
When you iterate through your list, you get the object which the iteration yields. Since ints are immutable in Python you can't change the state of int objects. Each time you add two ints a new int object is created. So your "version of a for-loop [which] cannot modify lists" is not really trying to modify anything because it will not assign the result of the addition back to the list.
Maybe you can guess now why your second approach is different. It uses a special slicing syntax which is not really creating a slice of your list and allows you to assign to the list (documentation). The newly created object created by the addition operation is stored in the list through this method.
For understanding your last (and your first) examples, it is important to know that slicing creates (at least for lists and tuples, technically you could override this in your own classes) a partial copy of your list. Any change to this new object will, as you already found out, not change anything in your original list.
This question already has answers here:
How does assignment work with list slices?
(5 answers)
Closed 5 years ago.
I came across the following code (sort of):
my_list = [1, [2, 3, 4], 5]
my_list[1:2] = my_list[1]
After running these two lines, the variable my_list will be [1, 2, 3, 4, 5]. Pretty useful for expanding nested lists.
But why does it actually do what it does?
I would have assumed that the statement my_list[1:2] = my_list[1] would do one of the following:
simply put [2, 3, 4] into the second position in the list (where it already is)
give some kind of "too many values to unpack" error, from trying to put three values (namely 2,3,4) into a container of only length 1 (namely my_list[1:2]). (Repeating the above with a Numpy array instead of a list results in a similar error.)
Other questions (e.g. How assignment works with python list slice) tend to not pay much attention to the discrepancy between the size of the slice to be replaced, and the size of the items you're replacing it with. (Let alone explaining why it works the way it does.)
Slice assignment replaces the specified part of the list with the iterable on the right-hand side, which may have a different length than the slice. Taking the question at face value, the reason why this is so is because it's convenient.
You are not really assigning to the slice, i.e. Python doesn't produce a slice object that contains the specified values from the list and then changes these values. One reason that wouldn't work is that slicing returns a new list, so this operation wouldn't change the original list.
Also see this question, which emphasizes that slicing and slice assignment are totally different.
Here is the relevant bit from the Python Language Reference
If the target is a slicing: The primary expression in the reference is
evaluated. It should yield a mutable sequence object (such as a list).
The assigned object should be a sequence object of the same type.
Next, the lower and upper bound expressions are evaluated, insofar
they are present; defaults are zero and the sequence’s length. The
bounds should evaluate to integers. If either bound is negative, the
sequence’s length is added to it. The resulting bounds are clipped to
lie between zero and the sequence’s length, inclusive. Finally, the
sequence object is asked to replace the slice with the items of the
assigned sequence. The length of the slice may be different from the
length of the assigned sequence, thus changing the length of the
target sequence, if the target sequence allows it.
This behavior makes sense qualitatively because when you slice a list you get a sub list so replacing that with another list shouldn't add a level of nesting. Allowing it to change the length of the list is a design choice. Other choices are possible as your numpy example demonstrates.
Short Answer:
my_list[1:2] = my_list[1] will replaced the content from 1st index to 2nd index of my_list with the content of present in 1st index of
my_list
Explanation:
Let's see two slicing operations, very similar but totally distinct
This creates the copy of list and stores it the variable
some_variable = some_list[2:5]
This replaces the content of the list inplace, which permits changing the length of the list too.
some_list[2:5] = [1, 2, 3, 4]
When you use assignment operator =, it invokes a __setitem__ function. Our focus here is the case 2 above. As per the Python's Assignment Statement document:
If the target is a slicing: The primary expression in the reference is
evaluated. It should yield a mutable sequence object (such as a list).
The assigned object should be a sequence object of the same type.
Next, the lower and upper bound expressions are evaluated, insofar
they are present; defaults are zero and the sequence’s length. The
bounds should evaluate to integers. If either bound is negative, the
sequence’s length is added to it. The resulting bounds are clipped to
lie between zero and the sequence’s length, inclusive. Finally, the
sequence object is asked to replace the slice with the items of the
assigned sequence. The length of the slice may be different from the
length of the assigned sequence, thus changing the length of the
target sequence, if the target sequence allows it.
In our case my_list[1:2] = my_list[1], python will also call __setitem__ as:
my_list.__setitem__(slice(1,2,None), [2, 3, 4])
Refer slice document to know what it does.
So, when you did my_list[1:2] = my_list[1], you replaced the content from 1st index to 2nd index of my_list with the content of present in 1st index of my_list i.e. [2, 3, 4].
I think now we can answer why your assumptions are incorrect:
put [2, 3, 4] into the second position in the list (where it already is)
No. Because __setitem__ is not called on the index but on the slice of the indices which you passed.
give some kind of "too many values to unpack" error, from trying to put three values (namely 2,3,4) into a container of only length 1 (namely my_list[1:2]).
Again No. Because the range of your indices creating your container is replaced with new set of values.
What you are doing is slice assignment.
Assignment to slices is also possible, and this can even change the size of the list or clear it entirely
my_list[1:2] = my_list[1]
This replaces the slice of my_list with the contents of my_list[1].
By specifying my_list[1:2] on the left side of the assignment operator =, you are telling Python you want to use slice assignment.
my_list[1:2] = my_list[1] is equivalent to my_list.__setitem__(slice(1, 2, None), my_list[1])
In slice(1, 2, None), 1 is start, 2 is stop, and None is step and is optional.
What you are trying here is called Slice Assingnment. In python it is possible to assign an iterable(my_list[1] in your case) to a slice of another iterable(my_list[0:1] in your case). Lets walk through some examples to understand what it really means:
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:3] = b
>>> l
>>> [6, 7, 8, 4, 5]
So what happened here is the portion of list l for 0,1,2 indices is
which covers elements 1,2,3is replaced by elements of list b 6,7,8. However in this case size of slice and replaced elements happens to be equal by chance.
So what happens when slice size and iterable to be replaced are not equal
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:4] = b
>>> l
>>> [6,7,8,5]
Notice that this operation didn't produce any error, instead, it just copied whatever elements are available with the entire sliced portion. In this case, sliced elements are 1,2,3,4 replaced by 6,7,8
In the previous example iterable to be replaced was smaller. What happens if slice portion is smaller
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:1] = b
>>> l
>>> [6,7,8,2,3,4,5]
So now we can see that only first element is replaced by entire iterable b.
You can also use this behaviour to remove a specific portion of the list ( Which I find convenient in some situations ).
>>> l = [1,2,3,4,5]
>>> l[0:2] = []
>>> l
>>> [3,4,5]
First two elements are removed very conveniently here.
So the example in your question is similar to the examples I posted above, except that in your case there is an additional step of unpacking list values. Unpacking list value happens every time when you assign list to another list. A short example
>>> l = [[1]]
>>> a = []
>>> a = l[0]
>>> a
>>> [1]
Your example now:
#Replace the slice [0:1] with my_list[1] perform unpacking list values as well
>>> my_list[1:2] = my_list[1]
>>> [1,2,3,4,5]
Also note that slice assignment is only possible if you assign iterable to a slice. If you try to assign an int or something that is not an iterable to a slice python will throw an error.
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:1] = b[1]
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only assign an iterable
That's why in your case my_list[1] don't raise an error since it is an iterable.
That what you are doing is insert an element through slicing. I will explain everything by parts. More than an inserting it could be interpreted as adding an item to your list after slicing target list in a range desired. Now to explain every line in detail:
my_list[1:2]
That part is like saying to Python; "I want to get the values (slice) from index 1 to the index before 2 (excluding 2, I will explain with another example later on)". After that you assign a value to those values:
my_list[1:2] = my_list[1] #The same as my_list[1:2] = [2,3,4]
Now that you know what the first part does, next it is going to add the item at the right side of the '=' operator so you could interpret it like this; "I want to slice from index 1 to everything before 2 (again, excluding index 2) and then add my list [2,3,4]". Now here comes another examples so you understand even better I hope.
problemList = [1, [2, 3, 4], 5]
problemList[1:2] = problemList[1] #The same as problemList[1:2] = [2,3,4]
analogProblemL = [1] + [2,3,4] + [5] #Output : [1,2,3,4,5]
insertList = [12,13,14]
myList = [1, [2, 3, 4], 5,6,7,8,9]
myList[3:6] = insertList
analogFstList = [1,[2,3,4] ,5] + insertList + [9] #Output : [1,[2,3,4],5,12,13,14,9]
myScnList = [1, [2, 3, 4], 5]
myScnList[1:3] = [2,3,4]
analogScnList = [1] + [2,3,4] + [5] #Output : [1,2,3,4,5]
The next lines will be like if it was an animation frames so it's easier to interpret:
[1,2,3,4,5] #List before line of code: myList[1:3] = [12,13,14]
[1,|2,3|,4,5] #The moment where you slice the list with: myList[1:3]. Please notice that the characters used '|' are for representing the slice.
[1] + [12,13,14] + [4,5] #After assigning what the slice should be changed for. It's like excracting from the whole list the values [2,3] and changing it for [12,13,14].
[1,12,13,14,4,5] #Final list after running the code.
Some references used for this answer:
http://effbot.org/zone/python-list.htm
Understanding Python's slice notation
How assignment works with python list slice
Hope it was useful for you.
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.