Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
If I use a list as a parameter for a function, I would expect that the original list would be unmodified. Why is it when I use the code below, x == z is True when it should be x == range(10) is True?
In [83]: def pop_three(collection):
....: new_collection = []
....: new_collection.append(collection.pop())
....: new_collection.append(collection.pop())
....: new_collection.append(collection.pop())
....: return new_collection, collection
....:
In [84]: x = range(10)
In [85]: x
Out[85]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [86]: y, z = pop_three(x)
In [87]: y
Out[87]: [9, 8, 7]
In [88]: z
Out[88]: [0, 1, 2, 3, 4, 5, 6]
In [89]: x
Out[89]: [0, 1, 2, 3, 4, 5, 6]
If I use a list as a parameter for a function, I would expect that the original list would be unmodified.
No. I think what you're missing here is that Python never automatically copies anything.
If you're used to a language like, say, C++, Python is very different. Ned Batchelder, as always, explains this brilliantly, but let me summarize:
In C++, a function parameter, or a variable, or an array element, is a location in memory, with a type. When you write a = b, or call f(b), that copies the value in memory location b to memory location a (possibly calling a casting operator or conversion constructor or copy constructor or copy assignment operator—or moving versions of half of those… but we're not here to learn about the intricacies of C++).
In Python, a function parameter, or a variable, or a list element, is just a name. The value has its own typed memory location, somewhere else. You can have as many names as you want for the same object. When you write a = b or call f(b), that just makes another name for the same object.
In particular, in your case, collection and x are names for the same object. If you added print(id(collection)) before the call and print(id(x)) inside the function, they would both print out the same number.
So, in Python, if you want a copy, you have to ask for it explicitly—e.g., pop_three(x[:]) or pop_three(copy.copy(x)).
Or, of course, you could do that inside pop_three.
Or, best of all, you could just avoid using mutating methods like pop in the first place:
def pop_three(collection):
return collection[-3:][::-1], collection[:-3]
I was confused since strings or integers passed into a function can be treated in this fashion.
Well, they can be treated in this fashion exactly as far as lists can. If you never mutate anything, the distinction between separate copies and shared references is irrelevant.* Because Python strings and integers are immutable, the distinction can't come up in any code with strings or integers are arguments. Lists, on the other hand, are mutable, so the distinction can arise—but it's perfectly possible, and in fact often best, to write code that uses them without mutating anything, as in the final example above.
Notice that this assumes the first point above: that, unlike C++ assignment, Python assignment is not a form of mutation. But it can get tricky at the edges. For example, is x[0] = 1 a mutation? Well, it's not a mutation of x[0], but it is a mutation of x. What about x += y? The language leaves that up to the type of x, and for mutable types it usually (but not always) is a mutation, while for immutable types of course it isn't.
* This isn't quite true; you can write code that relies on identity even though it isn't relevant, e.g., by using the id function or the is operator. Such code is almost never Pythonic, or a good idea… but it can be helpful to learning how things work. Try to print(id(x)) suggestion above with x = "a" + "b". However, this can be misleading, because the Python interpreter is allowed to "intern" objects that it knows are guaranteed to be immutable, and it does so with, e.g., small integers, and the Python compiler is allowed to fold constants, and it does so with, e.g., literal strings.
Because when you pass x to the pop_tree() and you called collection.pop(). This effectively did x.pop()
Therefore when you turned from the funciton, x only got 7 elements
Lists are mutable. When you pop from a list, you modify the list.
Any variable that originally pointed to the list will continue to point to the same modified list. Any new variable that you point at the list will point to the same modified list as well.
z = pop_three(x) means you are passing in the list x and then mutating that list x using collection.pop() x so x is being mutated therefore could not be == range(10)
y becomes new_collection and z is collection which is the mutated returned value of x.
In [2]: x = range(10)
In [3]: y, z = pop_three(x)
In [4]: z is x # z is the object x
Out[4]: True
A number of Python's list methods operate in place and return None (off the top of my head, insert, sort, reverse).
However, there is one behavior that frequently frustrates me. If I create a new list, which normally returns an object, and insert on it at the same time, the new list "disappears":
mytup = (0, 1, 2, 3, 4)
print mytup # (0, 1, 2, 3, 4)
mylist = list(mytup)
print mylist # [0, 1, 2, 3, 4]
newlist = list(mytup).insert(0, 10)
print newlist # None
So if I want to modify a tuple, it requires more lines:
newlist = list(mytup)
newlist.insert(0, 10)
print newlist # [10, 0, 1, 2, 3, 4]
So I have two questions:
Is it correct to say that when I call the list constructor, it returns the object, but when I call the list constructor with a method on it, the method "overrides" the return with None? Again, where does the list go?
Is there a way to insert into a tuple and return a list in one line? I am not trying to play code golf, I just don't think the two lines are logically different enough to merit separation.
insert,sort and reverse modify the list in-place and return None. And in your code you're actually storing that returned value in the newlist variable.
newlist = list(mytup).insert(0, 10)
And that newly created list(created on the fly) is garbage collected as there are no references to it any more.
In [151]: mytup = (0, 1, 2, 3, 4)
In [152]: lis=list(mytup) #create a new list object and add a reference to this new object
In [153]: newlist=lis.insert(0,10) #perform the insert operation on lis and store None
# in newlist
In [154]: print newlist
None
In [155]: print lis
[10, 0, 1, 2, 3, 4] #you can still access this list object with
#the help of `lis` variable.
The answer to your first question has already been given; you assign to the variable the result of the last function call, which is None. Here's the answer to your second question.
Rather than using insert, do something like this:
newlist = [10] + list(mytup)
It creates a new list containing the element to be inserted, appends it to the converted tuple and stores (a reference to) the resulting list.
This, of course, only works if you want to insert on either end.
If you need the new element to be inserted somewhere else, you have to slice the tuple, e.g. to insert after the third element in the tuple:
newlist = list(mytup[:3]) + [10] + list(mytup[3:])
Today I spent about 20 minutes trying to figure out why
this worked as expected:
users_stories_dict[a] = s + [b]
but this would have a None value:
users_stories_dict[a] = s.append(b)
Anyone know why the append function does not return the new list? I'm looking for some sort of sensible reason this decision was made; it looks like a Python novice gotcha to me right now.
append works by actually modifying a list, and so all the magic is in side-effects. Accordingly, the result returned by append is None. In other words, what one wants is:
s.append(b)
and then:
users_stories_dict[a] = s
But, you've already figured that much out. As to why it was done this way, while I don't really know, my guess is that it might have something to do with a 0 (or false) exit value indicating that an operation proceeded normally, and by returning None for functions whose role is to modify their arguments in-place you report that the modification succeeded.
But I agree that it would be nice if it returned the modified list back. At least, Python's behavior is consistent across all such functions.
The append() method returns a None, because it modifies the list it self by adding the object appended as an element, while the + operator concatenates the two lists and return the resulting list
eg:
a = [1,2,3,4,5]
b = [6,7,8,9,0]
print a+b # returns a list made by concatenating the lists a and b
>>> [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
print a.append(b) # Adds the list b as element at the end of the list a and returns None
>>> None
print a # the list a was modified during the last append call and has the list b as last element
>>> [1, 2, 3, 4, 5, [6, 7, 8, 9, 0]]
So as you can see the easiest way is just to add the two lists together as even if you append the list b to a using append() you will not get the result you want without additional work