Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
When I printed a list, I got a syntax error when using this method:
print i for i in [1,2,3]
I knew this is ok when using this method:
for i in [1, 2, 3]:
print i
and I knew that
(i for i in [1, 2, 3])
is a generator object, but I just don't get it that why
print i for i in [1, 2, 3]
does't work. Can anyone give me a clue?
The list comprehension syntax x for x in ... requires brackets around it. That's a syntactic rule of Python, just like requiring indentation, or requiring a colon after if or whatever. i for i in [1, 2, 3] by itself is not valid syntax. You need either [i for i in [1, 2, 3]] (for a list comprehension) or (i for i in [1, 2, 3]) (for a generator comprehension).
In Python 2, print is a statement, not an expression or a function, so you can't use it directly in a comprehension. Use this trick:
def f(x): print x
[f(i) for i in [1,2,3]]
Note that (f(i)...) doesn't work because this just creates a generator which would call f() if you iterated over it. The list comprehension [] actually invokes f().
[EDIT] If you use Python > 2.6, you can achieve the same using
from __future__ import print_function
[print(i) for i in [1, 2, 3]]
Note the () around the argument to print.
The list comprehension syntax ([expression for loop]) is a shorthand loop syntax for producing a list.
You are not producing a list, you want to print items in a loop. Since you are not producing a python list, you have to use a regular loop.
Alternatively, since all you are doing is printing the items on separate lines, just add the newlines yourself:
print '\n'.join(i for i in [1, 2, 3])
This produces the same output as:
for i in [1, 2, 3]:
print i
If you use Python 3, or use from __future__ import print at the top of your module and so use the print() function, you can send all values to the function in one call, and tell print() to use newlines in between:
values = [1, 2, 3]
print(*values, sep="\n")
As an expression (in the grammar):
[i for i in [1, 2, 3]] is a list comprehension.
(i for i in [1, 2, 3]) is a generator expression.
But i for i in [1, 2, 3] by itself is a syntax error, and that's just the way it is. There must be something surrounding it. Unless you have ( or [ around it, it's not a valid expression, because the for keyword is not valid at that point.
Inside the print statement, it wants an expression.
(As a red herring, func(i for i in [1, 2, 3]) is permitted as an expression, being a function call with the first argument being a generator expression.)
Print is not a function, it's a statement, and you can't have them in expressions. Just use a regular loop as you don't want to produce a list, that a list comprehension does. In theory you can do (not that you should. at all):
from __future__ import print_function
[print(my_item) for my_item in [1,2,3,4]]
1
2
3
4
Out[26]:
[None, None, None, None]
This is invalid python syntax. The i for i in [1, 2, 3] is only valid in a list or generator comprehension, ie. surrounded by [] or () respectively.
You'll want to use:
print '\n'.join(str(i) for i in [1, 2, 3])
Normally, list comprehensions are used to derive a new list from an existing list. Eg:
>>> a = [1, 2, 3, 4, 5]
>>> [i for i in a if i > 2]
[3, 4, 5]
Should we use them to perform other procedures? Eg:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> [b.append(i) for i in a]
[None, None, None, None, None]
>>> print b
[1, 2, 3, 4, 5]
or should I avoid the above and use the following instead?:
for i in a:
b.append(i)
You should indeed avoid using list comprehensions (along with dictionary comprehensions, set comprehensions and generator expressions) for side effects. Apart from the fact that they'd accumulate a bogus list and thus waste memory, it's also confusing. I expect a list comprehension to generate a (meaningful) value, and many would agree. Loops, on the other hand, are clearly a sequence of statements. They are expected to kick off side effects and generate no result value - no surprise.
From python documentation:
List comprehensions provide a concise way to create lists. Common
applications are to make new lists
Perhaps you want to learn more about reduce(), filter() and map() functions.
In the example you give it would make the most sense to do:
b = [i for i in a]
if for some reason you wanted to create b. In general, there is some common sense that must be employed. If using a comprehension makes your code unreadable, don't use it. Otherwise go for it.
Only use list comprehensions if you plan to use the created list. Otherwise you create it just for the GC to throw it again without ever being used.
So instead of [b.append(i) for i in a] you should use a proper for loop:
for i in a:
b.append(i)
Another solution would be through a generator expression:
b += (i for i in a)
However, if you want to append the whole list, you can simply do
b += a
And if you just need to apply a function to the elements before adding them to the list, you can always use map:
b += map(somefunc, a)
b = []
a = [1, 2, 3, 4, 5]
b.extend (a)
What does generator comprehension do? How does it work? I couldn't find a tutorial about it.
Do you understand list comprehensions? If so, a generator expression is like a list comprehension, but instead of finding all the items you're interested and packing them into list, it waits, and yields each item out of the expression, one by one.
>>> my_list = [1, 3, 5, 9, 2, 6]
>>> filtered_list = [item for item in my_list if item > 3]
>>> print(filtered_list)
[5, 9, 6]
>>> len(filtered_list)
3
>>> # compare to generator expression
...
>>> filtered_gen = (item for item in my_list if item > 3)
>>> print(filtered_gen) # notice it's a generator object
<generator object <genexpr> at 0x7f2ad75f89e0>
>>> len(filtered_gen) # So technically, it has no length
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'generator' has no len()
>>> # We extract each item out individually. We'll do it manually first.
...
>>> next(filtered_gen)
5
>>> next(filtered_gen)
9
>>> next(filtered_gen)
6
>>> next(filtered_gen) # Should be all out of items and give an error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> # Yup, the generator is spent. No values for you!
...
>>> # Let's prove it gives the same results as our list comprehension
...
>>> filtered_gen = (item for item in my_list if item > 3)
>>> gen_to_list = list(filtered_gen)
>>> print(gen_to_list)
[5, 9, 6]
>>> filtered_list == gen_to_list
True
>>>
Because a generator expression only has to yield one item at a time, it can lead to big savings in memory usage. Generator expressions make the most sense in scenarios where you need to take one item at a time, do a lot of calculations based on that item, and then move on to the next item. If you need more than one value, you can also use a generator expression and grab a few at a time. If you need all the values before your program proceeds, use a list comprehension instead.
A generator comprehension is the lazy version of a list comprehension.
It is just like a list comprehension except that it returns an iterator instead of the list ie an object with a next() method that will yield the next element.
If you are not familiar with list comprehensions see here and for generators see here.
List/generator comprehension is a construct which you can use to create a new list/generator from an existing one.
Let's say you want to generate the list of squares of each number from 1 to 10. You can do this in Python:
>>> [x**2 for x in range(1,11)]
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
here, range(1,11) generates the list [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], but the range function is not a generator before Python 3.0, and therefore the construct I've used is a list comprehension.
If I wanted to create a generator that does the same thing, I could do it like this:
>>> (x**2 for x in xrange(1,11))
<generator object at 0x7f0a79273488>
In Python 3, however, range is a generator, so the outcome depends only on the syntax you use (square brackets or round brackets).
Generator comprehension is an easy way of creating generators with a certain structure. Lets say you want a generator that outputs one by one all the even numbers in your_list. If you create it by using the function style it would be like this:
def allEvens( L ):
for number in L:
if number % 2 is 0:
yield number
evens = allEvens( yourList )
You could achieve the same result with this generator comprehension expression:
evens = ( number for number in your_list if number % 2 == 0 )
In both cases, when you call next(evens) you get the next even number in your_list.
Generator comprehension is an approach to create iterables, something like a cursor which moves on a resource. If you know mysql cursor or mongodb cursor, you may be aware of that the whole actual data never gets loaded into the memory at once, but one at a time. Your cursor moves back and forth, but there is always a one row/list element in memory.
In short, by using generators comprehension you can easily create cursors in python.
Another example of Generator comprehension:
print 'Generator comprehensions'
def sq_num(n):
for num in (x**2 for x in range(n)):
yield num
for x in sq_num(10):
print x
Generators are same as lists only, the minor difference is that in lists we get all the required numbers or items of the list at ones, but in generators the required numbers are yielded one at a time. So for getting the required items we have to use the for loop to get all the required items.
#to get all the even numbers in given range
def allevens(n):
for x in range(2,n):
if x%2==0:
yield x
for x in allevens(10)
print(x)
#output
2
4
6
8
We can understand this as a generator version of list comprehension. In the case of list comprehension, we create a using one-liners or a short code and for generator comprehensions, we do one-liner code or a small code for generators.
These have the same syntax just replace the [] (square brackets) with the () curly brackets.
generator_composition_object = (num**3 for num in range(5))
print(generator_composition_object)
this will give an address of the object of the type generator. We can also use functionalities like next() in these.