weird behavior of list.append( ) function [duplicate] - python

I was going through the topic about list in Learning Python 5E book.
I notice that if we do concatenation on list, it creates new object. Extend method do not create new object i.e. In place change.
What actually happens in case of Concatenation?
For example
l = [1,2,3,4]
m = l
l = l + [5,6]
print l,m
#output
([1,2,3,4,5,6], [1,2,3,4])
And if I use Augmented assignment as follows,
l = [1,2,3,4]
m = l
l += [5,6]
print l,m
#output
([1,2,3,4,5,6], [1,2,3,4,5,6])
What is happening in background in case of both operations?

There are two methods being used there: __add__ and __iadd__. l + [5, 6] is a shortcut for l.__add__([5, 6]). l's __add__ method returns the result of addition to something else. Therefore, l = l + [5, 6] is reassigning l to the addition of l and [5, 6]. It doesn't affect m because you aren't changing the object, you are redefining the name. l += [5, 6] is a shortcut for l.__iadd__([5, 6]). In this case, __iadd__ changes the list. Since m refers to the same object, m is also affected.
Edit: If __iadd__ is not implemented, for example with immutable types like tuple and str, then Python uses __add__ instead. For example, x += y would be converted to x = x + y. Also, in x + y if x does not implement __add__, then y.__radd__(x), if available, will be used instead. Therefore x += y could actually be x = y.__radd__(x) in the background.

You can understand this better by inspecting the objects referenced in memory. Lets use id for the inspection
In the first case:
>>> l = [1,2,3,4]
>>> id(l)
4497052232
>>> m = l
>>> id(m)
4497052232
>>> l = l + [5,6]
>>> id(l)
4497052448
>>> print l, id(l), m, id(m)
[1, 2, 3, 4, 5, 6] 4497052448 [1, 2, 3, 4] 4497052232
>>>
Notice that l = l + [5,6] creates a new object in memory, and l references that.
In the second case:
>>> l = [1,2,3,4]
>>> id(l)
4497052520
>>> m = l
>>> id(m)
4497052520
>>> l += [5,6]
>>> id(l)
4497052520
>>> print l, id(l), m, id(m)
[1, 2, 3, 4, 5, 6] 4497052520 [1, 2, 3, 4, 5, 6] 4497052520
>>>
l += [5,6] references to the same object in memory. Hence the result.
So, basically += is the equivalent of inplace add.. More on this can be read here

Related

Lists and Function Arguments

>>> def duplicate(l):
... l = l + l
...
>>> l1 = [1, 2, 3]
>>> duplicate(l1)
>>> l1
[1, 2, 3]
I believe the function above duplicates the list. But why is the result not [1, 2, 3, 1, 2, 3]?
Concatenation of two list objects (as you do with l + l) always creates a new list object. In your function you then assign that new list object back to the local variable l, which is independent of the global reference l1. The original list object is not affected because only the contents of the list were copied.
If you wanted to alter the list object in place, you need to extend l with itself:
def duplicate(l):
l.extend(l)
list.extend() copies all elements from the list you pass in and adds them to the end of the list object you called it on. Passing in the list itself is safe; it'll only copy the original elements.
Demo:
>>> def duplicate(l):
... l.extend(l)
...
>>> l1 = [1, 2, 3]
>>> duplicate(l1)
>>> l1
[1, 2, 3, 1, 2, 3]

Function parameter is reference? [duplicate]

consider the following code:
>>> x = y = [1, 2, 3, 4]
>>> x += [4]
>>> x
[1, 2, 3, 4, 4]
>>> y
[1, 2, 3, 4, 4]
and then consider this:
>>> x = y = [1, 2, 3, 4]
>>> x = x + [4]
>>> x
[1, 2, 3, 4, 4]
>>> y
[1, 2, 3, 4]
Why is there a difference these two?
(And yes, I tried searching for this).
__iadd__ mutates the list, whereas __add__ returns a new list, as demonstrated.
An expression of x += y first tries to call __iadd__ and, failing that, calls __add__ followed an assignment (see Sven's comment for a minor correction). Since list has __iadd__ then it does this little bit of mutation magic.
The first mutates the list, and the second rebinds the name.
1)'+=' calls in-place add i.e iadd method. This method takes two parameters, but makes the change in-place, modifying the contents of the first parameter (i.e x is modified). Since both x and y point to same Pyobject they both are same.
2)Whereas x = x + [4] calls the add mehtod(x.add([4])) and instead of changing or adding values in-place it creates a new list to which a points to now and y still pointing to the old_list.

How to re-assign items in a list in Python?

I want to re-assign each item in a list in Python.
In [20]: l = [1,2,3,4,5]
In [21]: for i in l:
....: i = i + 1
....:
....:
But the list didn't change at all.
In [22]: l
Out[22]: [1, 2, 3, 4, 5]
I want to know why this happened. Could any body explain the list iterating in detail? Thanks.
You can't do it like that, you are merely changing the value binded to the name i. On each iteration of the for loop, i is binded to a value in the list. It is not a pointer in the sense that by changing the value of i you are changing a value in the list. Instead, as I said before, it is simply a name and you are just changing the value that name refers to. In this case, i = i + 1, binds i to the value i + 1. So you aren't actually affecting the list itself, to do that you have to set it by index.
>>> L = [1,2,3,4,5]
>>> for i in range(len(L)):
L[i] = L[i] + 1
>>> L
[2, 3, 4, 5, 6]
Some pythonistas may prefer to iterate like this:
for i, n in enumerate(L): # where i is the index, n is each number
L[i] = n + 1
However you can easily achieve the same result with a list comprehension:
>>> L = [1,2,3,4,5]
>>> L = [n + 1 for n in L]
>>> L
[2, 3, 4, 5, 6]
For more info: http://www.effbot.org/zone/python-objects.htm
This is because of how Python handles variables and the values they reference.
You should modify the list element itself:
for i in xrange(len(l)):
l[i] += 1
>>> a = [1, 2, 3, 4, 5]
>>> a = [i + 1 for i in a]
>>> a
[2, 3, 4, 5, 6]
Initially i is a pointer to the item inside the list, but when you reassign it, it will point to the new number, that is why the list will not be changed.
For a list of mutable objects it would work:
class Number(object):
def __init__(self,n):
self.n=n
def increment(self):
self.n+=1
def __repr__(self):
return 'Number(%d)' % self.n
a = [Number(i) for i in xrange(5)]
print a
for i in a:
i.increment()
print a
But int are not mutable, when you do an operation on them you get a new int object, and that is why it doesn't work in your case.

What's the difference between a[] and a[:] when assigning values?

I happen to see this snippet of code:
a = []
a = [a, a, None]
# makes a = [ [], [], None] when print
a = []
a[:] = [a, a, None]
# makes a = [ [...], [...], None] when print
It seems the a[:] assignment assigns a pointer but I can't find documents about that. So anyone could give me an explicit explanation?
In Python, a is a name - it points to an object, in this case, a list.
In your first example, a initially points to the empty list, then to a new list.
In your second example, a points to an empty list, then it is updated to contain the values from the new list. This does not change the list a references.
The difference in the end result is that, as the right hand side of an operation is evaluated first, in both cases, a points to the original list. This means that in the first case, it points to the list that used to be a, while in the second case, it points to itself, making a recursive structure.
If you are having trouble understanding this, I recommend taking a look at it visualized.
The first will point a to a new object, the second will mutate a, so the list referenced by a is still the same.
For example:
a = [1, 2, 3]
b = a
print b # [1, 2, 3]
a[:] = [3, 2, 1]
print b # [3, 2, 1]
a = [1, 2, 3]
#b still references to the old list
print b # [3, 2, 1]
More clear example from #pythonm response
>>> a=[1,2,3,4]
>>> b=a
>>> c=a[:]
>>> a.pop()
4
>>> a
[1, 2, 3]
>>> b
[1, 2, 3]
>>> c
[1, 2, 3, 4]
>>>

For-loop over several variables from a single list

I'm trying to create a loop like this
newList = [a + b for a,b in list[::2], list[1::2]]
meaning, take two consecutive entries from a list, do something with them and put the into a new list.
How would that work?
You want to zip your two newly created lists:
newList = [a + b for a,b in zip(list[::2], list[1::2])]
You also do this in a somewhat more memory-efficient manner by using an iterator:
it = iter(list)
newList = [a + b for a, b in zip(it, it)]
or even more efficiently* by using the izip function, which returns an iterator:
import itertools
it = iter(list)
newList = [a + b for a, b in itertools.izip(it, it)]
* at least under Python 2.x; in Python 3, as I understand it, zip itself returns an iterator.
Note that you really should never call a variable list as this clobbers the builtin list constructor. This can cause confusing errors and is generally considered bad form.
>>> L=range(6)
>>> from operator import add
>>> map(add, L[::2], L[1::2])
[1, 5, 9]
alternatively you could use an iterator here
>>> L_iter = iter(L)
>>> map(add, L_iter, L_iter)
[1, 5, 9]
since you pass the same iterator twice, map() will consume two elements for each iteration
Another way to pass the iterator twice is to build a list with a shared reference. That avoids the temporary variable
>>> map(add, *[iter(L)]*2)
[1, 5, 9]
of course you can replace add with your own function
>>> def myfunc(a,b):
... print "myfunc called with", a, b
... return a+b
...
>>> map(myfunc, *[iter(L)]*2)
myfunc called with 0 1
myfunc called with 2 3
myfunc called with 4 5
[1, 5, 9]
And it's easy to expand to 3 variables or more
>>> def myfunc(*args):
... print "myfunc called with", args
... return sum(args)
...
>>> map(myfunc, *[iter(L)]*3)
myfunc called with (0, 1, 2)
myfunc called with (3, 4, 5)
[3, 12]
Zip and Map come in handy here.
>>> a = [1, 2, 3]
>>> b = [4, 5, 6]
>>> list(zip(a, b))
[(1, 4), (2, 5), (3, 6)]
>>> list(map <operation>, (zip(a, b)))
>>> ...
Or in your case,
>>> list(map(lambda n: n[0] + n[1], (zip(a, b))))
[5, 7, 9]
There's definitely a better way to pass the plus operation to the map. Feel free to add to it, anyone!

Categories