Modifying Python lists via slices and for-loops? - python

I was trying to modify the values in lists via slices and for-loops, and ran into some pretty interesting behavior. I would appreciate if someone could explain what's happening internally here.
>>> x = [1,2,3,4,5]
>>> x[:2] = [6,7] #slices can be modified
>>> x
[6, 7, 3, 4, 5]
>>> x[:2][0] = 8 #indices of slices cannot be modified
>>> x
[6, 7, 3, 4, 5]
>>> x[:2][:1] = [8] #slices of slices cannot be modified
>>> x
[6, 7, 3, 4, 5]
>>> for z in x: #this version of a for-loop cannot modify lists
... z += 1
...
>>> x
[6, 7, 3, 4, 5]
>>> for i in range(len(x)): #this version of a for-loop can modify lists
... x[i] += 1
...
>>> x
[7, 8, 4, 5, 6]
>>> y = x[:2] #if I assign a slice to a var, it can be modified...
>>> y[0] = 1
>>> y
[1, 8]
>>> x #...but it has no impact on the original list
[7, 8, 4, 5, 6]

Let's break down your comments 1 by 1:
1.) x[:2] = [6, 7] slices can be modified:
See these answers here. It's calling the __setitem__ method from the list object and assigning the slice to it. Each time you reference x[:2] a new slice object is created (you can simple do id(x[:2]) and it's apparent, not once will it be the same id).
2.) indices of slices cannot be modified:
That's not true. It couldn't be modified because you're performing the assignment on the slice instance, not the list, so it doesn't trigger the __setitem__ to be performed on the list. Also, int are immutable so it cannot be changed either way.
3.) slices of slices cannot be modified:
See above. Same reason - you are assigning to an instance of the slice and not modifying the list directly.
4.) this version of a for-loop cannot modify lists:
z being referenced here is the actual objects in the elements of x. If you ran the for loop with id(z) you'll note that they're identical to id(6), id(7), id(3), id(4), id(5). Even though list contains all 5 identical references, when you do z = ... you are only assigning the new value to the object z, not the object that is stored in list. If you want to modify the list, you'll need to assign it by index, for the same reason you can't expect 1 = 6 will turn x into [6, 2, 3, 4, 5].
5.) this version of a for-loop can modify lists:
See my answer above. Now you are directly performing item assignment on the list instead of its representation.
6.) if I assign a slice to a var, it can be modified:
If you've been following so far, you'll realize now you are assigning the instance of x[:2] to the object y, which is now a list. The story follows - you perform an item assignment by index on y, of course it will be updated.
7.) ...but it has no impact on the original list:
Of course. x and y are two different objects. id(x) != id(y), therefore any operation performed on x will not affect y whatsoever. if you however assigned y = x and then made a change to y, then yes, x will be affected as well.
To expand a bit on for z in x:, say you have a class foo() and assigned two instances of such to the list f:
f1 = foo()
f2 = foo()
f = [f1, f2]
f
# [<__main__.foo at 0x15f4b898>, <__main__.foo at 0x15f4d3c8>]
Note that the reference in question is the actual foo instance, not the object f1 and f2. So even if I did the following:
f1 = 'hello'
f
# [<__main__.foo at 0x15f4b898>, <__main__.foo at 0x15f4d3c8>]
f still remains unchanged since the foo instances remains the same even though object f1 now is assigned to a different value. For the same reason, whenever you make changes to z in for z in x:, you are only affecting the object z, but nothing in the list is changed until you update x by index.
If however the object have attribute or is mutable, you can directly update the referenced object in the loop:
x = ['foo']
y = ['foo']
lst = [x,y]
lst
# [['foo'], ['foo']]
for z in lst:
z.append('bar')
lst
# [['foo', 'bar'], ['foo', 'bar']]
x.append('something')
lst
# [['foo', 'bar', 'something'], ['foo', 'bar']]
That is because you are directly updating the object in reference instead of assigning to object z. If you however assigned x or y to a new object, lst will not be affected.

There is nothing odd happening here. Any slice that you obtain from a list is a new object containing copies of your original list. The same is true for tuples.
When you iterate through your list, you get the object which the iteration yields. Since ints are immutable in Python you can't change the state of int objects. Each time you add two ints a new int object is created. So your "version of a for-loop [which] cannot modify lists" is not really trying to modify anything because it will not assign the result of the addition back to the list.
Maybe you can guess now why your second approach is different. It uses a special slicing syntax which is not really creating a slice of your list and allows you to assign to the list (documentation). The newly created object created by the addition operation is stored in the list through this method.
For understanding your last (and your first) examples, it is important to know that slicing creates (at least for lists and tuples, technically you could override this in your own classes) a partial copy of your list. Any change to this new object will, as you already found out, not change anything in your original list.

Related

Why complain that 'tuple' object does not support item assignment when extending a list in a tuple? [duplicate]

This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.

How is this list expanded with the slicing assignment? [duplicate]

This question already has answers here:
How does assignment work with list slices?
(5 answers)
Closed 5 years ago.
I came across the following code (sort of):
my_list = [1, [2, 3, 4], 5]
my_list[1:2] = my_list[1]
After running these two lines, the variable my_list will be [1, 2, 3, 4, 5]. Pretty useful for expanding nested lists.
But why does it actually do what it does?
I would have assumed that the statement my_list[1:2] = my_list[1] would do one of the following:
simply put [2, 3, 4] into the second position in the list (where it already is)
give some kind of "too many values to unpack" error, from trying to put three values (namely 2,3,4) into a container of only length 1 (namely my_list[1:2]). (Repeating the above with a Numpy array instead of a list results in a similar error.)
Other questions (e.g. How assignment works with python list slice) tend to not pay much attention to the discrepancy between the size of the slice to be replaced, and the size of the items you're replacing it with. (Let alone explaining why it works the way it does.)
Slice assignment replaces the specified part of the list with the iterable on the right-hand side, which may have a different length than the slice. Taking the question at face value, the reason why this is so is because it's convenient.
You are not really assigning to the slice, i.e. Python doesn't produce a slice object that contains the specified values from the list and then changes these values. One reason that wouldn't work is that slicing returns a new list, so this operation wouldn't change the original list.
Also see this question, which emphasizes that slicing and slice assignment are totally different.
Here is the relevant bit from the Python Language Reference
If the target is a slicing: The primary expression in the reference is
evaluated. It should yield a mutable sequence object (such as a list).
The assigned object should be a sequence object of the same type.
Next, the lower and upper bound expressions are evaluated, insofar
they are present; defaults are zero and the sequence’s length. The
bounds should evaluate to integers. If either bound is negative, the
sequence’s length is added to it. The resulting bounds are clipped to
lie between zero and the sequence’s length, inclusive. Finally, the
sequence object is asked to replace the slice with the items of the
assigned sequence. The length of the slice may be different from the
length of the assigned sequence, thus changing the length of the
target sequence, if the target sequence allows it.
This behavior makes sense qualitatively because when you slice a list you get a sub list so replacing that with another list shouldn't add a level of nesting. Allowing it to change the length of the list is a design choice. Other choices are possible as your numpy example demonstrates.
Short Answer:
my_list[1:2] = my_list[1] will replaced the content from 1st index to 2nd index of my_list with the content of present in 1st index of
my_list
Explanation:
Let's see two slicing operations, very similar but totally distinct
This creates the copy of list and stores it the variable
some_variable = some_list[2:5]
This replaces the content of the list inplace, which permits changing the length of the list too.
some_list[2:5] = [1, 2, 3, 4]
When you use assignment operator =, it invokes a __setitem__ function. Our focus here is the case 2 above. As per the Python's Assignment Statement document:
If the target is a slicing: The primary expression in the reference is
evaluated. It should yield a mutable sequence object (such as a list).
The assigned object should be a sequence object of the same type.
Next, the lower and upper bound expressions are evaluated, insofar
they are present; defaults are zero and the sequence’s length. The
bounds should evaluate to integers. If either bound is negative, the
sequence’s length is added to it. The resulting bounds are clipped to
lie between zero and the sequence’s length, inclusive. Finally, the
sequence object is asked to replace the slice with the items of the
assigned sequence. The length of the slice may be different from the
length of the assigned sequence, thus changing the length of the
target sequence, if the target sequence allows it.
In our case my_list[1:2] = my_list[1], python will also call __setitem__ as:
my_list.__setitem__(slice(1,2,None), [2, 3, 4])
Refer slice document to know what it does.
So, when you did my_list[1:2] = my_list[1], you replaced the content from 1st index to 2nd index of my_list with the content of present in 1st index of my_list i.e. [2, 3, 4].
I think now we can answer why your assumptions are incorrect:
put [2, 3, 4] into the second position in the list (where it already is)
No. Because __setitem__ is not called on the index but on the slice of the indices which you passed.
give some kind of "too many values to unpack" error, from trying to put three values (namely 2,3,4) into a container of only length 1 (namely my_list[1:2]).
Again No. Because the range of your indices creating your container is replaced with new set of values.
What you are doing is slice assignment.
Assignment to slices is also possible, and this can even change the size of the list or clear it entirely
my_list[1:2] = my_list[1]
This replaces the slice of my_list with the contents of my_list[1].
By specifying my_list[1:2] on the left side of the assignment operator =, you are telling Python you want to use slice assignment.
my_list[1:2] = my_list[1] is equivalent to my_list.__setitem__(slice(1, 2, None), my_list[1])
In slice(1, 2, None), 1 is start, 2 is stop, and None is step and is optional.
What you are trying here is called Slice Assingnment. In python it is possible to assign an iterable(my_list[1] in your case) to a slice of another iterable(my_list[0:1] in your case). Lets walk through some examples to understand what it really means:
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:3] = b
>>> l
>>> [6, 7, 8, 4, 5]
So what happened here is the portion of list l for 0,1,2 indices is
which covers elements 1,2,3is replaced by elements of list b 6,7,8. However in this case size of slice and replaced elements happens to be equal by chance.
So what happens when slice size and iterable to be replaced are not equal
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:4] = b
>>> l
>>> [6,7,8,5]
Notice that this operation didn't produce any error, instead, it just copied whatever elements are available with the entire sliced portion. In this case, sliced elements are 1,2,3,4 replaced by 6,7,8
In the previous example iterable to be replaced was smaller. What happens if slice portion is smaller
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:1] = b
>>> l
>>> [6,7,8,2,3,4,5]
So now we can see that only first element is replaced by entire iterable b.
You can also use this behaviour to remove a specific portion of the list ( Which I find convenient in some situations ).
>>> l = [1,2,3,4,5]
>>> l[0:2] = []
>>> l
>>> [3,4,5]
First two elements are removed very conveniently here.
So the example in your question is similar to the examples I posted above, except that in your case there is an additional step of unpacking list values. Unpacking list value happens every time when you assign list to another list. A short example
>>> l = [[1]]
>>> a = []
>>> a = l[0]
>>> a
>>> [1]
Your example now:
#Replace the slice [0:1] with my_list[1] perform unpacking list values as well
>>> my_list[1:2] = my_list[1]
>>> [1,2,3,4,5]
Also note that slice assignment is only possible if you assign iterable to a slice. If you try to assign an int or something that is not an iterable to a slice python will throw an error.
>>> l = [1,2,3,4,5]
>>> b = [6,7,8]
>>> l[0:1] = b[1]
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only assign an iterable
That's why in your case my_list[1] don't raise an error since it is an iterable.
That what you are doing is insert an element through slicing. I will explain everything by parts. More than an inserting it could be interpreted as adding an item to your list after slicing target list in a range desired. Now to explain every line in detail:
my_list[1:2]
That part is like saying to Python; "I want to get the values (slice) from index 1 to the index before 2 (excluding 2, I will explain with another example later on)". After that you assign a value to those values:
my_list[1:2] = my_list[1] #The same as my_list[1:2] = [2,3,4]
Now that you know what the first part does, next it is going to add the item at the right side of the '=' operator so you could interpret it like this; "I want to slice from index 1 to everything before 2 (again, excluding index 2) and then add my list [2,3,4]". Now here comes another examples so you understand even better I hope.
problemList = [1, [2, 3, 4], 5]
problemList[1:2] = problemList[1] #The same as problemList[1:2] = [2,3,4]
analogProblemL = [1] + [2,3,4] + [5] #Output : [1,2,3,4,5]
insertList = [12,13,14]
myList = [1, [2, 3, 4], 5,6,7,8,9]
myList[3:6] = insertList
analogFstList = [1,[2,3,4] ,5] + insertList + [9] #Output : [1,[2,3,4],5,12,13,14,9]
myScnList = [1, [2, 3, 4], 5]
myScnList[1:3] = [2,3,4]
analogScnList = [1] + [2,3,4] + [5] #Output : [1,2,3,4,5]
The next lines will be like if it was an animation frames so it's easier to interpret:
[1,2,3,4,5] #List before line of code: myList[1:3] = [12,13,14]
[1,|2,3|,4,5] #The moment where you slice the list with: myList[1:3]. Please notice that the characters used '|' are for representing the slice.
[1] + [12,13,14] + [4,5] #After assigning what the slice should be changed for. It's like excracting from the whole list the values [2,3] and changing it for [12,13,14].
[1,12,13,14,4,5] #Final list after running the code.
Some references used for this answer:
http://effbot.org/zone/python-list.htm
Understanding Python's slice notation
How assignment works with python list slice
Hope it was useful for you.

Python: tuple with mutable items [duplicate]

This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.

Python Variable assignment in a for loop

I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]

Modifying a list iterator in Python not allowed?

Simple example:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print myList
prints
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]
while:
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
prints
[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]
Conclusion:
Lists can be modified in place using global list access Lists can
List items can NOT be modified in place using the iterator object
All example code I can find uses the global list accessors to modify the list inplace.
Is it so evil to modify a list iterator?
The reason obj += 1 does not do what you expect is that this statement does not modify obj in-place. Instead, it computes the new value, and rebinds the variable obj to point to the new value. This means that the contents of the list remain unchanged.
In general it is possible to modify the list while iterating over it using for obj in myList. For example:
myList = [[1], [2], [3], [4], [5]]
for obj in myList:
obj[0] += 1
print(myList)
This prints out:
[[2], [3], [4], [5], [6]]
The difference between this and your first example is that here, the list contains mutable objects, and the code modifies those objects in-place.
Note that one could also write the loop using a list comprehension:
myList = [val+1 for val in myList]
I think you've misunderstood what an "iterator object" is. A for loop is not an iterator object. For all intents and purposes, a for loop like this:
myList = [0, 1, 2, 3, 4]
for x in myList:
print x
does this (but more efficiently and less verbosely):
i = 0
while i < len(myList)
x = myList[i]
print x
i += 1
So you see, any changes made to x are lost as soon as the next loop starts, because the value of x is overwritten by the value of the next item in the list.
As others have observed, it is possible to alter the value of a list while iterating over it. (But don't change its length! That's where you get into trouble.) One elegant way to do so is as follows:
for i, x in enumerate(myList):
myList[i] = some_func(x)
Update: It's also important to understand that no copying goes on in a for loop. In the above example, i and x -- like all variables in Python -- are more like pointers in C/C++. As the for loop progresses, obj points at myList[0], myList[1], etc, in turn. And like a C/C++ pointer, the properties of the object pointed to are not changed when the pointer is changed. But also like a C pointer, you can directly modify the thing pointed at, because it's not a copy. In C, this is done by dereferencing the pointer; in Python, this is done by using a mutable object. That's why NPE's answer works. If i and x were even shallow copies, it wouldn't be possible to do what he does.
The reason you can't directly change ints the way you can change lists (as in NPE's answer), is that ints aren't mutable. Once a 5 object is created, nothing can change its value. That's why passing around a pointer to 5 is safe in Python -- no side-effects can occur, because the thing pointed to is immutable.
in for obj in myList:, in every iteration, obj is a (shallow) copy of the element in myList. So the change on the obj does nothing to myList's elements.
It's different with the Perl for my $obj (#myList) {}
You are confused. Consider your first snippet:
myList = [1, 2, 3, 4, 5]
for obj in myList:
obj += 1
print a
obj is not some kind of magical pointer into the list. It is a variable which holds a reference to an object which happens to also be in myList. obj += 1 has the effect of increasing the value stored in obj. Your code then does nothing with that value.
To be clear: There are no copies in this code example. obj is a variable, which holds an object in the list. That is all.
In the first example the integer is copied into obj which is increased by 1.
The list is not changed.
If you would use a class instance and perform operations on it, it would be changed.
Modification in list is allowed. Your code examples arbove are pretty garbled...
myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
myList[index] += 1
print myList
This works.

Categories