This question already has answers here:
How to remove items from a list while iterating?
(25 answers)
Closed 5 years ago.
I want to remove specific items from python list by iterating and checking if it meet some requirements. At first, I just operate on a list of customized class objects, but it actually meet some errors, and I experiment on a python list of primitive type int, just to find strange result!
Here is some code excerpts:
>>> a=[1,2,3,4,5]
>>> for i in a:
... a.remove(i)
...
>>> a
[2, 4]
I expect the a should be [] after the loop, but it proves to be [2,4], I wonder why actually. I found a related question in Remove items from a list while iterating, but it only gives a solution on how to remove specific items, not concerning the mechanism actually. I really want to know the reason of this strange result.
lets try to print the values of i and a while iterating.
for i in a:
print i, a
a.remove(i)
the output will be:
1 [1, 2, 3, 4, 5]
3 [2, 3, 4, 5]
5 [2, 4, 5]
so when you remove an element, the indices will change, so while value at index 1 was 2 earlier, now it is 3. This will be the value of i.
So you've exposed a little of the python implementation. Basically, an array powers the python list, and it is simply incrementing the array index by 1. So it'll go to a[0], a[1], a[2]... and check before each iteration that it's not gonna run off the end of the array. As you remove the first item '1' from the list, '2' moves to a[0]. The array now looks like [2,3,4,5]. The iterator is now pointing to a[1], so now '3' gets removed. Finally, skipping over '4', '5' gets removed.
a = [1,2,3,4,5]
for i in a:
print("a:%s i=%s"%(a,i))
a.remove(i)
print("final a: %s"%a)
Gives the output
a:[1, 2, 3, 4, 5] i=1
a:[2, 3, 4, 5] i=3
a:[2, 4, 5] i=5
final a: [2, 4]
Here's the real nuts and bolts if you're interested.
https://github.com/python/cpython/blob/master/Objects/listobject.c#L2832
The reason your solution doesn't work as expected is because the iterator doesn't behave the way you'd expect if the list is modified. If you're example was rewritten this way, you'd get the result you expect.
>>> a=[1,2,3,4,5]
>>> b = a[:]
>>> for i in b:
... a.remove(i)
...
>>> a
[]
This is because 'b' is a copy of 'a', so be doesn't get modified when a does. This means the iterator doesn't have the data structure modified underneath it.
A more efficient solution is:
a = [1,2,3,4,5]
a = [i for i in a if not condition(i)]
This list comprehension copies as it goes through the source list, and only bothers to copy the elements that aren't being removed.
Related
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 2 years ago.
lst = [1,2,3,4,5]
def rem_items(lst):
for i in lst:
print(lst)
time.sleep(1)
lst.remove(i)
rem_items(lst)
returns
[1, 2, 3, 4, 5]
[2, 3, 4, 5]
[2, 4, 5]
I can understand the first two lines, but why does it remove 3 instead of 2?
And why is it stopping after removing 3, not 2,4,5?
Thanks in advance
You are iterating over the list [1,2,3,4,5] and modifying its length.
Writing
for i in lst:
print(lst)
time.sleep(1)
lst.remove(i)
is the same as writing
lst_iter = iter(lst):
for i in lst_iter:
print(lst)
time.sleep(1)
lst.remove(i)
Internally iterator holds a pointer aka index aka position of current item in the list. The loop directive calls i=next(lst_iter) internally, until StopIteration exception is raised.
In your case calling next(lst_iter) for the 1st time returns first element of list [1,2,3,4,5], which is 1. Calling next(lst_iter) for the 2nd time returns second element of list [2,3,4,5], which is 3.
You are modifying the list while iterating over it. This is generally a Bad Thing to do and can lead to unpredictable results.
See here for more info:
Modifying list while iterating
This question already has answers here:
Python for loop skipping every other value
(2 answers)
Floats not evaluating as negative (Python)
(2 answers)
Closed 7 years ago.
Could someone please explain why the following is happening please?
>>> ls = ['a', 'b']
>>> for i in ls:
... ls.remove(i)
...
>>> ls
['b']
>>> ls = [1, 2, 3, 4, 5]
>>> for i in ls:
... ls.remove(i)
...
>>> ls
[2, 4]
I would expect the for loop to run through every element in the list, and remove it, leaving me with an empty list. Why does the first list end up with one element left, and the second example seems to have lost all the odd numbers? How should I do this instead? Thanks! :)
Let's show the second example, it's easier to see what's going on.
Take the first element: 1
Remove it; the list is now [2, 3, 4, 5]
Take the second element: 3
Remove it; the list is now [2, 4, 5]
Take the third element: 5
Remove it; the list is now [2, 4]
Next element would be the fourth element, but the list only has two; terminate.
The point to note is that in each iteration, you're not getting the n-th element of the list as it was in past, before the loop started; you're getting the n-th element of the array as it is in that moment.
The first example is the same, just stops sooner.
The Pythonic way to clear a list:
del ls[:]
EDIT: To conditionally delete elements, the most Pythonic way is to make a new filtered list:
filtered_ls = [x for x in ls if okay(x)]
If you want to not lose reference to the old list, you can do this:
filtered_ls[:] = [x for x in ls if okay(x)]
And finally, you can also iterate from the back; this way, since you're always working on the last element, removing it doesn't screw up the list.
If you want to loop through a list item-by-item and remove them, use a while loop:
>>> ls=['a', 'b', 'c']
>>> while ls:
... e=ls.pop()
... print e
...
c
b
a
>>> ls
[]
If you want that to be performant, use deque for faster pop's
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I'm learning Python, coming from a C#/Java background, and was playing around with list behavior. I've read some of the documentation, but I don't understand how, or why, slices with indices larger than length-1 make is possible to append items.
ls = ["a", "b", "c", "d"]
n = len(ls) # n = 4
letter = ls[4] # index out of range (list indexes are 0 - 3)
# the following all do the same thing
ls += ["echo"]
ls.append("foxtrot")
ls[len(ls):] = ["golf"] # equivalent to ls[len(ls): len(ls)]
print(ls)
Although it seems odd to me, I understand how slices can modify the list they operate on. What I don't get is why list[len(list)] results in the expected out of bounds error, but list[len(list):] doesn't. I understand that slices are fundamentally different than indexes, just not what happens internally when a slice begins with an index outside of the list values.
Why can I return a slice from a list that starts with a non-existent element (len(list))? And why does this allow me to expand the list?
Also, of the first three above methods for appending an item, which one is preferred from a convention or performance perspective? Are there performance disadvantages to any of them?
While a.append(x) and a[len(a):] = [x] do the same thing, the first is clearer and should almost always be used. Slice replacement may have a place, but you shouldn't look for reasons to do it. If the situation warrants it, it should be obvious when to do it.
Start with the fact that indexing returns an element of the list, and slicing returns a "sublist" of a list. While an element at a particular either exists or does not exist, it is mathematically convenient to think of every list have the empty list as a "sublist" between any two adjacent positions. Consider
>>> a = [1, 2, 3]
>>> a[2:2]
[]
Now consider the following sequence. At each step, we increment the starting index of the slice
>>> a[0:] # A copy of the list
[1, 2, 3]
>>> a[1:]
[2, 3]
>>> a[2:]
[3]
Each time we do so, we remove one more element from the beginning of the list for the resulting slice. So what should happen when the starting index finally equals the final index (which is implicitly the length of the list)?
>>> a[3:]
[]
You get the empty list, which is consistent with the following identities
a + [] = a
a[:k] + a[k:] = a
(The first is a special case of the second where k >= len(a).
Now, once we see why it makes sense for the slice to exist, what does it mean to replace that slice with another list? In other words, what list do we get if we replace the empty list at the end of [1, 2, 3] with the list [4]? You get [1, 2, 3, 4], which is consistent with
[1, 2, 3] + [4] = [1, 2, 3, 4]
Consider another sequence, this time involving slice replacements.
>>> a = [1,2,3]; a[0:] = ['a']; a
['a']
>>> a = [1,2,3]; a[1:] = ['a']; a
[1, 'a']
>>> a = [1,2,3]; a[2:] = ['a']; a
[1, 2, 'a']
>>> a = [1,2,3]; a[3:] = ['a']; a
[1, 2, 3, 'a']
Another way of looking at slice replacement is to treat it as appending a list to a sublist of the original. When that sublist is the list itself, then it is equal to appending to the original.
Why can I return a slice from a list that starts with a non-existent element?
I can't tell for sure, but my guess would be that it's because a reasonable value can be returned (there are no elements this refers to, so return an empty list).
And why does this allow me to expand the list?
Your slice selects the last 0 elements of the list. You then replace those elements with the elements of another list.
Also, of the first three above methods for appending an item, which one is preferred
Use .append() if you're adding a single element, and .extend() if you have elements inside another list.
I'm not sure about performance, but my guess would be that those are pretty well optimized.
As I am new to programming in Python. I am trying to remove particular elements from array using for loop which looks like
a=[2,3,1,4,1,1,1,5]
n=a.count(1)
for i in range (len(a)-n):
if (a[i]==1):
del a[i]
else:
a[i]=a[i]
print (a)
I want to remove 1 from array a. But, I am getting result as:
[2, 3, 4, 1, 1, 5].
That is 1 still exists in my new array. Can somebody please answer my problem?
try like this:
a = [2,3,1,4,1,1,1,5]
a = [x for x in a if x!=1] # this is called list comprehension
note Never modify list while iterating
Use a while loop and the remove method:
a = [2, 3, 1, 4, 1, 1, 1, 5]
while 1 in a:
a.remove(1)
print a
The real answer to your question (which none of the other answers addresses) is that every time you remove an item, the index i moves past it.
in your case:
a = [2,3,1,4,1,1,1,5]
after deleting the 5th item in the original list, the pointer moves to the 6th item, and the new 5th item (the second 1 in the sequence of three 1s) is skipped.
Regarding the comment never modify a list in a loop, try to implement an in-place algorithm like Fisher-Yates without modifying the list. Never say never. Know what you're doing.
The OP changes the list in-place, not creating a new list.
There are two methods, the second is safe, the first might be faster.
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(len(a)-1, -1, -1):
if a[i] == toremove:
del a[i]
and
a = [2, 3, 1, 4, 1, 1, 1, 5]
toremove = 1
for i in range(a.count(toremove)):
a.remove(toremove)
The second removes the element however many times it exists (before the loop). Since we are not iterating on the list, it is safe to use the remove method.
Both fragments should be O(n) (but haven't done the calculations).
You can copy a and then remove but you cannot iterate over and delete elements from the same list, if your list starts with n elements python will have n pointers to each element so removing elements from the list as your are iterating over it will cause elements to be missed.python has no way of knowing you have removed elements from the list:
a = [2,3,1,4,1,1,1,5]
for ele in a[:]:
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
You can also use reversed which returns and iterator avoiding creating a whole copy of the list at once:
a = [2,3,1,4,1,1,1,5]
for ele in reversed(a):
if ele == 1:
a.remove(1)
print(a)
[2, 3, 4, 5]
Or using a list comprehension with the [:] syntax so we actually update the original object:
a[:] = (ele for ele in a if ele != 1)
All the above are linear operations using a single pass over a.
Actually as the del statement will remove elements from your list , and as the list that you bound in your loop doesn't been update after the first deleting you remove incorrect elements from your list , so if you want to use del you need to make the list name in your loop to reference to new list , that you can use a function for this aim , but as a more python way you can just use a list comprehension:
>>> a=[2,3,1,4,1,1,1,5]
>>> a=[i for i in a if i !=1]
>>> a
[2, 3, 4, 5]
Or you can use filter :
>>> a=[2,3,1,4,1,1,1,5]
>>> a=filter(lambda x: x !=1,a)
>>> a
[2, 3, 4, 5]
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 months ago.
In my Python 2.7.2 IDLE interpreter:
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist:
mylist.remove(item)
>>> mylist
[2, 4]
Why?
It's because when you iterate over a list, python keeps track of the index in the list. Consider the following code instead:
for i in range(len(mylist)):
if i >= len(mylist):
break
item = mylist[i]
mylist.remove(item)
If we track this (which is essentially what python is doing in your code), then we see that when we remove an item in the list, the number to the right shifts one position to the left to fill the void left when we removed the item. The right item is now at index i and so it will never actually get seen in the iteration because the next thing that happens is we increment i for the next iteration of the for loop.
Now for something a little clever. If instead we iterate over the list backward, we'll clear out the list:
for item in reversed(mylist):
mylist.remove(item)
The reason here is that we're taking an item off the end of the list at each iteration of the for loop. Since we're always taking items off the end, nothing needs to shift (assuming uniqueness in the list -- If the list isn't unique, the result is the same, but the argument gets a bit more complicated).
Of course, If you're looking to remove all the items from a list, you can do that really easily:
del mylist[:]
or even with slice assignment:
mylist[:] = []
(I mention the latter because it can be useful to replace segments of a list with other items which don't even need to be the same length).
That's because you're modifying the list while iterating over it, iterate over a shallow copy instead:
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist[:]: #use mylist[:] or list(mylist)
mylist.remove(item)
...
>>> mylist
[]
You are modifying your list while you are looping through it, which is very bad practice.
Problem is that you are altering the list while iterating on it. Use a list comprehension instead:
mylist = [1, 2, 3, 4, 5]
mylist = [x for x in mylist if condition(x)]
>>> mylist = [1, 2, 3, 4, 5]
>>> for item in mylist:
mylist.remove(item)
in this loop :
1st time it will take 0th element (1) as an item and remove it from mylist.so after that mylist will be [2,3,4,5] but the pointer will be in the 1th element of the new mylist ([2,3,4,5]) that is 3. and it will remove 3.so 2 will not be remove from the list.
thats why after the full operation [2,4] will be left.
using for loop:-
>>>for i in range(len(mylist)):
mylist.remove(mylist[0])
i-=1
>>>mylist
[]
you can do this using while loop:
>>>mylist = [1, 2, 3, 4, 5]
>>>while (len(mylist)>0):
mylist.remove(mylist[0])
print mylist
Use a deep copy of mylist
mylist = [1, 2, 3, 4, 5]
l = [k for k in mylist] # deep copy
for i in range(len(mylist)):
mylist.remove(l[i])
print
print mylist