I have this Python code which confronts, one by one, the items in a list of integers (named 'seen' in the code posted) with all the items in the .f field of another list (named 'maxx' in the code posted).
At every iteration I'm counting (through the c variable) how many times does the j-th item appear in the 'maxx' list, and I want to pop() it from the list if it appears less than three times.
The code works perfectly, but popping an item 'pulls' any subsequent item in the 'seen' list back by one position, therefore every time the if condition is satisfied the loop misses the very next item of the list.
Here is the code:
for indj,j in enumerate(seen): # every item in the 'seen' list..
c=0
for k in maxx: # ..checks for a matching item in the 'maxx' list
if j==k.f:
c=c+1;
if c<3: # if the item appears less than 3 times we pop it
seen.pop(indj)
I have tried to add:
indj=indj-1
j=seen[indj]
At the end of the if construct, but it didn't work
You have to make a new list or work with a copy. When you change a list while looping over it you skip some items. I'd do this:
def filter_low(lst, maxk, threshold=3):
for item in lst:
c = sum(1 for k in maxx if item==k.f)
if c >= threshold:
yield item
new_seen = list(filter_low(seen, maxk, 3))
Which is the same as:
new_seen = [item for item in seen
if sum(1 for k in maxx if item==k.f) >= 3]
You can change the original list by doing
seen[:] = [item for item in seen
if sum(1 for k in maxx if item==k.f) >= 3]
Modifying the list you're iterating over is never a good idea. You could iterate over a copy and modify the actual list with
popped = 0
for indj, j in enumerate(seen[:]):
s = sum(j == k.f for k in maxx)
if s < 3:
seen.pop(indj - popped)
popped += 1
If the seen list is very large, this might be inefficient.
Related
I need to limit number of elements that will be executed through one loop iteration, but after that, I need to continue from the point where loop stopped, and re-run the loop, until all the elements are processed through multiple loop iterations..
Let's say I have a list:
list1=[1,2,3,4,5,6,7,8,9,10]
and static limiter:
limit = 4
so in this case I want to add only 4 elements from the list (1,2,3,4) into some other list and do some kind of processing and parsing, and after that I want to add next 4 elements (5,6,7,8), and do the processing, and at the end the rest of the elements (9,10).
My current code only process the first 4 elements.
I used islice for this.
from itertools import islice
list1=[1,2,3,4,5,6,7,8,9,10]
new_list=[]
limit = 4
for item in islice(list1, limit):
new_list.append(item)
print (new_list)
So, the desired output is to have new_list printed first only with elements [1,2,3,4], after that list is being reset and populated with elements [5,6,7,8] and at the end list is reset with elements [9,10]. Solution should not involve dealing with additional new_lists - only one new list object.
Number of elements in list1 can be less than a limit value.
Note: I am using python2.7 (can't use python 3 on the customer server).
Thanks a lot!
Something like this should work:
list1 = [1,2,3,4,5,6,7,8,9,10]
new_list = []
limit = 4
for item in list1:
new_list.append(item)
if len(new_list) == limit:
print(new_list)
new_list = []
print(new_list)
Essentially, the loop will add entries to new_list, and when there are limit entries in new_list, new_list is emptied.
This may help:
This code initially makes a deep copy of your original list, then the while loop process the list in steps delimited for the number defined in the limit. So in this case it will take 4 elements "do something" (in this case is just poping the elements from the list). When the range is less than the limit it will just take the last elements.
import copy
list1=[1,2,3,4,5,6,7,8,9,10]
list2 = copy.deepcopy(list1)
new_list=[]
limit = 4
counter = 0
while list2:
try:
for i in range(0, limit):
removed_element = list2.pop()
new_list.append(removed_element)
print("cicle finished")
except:
for i in list2:
removed_element = list2.pop()
new_list.append(removed_element)
print("last cicle")
I'm new to python. Could someone help me understand why the following function doesn't work? It is supposed to return a new list with the duplicates removed but instead prints [4,6].
def remove_duplicates(l):
solution = []
for item in l:
if l.count(item) < 2:
solution.append(item)
else:
l.remove(item)
return solution
print (remove_duplicates([4,5,5,5,4,6]))
I thought it iterates one item at a time in a list. So the first 5 would have a count of 3 and be remove, the second five would have a count of 2 and be removed and the third 5 would have a count of 1 and be appended to the solution list. I can't wrap my head around why the 5s would be completely removed but the 4s would not.
You must not remove items from a list, you are iterating at the moment. Iterating is done by incrementing an index internally.
If you want to keep the last occurence of an item, best, count them at first:
from collections import Counter
def remove_duplicates(l):
solution = []
counts = Counter(l)
for item in l:
if counts[item] == 1:
solution.append(item)
else:
counts[item] -= 1
return solution
Use set data type in python to remove the duplicates.
a = [4,5,5,5,4,6]
solution = list(set(a))
Output:
[4,5,6]
Need a solution to fixing the list size after deleting an element while iterating it.
Cannot create a new list, as the issue for a bigger problem (800 iterations) and it is computational expensive to make a new list every time
list = [1,2,3,4,5,6,7,8,9,10]
for i in range (len(list)):
if a[i] < 6:
del a[i]
output should be the original list, with values now
[6,7,8,9,10]
just overwrite the current list ,
lst = [1,2,3,4,5,6,7,8,9,10]
lst =[i for i in lst if i>=6]
# lst = [6,7,8,9,10]
See the UPDATE at the end of this answer for performance measurements
If you're going to only delete one element from the list (or elements at the beginning of the list), you may want to find the index instead of going through all of them:
index = lst.index(6)
del lst[:index]
If you're concerned with index offsets while traversing the list, you can keep track of the number of deleted entries and compute the actual index accordingly:
originalLen = len(lst)
for originalIndex in range(originalLen):
i = originalIndex - originalLen + len(lst)
if lst[i] < 6:
del lst[i]
You could even generalize this by creating an iterator that does the housekeeping for you:
def stableIndexes(lst):
oLen = len(lst)
for oi in range(oLen): yield oi - oLen + len(lst)
for i in stableIndexes(lst):
if lst[i] < 6:
del lst[i]
If you're going to delete multiple items, you could create a list of indexes to delete and process them in reverse order at the end of the loop:
indexes = []
for i,a in enumerate(lst):
if a > 2 and a < 5:
indexes.append(i)
for index in reversed(indexes):
del lst[index]
or you can process the list in reverse order and delete as you go without indexes getting mixed up:
for i in range(len(lst)-1,-1,-1):
if lst[i] > 2 and lst[i] < 5:
del lst[i]
Another way to do it would be to manually shift the subsequent items after your delete at least one and truncate the list at the end:
i = 0
for index,a in enumerate(lst):
if a > 2 and a < 5:
continue
if i < index:
lst[i] = a
i += 1
del lst[i:]
Finally, an alternative approach could be to assign None to the items that you want to delete and skip the None values on subsequent iterations:
for i,a in enumerate(lst):
if a is None: continue
if a > 2 and a < 5:
lst[i] = None
...
UDATE
I made a few performance tests deleting entries from a 1,000,000 element list. It turns out that using a list comprehension (i.e. making a second copy of the list) is faster than all of the schemes I described above:
Method del 1 in 13 del only 1
---------------------------------- ----------- ----------
New list (using comprehension): 0.00650 0.00799
Assign None to deleted items: 0.00983 0.01152
Manually shifting elements: 0.01558 0.01741
Delete as you go in reverse order: 0.07436 0.00942
List of indexes to delete: 0.09998 0.01044
So, the "computationally expensive" theory of making a new list doesn't hold true in practice. When compared to all other methods, it is actually the most economical approach in terms of processing time.
This is most likely caused by COW (Copy On Write) memory management which will allocate memory as soon as you change something in the list. Thus there will always be a new list created (internally).
By explicitly creating a new list yourself, you take advantage of this memory allocation and save on any additional shifting of data within the list's memory space.
In conclusion, you should let Python handle these concerns.
First of all, it's easier to use list comprehensions:
lst = [1,2,3,4,5,6,7,8,9,10]
t = [x for x in lst if x >= 6]
print(t)
And if you're not allowed to create a new list object, you can always use a generator expression:
lst = [1,2,3,4,5,6,7,8,9,10]
g = (x for x in lst if x >= 6)
for val in g:
print(val)
A safe (if modifying a list over which one iterates may be called "safe" to any extent) solution is to iterate backwards:
n = len(list_)
for i in xrange(n):
idx = n - i - 1
if list_[idx] < 6:
del list_[idx]
This way, you will at least not have changes impact the unchanged part of the list.
I want to delete some array from list. But I'm using wrong range.
At start the range is correct.
This should work, if string in variable result[b][2:3] then delete result[b]
for b in range(len(result)):
if 'FillLevel' in result[b][2:3]:
del result[b]
After that I have error: IndexError: list index out of range
I want to find this string and delete whole line (array):
V;4;FillLevel[1];CPUA.DB1610.0,I0,64;RW
V;4;FillLevel[2];CPUA.DB1610.0,I;RW
V;4;FillLevel[5];CPUA.DB1610.6,I;RW
V;4;FillLevel[6];CPUA.DB1610.8,I;RW
V;4;FillLevel[11];CPUA.DB1610.18,I;RW
Why this code:
print(result[4][2:3])
print(result[5][2:3])
print(result[6][2:3])
print(result[7][2:3])
print(result[8][2:3])
print(result[9][2:3])
print(result[10][2:3])
b = 0
while b < len(result):
if 'FillLevel' in result[b][2:3]:
del result[b]
del adress[b]
print('yes')
b += 1
Showing only once 'yes' ?
['FillLevel']
['FillLevel[1]']
['FillLevel[2]']
['FillLevel[3]']
['FillLevel[4]']
['FillLevel[5]']
['FillLevel[6]']
yes
The issue is that del result[b] changes the composition (and the length of) result, thereby interfering with your loop.
Perhaps the easiest way to fix this is by rephrasing your code as a list comprehension:
result = [r for r in result if 'FillLevel' not in r[2:3]]
Alternatively, you could fix it by iterating in reverse:
for b in range(len(result) - 1, -1, -1):
if 'FillLevel' in result[b][2:3]:
del result[b]
Let's say there are 10 items in the list.
Half-way through you delete one of the items; now there are 9 items in the list.
In the last cycle, your loop asks for the tenth item. My guess is that's where the index error is happening (though it could be due to the [2:3] call as well, depending on the contents of your list)
A more pythonic solution would be
result = [val for val in result if 'FillLevel' not in val[2:3]]
If you want to preserve the same list and parse it in the strait order you can use a while loop which evaluate the len(result) in each iteration
b = 0
while b < len(result) :
if 'FillLevel' in result[b][2:3]:
del result[b]
b += 1
for first
- it's mach easyer to iterate by list withot getting length, probably you are got an error coz length of list is changing during loop
for second
- you are trying to check 'FillLevel' in slice of string. slice return one character
- try to not midify your list but make new one with filtered items
like this:
new_list = []
for b in result:
if 'FillLevel' not in b:
new_list.append(b)
or check about List Comprehensions and type this:
[i for i in result if 'FillLevel' not in i]
Hey, I was trying to delete an item form a list (without using set):
list1 = []
for i in range(2,101):
for j in range(2,101):
list1.append(i ** j)
list1.sort()
for k in range(1,len(list1) - 1):
if (list1[k] == list1[k - 1]):
list1.remove(list1[k])
print "length = " + str(len(list1))
The set function works fine, but i want to apply this method. Except I get:
IndexError: list index out of range
on the statement:
if (list1[k] == list1[k - 1]):
Edited to add
(Thanks to Ned Batchelder) the working code is:
list1 = []
for i in range(2,101):
for j in range(2,101):
list1.append(i ** j)
list1.sort()
k = 0
while k < len(list1) - 1: # while loop instead of for loop because "The range function is evaluated once before the loop is entered"
k += 1
if (list1[k] == list1[k - 1]):
list1.remove(list1[k])
list1.sort()
k -= 1 # "If you find a duplicate, you don't want to move onto the next iteration, since you'll miss potential runs of more than two duplicates"
print "length = " + str(len(list1))
Your code doesn't work because in your loop, you are iterating over all the indexes in the original list, but shortening the list as you go. At the end of the iteration, you will be accessing indexes that no longer exist:
for k in range(1,len(list1) - 1):
if (list1[k] == list1[k - 1]):
list1.remove(list1[k])
The range function is evaluated once before the loop is entered, creating a list of all the indexes in the list. Each call to remove shortens the list by one, so if you remove any elements, you're guaranteed to get your error at the end of the list.
If you want to use a loop like this, try:
k = 1
while k < len(list1):
if list1[k] == list1[k-1]:
del list1[k]
else:
k += 1
I fixed a few other things:
You don't need parentheses around the condition in Python if statements.
If you find a duplicate, you don't want to move onto the next iteration, since you'll miss potential runs of more than two duplicates.
You want to start from index 1, not zero, since k=0 will access list1[-1].
It looks as if you're trying to uniquify a list (clarification would be awesome) so take a look here: http://www.peterbe.com/plog/uniqifiers-benchmark
There is also this question here on SO: In Python, what is the fastest algorithm for removing duplicates from a list so that all elements are unique *while preserving order*?
Instead of removing items Write a list comprehension of the things you want in the new list:
list1[:] = [list1[k] for k in range(1,len(list1) - 1)
if not list1[k] == list1[k - 1] ]
Your method breaks because you remove items from the list. When you do that, the list becomes shorter and the next loop iteration has skipped a item. Say you look at k=0 and L = [1,2,3]. You delete the first item, so L = [2,3] and the next k=1. So you look at L[1] which is 3 -- you skipped the 2!
So: Never change the list you iterate on
You can use del :
l = [1, 2, 3, 4]
del l[2]
print l
[1, 2, 4]