Why isn't this for loop working? My goal is to delete every 1 from my list.
>>> s=[1,4,1,4,1,4,1,1,0,1]
>>> for i in s:
... if i ==1: s.remove(i)
...
>>> s
[4, 4, 4, 0, 1]
Never change a list while iterating over it. The results are unpredictable, as you're seeing here. One simple alternative is to construct a new list:
s = [i for i in s if i != 1]
If for some reason you absolutely have to edit the list in place rather than constructing a new one, my off-the-cuff best answer is to traverse it finding the indices that must be deleted, then reverse traverse that list of indices removing them one by one:
indices_to_remove = [i for (i, val) in enumerate(s) if val == 1]
for i in reversed(indices_to_remove):
del s[i]
Because that removes elements from the end of the list first, the original indices computed remain valid. But I would generally prefer computing the new list unless special circumstances apply.
Consider this code:
#!/usr/bin/env python
s=[1, 4, 1, 4, 1, 4, 1, 1, 0, 1]
list_size=len(s)
i=0
while i!=list_size:
if s[i]==1:
del s[i]
list_size=len(s)
else:
i=i + 1
print s
Result:
[4, 4, 4, 0]
For short, your code get some undesirable result because of "size" and "index positions" of your list are changed every times you cut the number 1 off and your code is clearly proved that for each loop in Python can not handle a list with a dynamic size.
You should not change the content of list while iterating over it
But you could iterate over the copy of the list content and change it in your case
Code:
s=[1,4,1,4,1,4,1,1,0,1]
for i in s[:]:
if i ==1: s.remove(i)
print s
Output:
[4, 4, 4, 0]
As #metatoaster stated you could use filter
Code:
s=[1,4,1,4,1,4,1,1,0,1]
s=list(filter(lambda x:x !=1,s))
print s
[4, 4, 4, 0]
You could use filter to remove multiple things example
Code:
s=[1,4,1,4,1,4,1,1,0,1,2,3,5,6,7,8,9,10,20]
remove_element=[1,2,3,5,6,7,8,9]
s=list(filter(lambda x:x not in remove_element,s))
print s
[4, 4, 4, 0, 10, 20]
This doesn't work because you are modifying the list as it is iterating, and the current pointer moves past one of the 1 you check against. We can illustrate this:
>>> for i in s:
... print(s)
... if i == 1:
... s.remove(i)
...
[1_, 4, 1, 4, 1, 4, 1, 1, 0, 1]
[4, 1_, 4, 1, 4, 1, 1, 0, 1]
[4, 4, 1_, 4, 1, 1, 0, 1]
[4, 4, 4, 1_, 1, 0, 1]
[4, 4, 4, 1, 0_, 1]
[4, 4, 4, 1, 0, 1_]
I added _ to the element being compared. Note how there was only 6 passes in total and with one of the 1s actually skipped over from being ever looked at. That ends up being the element that was removed because list.remove removes the first occurrence of the element specified, and it is an O(n) operation on its own which gets very expensive once your list gets big - this is O(n) even if the item is in the beginning, as it has to copy every single item from everything after the item one element forward as python lists are more like C styled arrays than Java linked-lists (if you want to use linked-lists, use collections.deque). O(n) towards the end because it has to iterate through the entire list to do its own comparison too. Your resulting code can result in a worst case runtime complexity of O(n log n) if you make use of remove.
See Python's data structure time complexity
Peter's answer already covered the generation of a new list, I am only answering why and how your original code did not work exactly.
Related
Given a sorted list such as:
[1, 2, 2, 3, 3, 4]
My goal is to check if there are any numbers repeated, and if so, shift the element and all the numbers before it by one to the left as such:
[1, 2, 2, 3, 3, 4]
[0, 1, 2, 3, 3, 4]
[-1, 0, 1, 2, 3, 4]
right now this is my approach:
def shifting(data):
i = 0
while i< len(data)-1:
if data[i]==data[i+1]:
j=i
while j>=0:
data[j]-=1
j-=1
i+=1
return data
But this is an O(n^2) algorithm and takes a lot of time to run with very long lists. I want to find a more efficient approach. Any ideas?
I would agree with Stef. The main idea is that you don't really need to have this nested while loop. All you need is a single pass to pinpoint where the duplications occur and apply a shift accordingly.
I'll propose something a little bit more complex but might be more compact:
import numpy as np
input_list = [1, 2, 2, 3, 3, 4]
# Convert to array for easier indexing
output_ary = np.array(input_list)
# Pinpoint the location at which duplications occur
duplication_indicator = output_ary[:-1] - output_ary[1:] == 0
# Compute the corresponding shift
shift = np.cumsum(duplication_indicator[::-1])[::-1]
# Apply the shift
output_ary[:-1] -= shift
# Convert back to list
output_list = output_ary.tolist()
The main idea is that after you've pinpointed the duplication locations, you can compute the corresponding shift by looking at how many more duplications occur to the right. This could be done by simply doing a reversed cumulative sum (summing from the right to left). Applying this shift to the original list then gives the desired output.
Iterate on the data from right to left. Keep a counter decrement that tells you how many duplicates you've encountered so far, and thus, by how much you want to decrement every element you see.
This is linear instead of quadratic: you only iterate on the data once.
When writing python code, I strongly suggest using for-loops rather than while-loops whenever you can, and in particular when you know the length of the loop by advance.
In your code, i = 0; while i < len(data) - 1: i += 1 can be replaced by for i in range(len(data)-1):.
To iterate from right to left: for i in range(len(data)-1, -1, -1):
The logic is no fully clear, but IIUC is seems easy to remove the duplicates with np.unique, then to left fill the array with a range to go to the initial length:
a2 = np.unique(a)
out = np.r_[np.arange(-(a.shape[0]-a2.shape[0]), 0)+1, a2]
output:
array([-1, 0, 1, 2, 3, 4])
on a = np.array([1,2,2,6,6,7]):
array([-1, 0, 1, 2, 6, 7])
Suppose that I have a list that has [0, 1, 2, 3 , 4, 5, 6] in it. I want to remove those elements that are greater than or equal to 3 and add those removed elements to the beginning of the list. So I wrote the code below:
list = [0, 1, 2, 3, 4, 5, 6]
new_list =[]
for number in list:
if number >= 3:
dropped_number = list.pop()
new_list.append(dropped_number)
new_list.sort()
new_list += list
print(new_list)
However, when I ran the code, the result was displayed as [5, 6, 0, 1, 2, 3 , 4]. Could anyone please explain to me at which step I did wrong here?
There are two issues with your code.
the number you obtain with list.pop() is not the one you just checked with your condition (it is merely the last one in the list)
When you reach 3, list.pop() removes 6,
When you reach 4, list.pop() removes 5,
You never reach 5 because you're at the end of what remains of the list at that point.
removing items from a list within a for-loop on the same list will cause the for-loop to skip items or complain that the list changed during iterations. So, even if you were to pop the appropriate number, your loop would miss items.
You also don't need to sort new_list every time you add to it, you can do it once at the end, but that just optimization.
Instead of a for-loop, you could use the sort method with a key parameter that returns a boolean indicating True for elements that do not meet your conditions (i.e that will be shifted to the right). Because Python's sort is stable, this will only place elements in two groups without otherwise changing their relative order.
L = [0, 2, 4, 6, 1, 3, 5]
L.sort(key=lambda x: not x>=3)
print(L) # [4, 6, 3, 5, 0, 2, 1]
If you need a more procedural solution, you can separate the values in two lists that you stick together at the end:
L = [0, 2, 4, 6, 1, 3, 5]
left,right = [], []
for x in L:
if x >= 3: left.append(x)
else: right.append(x)
L = left + right
# [4, 6, 3, 5, 0, 2, 1]
Modifying a list while iterating over it is usually problematic. What if instead you thought of the problem as building a new list out of two subsets of the original list?
>>> old_list = list(range(7))
>>> [i for i in old_list if i >= 3] + [i for i in old_list if i < 3]
[3, 4, 5, 6, 0, 1, 2]
The reason your program doesn't work is because you are modifying the list whilst searching through it. Instead, you can start by adding the elements >= 3 to a new list and then separately appending the elements < 3 to the list. Also, considering you are created a second 'new_list', there is no need to remove the elements from the first list.
Your new code:
list = [0, 1, 2, 3, 4, 5, 6]
new_list = []
# Append numbers greater than 3 to the new list
for number in list:
if number >= 3:
new_list.append(number)
# Append the numbers less than 3 to the new list
new_list += list[0:list.index(new_list[0])]
print(new_list)
Just to note, this method takes a section of the original list from position 0, to the position (.index) of the first item in the new list, which automatically generates the < 3 condition as the first item in the new list corresponds to the items before the >= 3 condition is met.
list[0:list.index(new_list[0])]
This question already has answers here:
Sort a part of a list in place
(3 answers)
Closed 3 years ago.
Suppose I have a list [2, 4, 1, 3, 5].
I want to sort the list just from index 1 to the end, which gives me [2, 1, 3, 4, 5]
How can I do it in Python?
(No extra spaces would be appreciated)
TL;DR:
Use sorted with a slicing assignment to keep the original list object without creating a new one:
l = [2, 4, 1, 3, 5]
l[1:] = sorted(l[1:])
print(l)
Output:
[2, 1, 3, 4, 5]
Longer Answer:
After the list is created, we will make a slicing assignment:
l[1:] =
Now you might be wondering what does [1:], it is slicing the list and starts from the second index, so the first index will be dropped. Python's indexing starts from zero, : means get everything after the index before, but if it was [1:3] it will only get values that are in between the indexes 1 and 3, let's say your list is:
l = [1, 2, 3, 4, 5]
If you use:
print(l[1:])
It will result in:
[2, 3, 4, 5]
And if you use:
print(l[1:3])
It will result in:
[2, 3]
About slicing, read more here if you want to.
And after slicing we have an equal sign =, that just simply changes what's before the = sign to what's after the = sign, so in this case, we use l[1:], and that gives [2, 3, 4, 5], it will change that to whatever is after the = sign.
If you use:
l[1:] = [100, 200, 300, 400]
print(l)
It will result in:
[1, 100, 200, 300, 400]
To learn more about it check out this.
After that, we got sorted, which is default builtin function, it simple sorts the list from small to big, let's say we have the below list:
l = [3, 2, 1, 4]
If you use:
print(sorted(l))
It will result in:
[1, 2, 3, 4]
To learn more about it check this.
After that we come back to our first topic about slicing, with l[1:], but from here you know that it isn't only used for assignments, you can apply functions to it and deal with it, like here we use sorted.
Maybe temporarily put something there that's smaller than the rest? Should be faster than the other solutions. And gets as close to your "No extra spaces" wish as you can get when using sort or sorted.
>>> tmp = l[0]
>>> l[0] = float('-inf')
>>> l.sort()
>>> l[0] = tmp
>>> l
[2, 1, 3, 4, 5]
Benchmarks
For the example list, 1,000,000 iterations (and mine of course preparing that special value only once):
sort_u10 0.8149 seconds
sort_chris 0.8569 seconds
sort_heap 0.7550 seconds
sort_heap2 0.5982 seconds # using -1 instead of -inf
For 50,000 lists like [int(x) for x in os.urandom(100)]:
sort_u10 0.4778 seconds
sort_chris 0.4786 seconds
sort_heap 0.8106 seconds
sort_heap2 0.4437 seconds # using -1 instead of -inf
Benchmark code:
import timeit, os
def sort_u10(l):
l[1:] = sorted(l[1:])
def sort_chris(l):
l = l[:1] + sorted(l[1:])
def sort_heap(l, smallest=float('-inf')):
tmp = l[0]
l[0] = smallest
l.sort()
l[0] = tmp
def sort_heap2(l):
tmp = l[0]
l[0] = -1
l.sort()
l[0] = tmp
for _ in range(3):
for sort in sort_u10, sort_chris, sort_heap, sort_heap2, sort_rev:
number, repeat = 1_000_000, 5
data = iter([[2, 4, 1, 3, 5] for _ in range(number * repeat)])
# number, repeat = 50_000, 5
# data = iter([[int(x) for x in os.urandom(100)] for _ in range(number * repeat)])
t = timeit.repeat(lambda: sort(next(data)), number=number, repeat=repeat)
print('%10s %.4f seconds' % (sort.__name__, min(t)))
print()
Use sorted with slicing:
l[:1] + sorted(l[1:])
Output:
[2, 1, 3, 4, 5]
For the special case that you actually have, according to our comments:
Q: I'm curious: Why do you want this? – Heap Overflow
A: I'm trying to make a next_permutation() in python – nwice13
Q: Do you really need to sort for that, though? Not just reverse? – Heap Overflow
A: Yup, reverse is ok, but I just curious to ask about sorting this way. – nwice13
I'd do that like this:
l[1:] = l[:0:-1]
You can define your own function in python using slicing and sorted and this function (your custom function) should take start and end index of the list.
Since list is mutable in python, I have written the function in such a way it doesn't modify the list passed. Feel free to modify the function. You can modify the list passed to this function to save memory if required.
def sortedList(li, start=0, end=None):
if end is None:
end = len(li)
fi = []
fi[:start] = li[:start]
fi[start:end] = sorted(li[start:end])
return fi
li = [2, 1, 4, 3, 0]
print(li)
print(sortedList(li, 1))
Output:
[2, 1, 4, 3, 0]
[2, 0, 1, 3, 4]
I'm trying to go through a list in reverse order, starting with the -0 indexed item (which is also the 0th item), rather than the -1 indexed item, so that I'll now have the new list to use. I've come up with two ways to do this, but neither seems both concise and clear.
a_list = [1, 2, 3, 4, 5]
print(a_list[:1] + a_list[:0:-1]) # take two slices of the list and add them
# [1, 5, 4, 3, 2]
list_range = range(-len(a_list)+1,1)[::-1] # create an appropriate new index range mapping
print([a_list[i] for i in list_range]) # list comprehension on the new range mapping
# [1, 5, 4, 3, 2]
Is there a way in python 3 to use slicing or another method to achieve this more simply?
If you are up for a programming golf:
>>> a_list = [1, 2, 3, 4, 5]
>>> [a_list[-i] for i in range(len(a_list))]
[1, 5, 4, 3, 2]
I think your first suggestion is the cleanest way of doing this. If you're really optimizing for character count, you can remove two characters from the first slice:
print(a_list[:1] + a_list[:0:-1])
Shift everything left by one and reverse.
my_list.append(my_list.pop(0))
print my_list[::-1]
I am new to Python and learning data structure in Python. I am trying to implement a bubble sort algorithm in python and I did well but I was not getting a correct result. Then I found some tutorial and there I saw that they are first setting a base range for checking.
So the syntax of range in python is:
range([start], stop[, step])
And the bubble sort algorithm is:
def bubbleSort(alist):
for i in range(len(alist) - 1, 0, -1):
for j in range(i):
if alist[j] > alist[j+1]:
temp = alist[j]
alist[j] = alist[j+1]
alist[j+1] = temp
return alist
print(bubbleSort([5, 1, 2, 3, 9, 8, 0]))
I understood all the other logic of the algorithm but I am not able to get why the loop is starting from the end of the list and going till first element of the list:
for i in range(len(alist) - 1, 0, -1):
Why is this traversing the list in reverse? The main purpose of this loop is setting the range condition only so why can't we traverse from the first element to len(list) - 1 like this:
for i in range(0, len(alist) - 1, 1):
In your code, the index i is the largest index that the inner loop will consider when swapping the elements. The way bubble sort works is by swapping sibling elements to move the largest element to the right.
This means that after the first outer iteration (or the first full cycle of the inner loop), the largest element of your list is positioned at the far end of the list. So it’s already in its correct place and does not need to be considered again. That’s why for the next iteration, i is one less to skip the last element and only look at the items 0..len(lst)-1.
Then in the next iteration, the last two elements will be sorted correctly, so it only needs to look at the item 0..len(lst)-2, and so on.
So you want to decrement i since more and more elements at the end of the list will be already in its correct position and don’t need to be looked at any longer. You don’t have to do that; you could also just always have the inner loop go up to the very end but you don’t need to, so you can skip a few iterations by not doing it.
I asked why we are going reverse in the list like len(list)-1,0. Why are we not going forward way like 0,len(list)-1?
I was hoping that the above explanation would already cover that but let’s go into detail. Try adding a print(i, alist) at the end of the outer loop. So you get the result for every iteration of i:
>>> bubbleSort([5, 1, 3, 9, 2, 8, 0])
6 [1, 3, 5, 2, 8, 0, 9]
5 [1, 3, 2, 5, 0, 8, 9]
4 [1, 2, 3, 0, 5, 8, 9]
3 [1, 2, 0, 3, 5, 8, 9]
2 [1, 0, 2, 3, 5, 8, 9]
1 [0, 1, 2, 3, 5, 8, 9]
As you can see, the list will be sorted from the right to the left. This works well for our index i which will limit how far the inner loop will go: For i = 4 for example, we already have 3 sorted elements at the end, so the inner loop will only have to look at the first 4 elements.
Now, let’s try changing the range to go in the other direction. The loop will be for i in range(0, len(alist)). Then we get this result:
>>> bubbleSort([5, 1, 3, 9, 2, 8, 0])
0 [5, 1, 3, 9, 2, 8, 0]
1 [1, 5, 3, 9, 2, 8, 0]
2 [1, 3, 5, 9, 2, 8, 0]
3 [1, 3, 5, 9, 2, 8, 0]
4 [1, 3, 5, 2, 9, 8, 0]
5 [1, 3, 2, 5, 8, 9, 0]
6 [1, 2, 3, 5, 8, 0, 9]
As you can see, this is not sorted at all. But why? i still limits how far the inner loop will go, so at i = 1, the loop will only look at the first pair and sort that; the rest will stay the same. At i = 2, the loop will look at the first two pairs and swap those (once!); the rest will stay the same. And so on. By the time the inner loop can reach the last element (which is only on the final iteration), there aren’t enough iterations left to swap the zero (which also happens to be the smallest element) to the very left.
This is again because bubble sort works by sorting the largest elements to the rightmost side first. So we have to start the algorithm by making the inner loop be able to reach that right side completely. Only when we are certain that those elements are in the right position, we can stop going that far.
There is one way to use a incrementing outer loop: By sorting the smallest elements first. But this also means that we have to start the inner loop on the far right side to make sure that we check all elements as we look for the smallest element. So we really have to make those loops go in the opposite directions.
It's because when you bubble from the start of the list to the end, the final result is that the last item in the list will be sorted (you've bubbled the largest item to the end). As a result, you don't want to include the last item in the list when you do the next bubble (you know it's already in the right place). This means the list you need to sort gets shorter, starting at the end and going down towards the start. In this code, i is always the length of the remaining unsorted list.
You can use this for:
for i in range(0,len(alist)-1,1):
but consequently you should change your second iteration:
for j in range(0,len(alist)-i,1):
I think the purpose of using reverse iteration in the first line is to simplify the second iteration. This is the advantage of using python
as #Jeremy McGibbon's answer, the logic behind bubble sort is to avoid j reach the "sorted part" in the behind of list. When using the example code, j range will be decreased as the value of i decrease. When you change i to increasing, you should handle j iteration differently
You can write the code as follow
lst = [9,6,5,7,8,3,2,1,0,4]
lengthOfArray = len(lst) - 1
for i in range(lengthOfArray):
for j in range(lengthOfArray - i):
if lst[j] > lst[j + 1]:
lst[j], lst[j + 1] = lst[j + 1], lst[j]
print(lst)