We Keep Coding

The same list has different lengths please help me understand [duplicate]

I'm currently developing a program in python and I just noticed that something was wrong with the foreach loop in the language, or maybe the list structure. I'll just give a generic example of my problem to simplify, since I get the same erroneous behavior on both my program and my generic example:
x = [1,2,2,2,2]
for i in x:
x.remove(i)
print x
Well, the problem here is simple, I though that this code was supposed to remove all elements from a list. Well, the problem is that after it's execution, I always get 2 remaining elements in the list.
What am I doing wrong? Thanks for all the help in advance.
Edit: I don't want to empty a list, this is just an example...

This is a well-documented behaviour in Python, that you aren't supposed to modify the list being iterated through. Try this instead:
for i in x[:]:
x.remove(i)
The [:] returns a "slice" of x, which happens to contain all its elements, and is thus effectively a copy of x.

When you delete an element, and the for-loop incs to the next index, you then skip an element.
Do it backwards. Or please state your real problem.

I think, broadly speaking, that when you write:
for x in lst:
# loop body goes here
under the hood, python is doing something like this:
i = 0
while i < len(lst):
x = lst[i]
# loop body goes here
i += 1
If you insert lst.remove(x) for the loop body, perhaps then you'll be able to see why you get the result you do?
Essentially, python uses a moving pointer to traverse the list. The pointer starts by pointing at the first element. Then you remove the first element, thus making the second element the new first element. Then the pointer move to the new second – previously third – element. And so on. (it might be clearer if you use [1,2,3,4,5] instead of [1,2,2,2,2] as your sample list)

Why don't you just use:
x = []
It's probably because you're changing the same array that you're iterating over.
Try Chris-Jester Young's answer if you want to clear the array your way.

I know this is an old post with an accepted answer but for those that may still come along...
A few previous answers have indicated it's a bad idea to change an iterable during iteration. But as a way to highlight what is happening...
>>> x=[1,2,3,4,5]
>>> for i in x:
... print i, x.index(i)
... x.remove(i)
... print x
...
1 0
[2, 3, 4, 5]
3 1
[2, 4, 5]
5 2
[2, 4]
Hopefully the visual helps clarify.

I agree with John Fouhy regarding the break condition. Traversing a copy of the list works for the remove() method, as Chris Jester-Young suggested. But if one needs to pop() specific items, then iterating in reverse works, as Erik mentioned, in which case the operation can be done in place. For example:
def r_enumerate(iterable):
"""enumerator for reverse iteration of an iterable"""
enum = enumerate(reversed(iterable))
last = len(iterable)-1
return ((last - i, x) for i,x in enum)
x = [1,2,3,4,5]
y = []
for i,v in r_enumerate(x):
if v != 3:
y.append(x.pop(i))
print 'i=%d, v=%d, x=%s, y=%s' %(i,v,x,y)
or with xrange:
x = [1,2,3,4,5]
y = []
for i in xrange(len(x)-1,-1,-1):
if x[i] != 3:
y.append(x.pop(i))
print 'i=%d, x=%s, y=%s' %(i,x,y)

If you need to filter stuff out of a list it may be a better idea to use list comprehension:
newlist = [x for x in oldlist if x%2]
for instance would filter all even numbers out of an integer list

The list stored in the memory of a computer. This deals with the pointer to a memory artifact. When you remove an element, in a by-element loop, you are then moving the pointer to the next available element in the memory address
You are modifying the memory and iterating thru the same.
The pointer to the element moves through the list to the next spot available.
So in the case of the Size being 5...enter code here
[**0**,1,2,3,4]
remove 0 ---> [1,**2**,3,4] pointer moves to second index.
remove 2 ---> [1,3,**4**] pointer moves to 3rd index.
remove 4 ---> [1,3]
I was just explaining this to my students when they used pop(1). Another very interesting side-effect error.
x=[1,**2**,3,4,5]
for i in x:
x.pop(1)
print(x,i)
[1, **3**, 4, 5] 1 at index 0 it removed the index 1 (2)
[1, **4**, 5] 3 at index 1 it removed the index 1 (3)
[1, 5] 5 at index 2 it removed the index 1 (4)
heh.
They were like why isnt this working... I mean... it did... exactly what you told it to do. Not a mind reader. :)

Index error trying to deduplicate a list only using if and for

My goal is to write a program that removes duplicated elements from a list – E.g., [2,3,4,5,4,6,5] →[2,3,4,5,6] without the function set (only if and for)
At the end of my code I got stuck. I have tried to change everything in the if statement but it got me to nowhere, same error repeating itself here:
n=int(input('enter the number of elements in your list '))
mylist=[]
for i in range (n):
for j in range (n):
ele=input(' ')
if mylist[i]!=mylist[j]: ***here is the error exactly , I dont really understand what does the out-of-range above problem have to do with the if statement right here***
mylist.append(ele)
print(mylist)
However, I changed nearly everything and I still got the following error:
if mylist[i]!=mylist[j]:
IndexError: list index out of range
Why does this issue keep coming back? ps: I cant use the function set because I am required to use if and for only

You can do this all with a list comprehension that checks if the value has previously appeared in the list slice that you have already looped over.
mylist = [1,2,3,3,4,5,6,4,1,2,4,3]
outlist = [x for index, x in enumerate(mylist) if x not in mylist[:index]]
print(outlist)
Output
[1, 2, 3, 4, 5, 6]
In case you're not familiar with list comprehensions yet, the above comprehension is functionally equivalent to:
outlist = []
for index, x in enumerate(mylist):
if x not in mylist[:index]:
outlist.append(x)
print(outlist)

A set would be more efficient, but
>>> def dedupe(L):
... seen = []
... return [seen.append(e) or e for e in L if e not in seen]
...
>>> dedupe([2,3,4,5,4,6,5])
[2, 3, 4, 5, 6]
This is not just a toy solution. Because sets can only contain hashable types, you might really have reason to resort to this kind of thing. This approach entails a linear search of the seen items, which is fine for small cases, but is slow compared to hashing.
It still may be possible to do better than this in some cases. Even if you can't hash them, if you can at least sort the seen elements, then you can speed up the search to logarithmic time by using a binary search. (See bisect).

I want to write a function that takes a list and returns it with all duplicates removed, without creating another list or string

I have tried this, it gives the output it should give([1,3,2]), however the problem is it keeps printing the output for infinite times without stop,, is there any solutions with out changing the idea of the code.
a= [1,2,2,2,1,3,2]
def rem_dup(L):
while len(L):
for i in L:
y= L.count(i)
if y>1:
L.remove(i)
print L
rem_dup(a)

Unless the point of this function is to exercise your python skills, it sounds like you want a set. A set is like a list but does not allow duplicate values. If you want your final data structure to be a list, you could do something like this:
final_list = list(set(original_list))

One way to safely do this is to loop over the list in reverse and remove only from the back:
>>> for i in range(len(a) - 1, -1, -1):
... if a.count(a[i]) > 1:
... del a[i]
...
>>> a
[1, 2, 3]
But this will be polynomial time, since a.count is linear and so is del a[i].

while len(L) will always be true as long as L had something in it to begin with
Modifying L while using it with the for loop can cause items to be skipped, so you have a bug for some inputs.
If you fix that problem, you shouldn't need the while loop.

As long as the items in a are hashable and you don't mind that the remaining items aren't in the same order when you started, you can create an intermediate set and replace the original contents in-place.
a[:] = set(a)

Linear merging for lists in Python

I'm working through Google's Python class exercises. One of the exercises is this:
Given two lists sorted in increasing order, create and return a merged list of all the elements in sorted order. You may modify the passed in lists. Ideally, the solution should work in "linear" time, making a single pass of both lists.
The solution I came up with was:
def linear_merge(list1, list2):
list1.extend(list2)
return sorted(list1)
It passed the the test function, but the solution given is this:
def linear_merge(list1, list2):
result = []
# Look at the two lists so long as both are non-empty.
# Take whichever element [0] is smaller.
while len(list1) and len(list2):
if list1[0] < list2[0]:
result.append(list1.pop(0))
else:
result.append(list2.pop(0))
# Now tack on what's left
result.extend(list1)
result.extend(list2)
return result
Included as part of the solution was this:
Note: the solution above is kind of cute, but unfortunately list.pop(0) is
not constant time with the standard python list implementation, so the
above is not strictly linear time. An alternate approach uses pop(-1) to
remove the endmost elements from each list, building a solution list which
is backwards. Then use reversed() to put the result back in the correct
order. That solution works in linear time, but is more ugly.
Why are these two solutions so different? Am I missing something, or are they being unnecessarily complicated?

They're encouraging you to think about the actual method (algorithm) of merging two sorted lists. Suppose you had two stacks of paper with names on them, each in alphabetical order, and you wanted to make one sorted stack from them. You wouldn't just lump them together and then sort that from scratch; that would be too much work. You'd make use of the fact that each pile is already sorted, so you can just take the one that comes first off of one pile or the other, and put them into a new stack.

As you noted, your solution works perfectly. So why the complexity? Well, for a start
Ideally, the solution should work in "linear" time, making a single
pass of both lists.
Well, you're not explicitly passing through any lists, but you are calling sorted(). So how many times will sorted() pass over the lists?
Well, I don't actually know. Normally, a sorting algorithm would operate in something like O(n*log(n)) time, though look at this quote from the Python docs:
The Timsort algorithm used in Python does multiple sorts efficiently
because it can take advantage of any ordering already present in a
dataset.
Maybe someone who knows timsort better can figure it out.
But what they're doing in the solution, is using the fact that they know they have 2 sorted lists. So rather than starting from "scratch" with sorted, they're picking off elements 1 by 1.

I like the #Abhijit approach the most. Here is a slightly more pythonic/readable version of his code snippet:
def linear_merge(list1, list2):
result = []
while list1 and list2:
result.append((list1 if list1[-1] > list2[-1] else list2).pop(-1))
return (result + list1 + list2)[-1::-1]
With the help of the built-in python features, we:
don't need to explicitly check if the lists are empty with the
len function.
can merge/append empty lists and the result will remain unchanged, so no need for explicit checking.
we can combine multiple statements (if the readability allows), which sometimes makes the code more compact.

result = []
while list1 and list2:
result.append((list1 if list1[-1] > list2[-1] else list2).pop(-1))
if len(list1):
result += list1[-1::-1]
if len(list2):
result += list2[-1::-1]
return result[-1::-1]
The solution by #Abhijit and #intel do not work in all cases because they have not reversed the leftover parts of the original lists. If we have list1 = [1, 2, 3, 5, 9, 11, 13, 17] and list2 = [6, 7, 12, 15] then their solution would give [5, 3, 2, 1, 6, 7, 9, 11, 12, 13, 15, 17] where we would want [1, 2, 3, 5, 6, 7, 9, 11, 12, 13, 15, 17].

Your solution is O(n log n), which means that if your lists were 10 times as long, the program would take (roughly) 30 times as much time. Their solution would only take 10 times as long.

Pop off the end of the lists until one is empty. I think this is linear, and also the reverses are linear too. Ugly, but a solution.
def linear_merge(list1, list2):
# NOT return sorted (list1 + list2), as this is not linear
list3 = []
rem = []
empty = False
while not empty:
# Get last items from each list, if they exist
if len (list1) > 0:
a = list1[-1]
else:
rem = list2[:]
empty = True
if len (list2) > 0:
b = list2[-1]
else:
rem = list1[:]
empty = True
# Pop the one that's largest onto the new list
if not empty:
if a > b:
list3.append (a)
list1.pop ()
else:
list3.append (b)
list2.pop ()
# add the (reversed) remainder to the list
rem.reverse ()
list3 += rem
# reverse the entire list
list3.reverse ()
return list3

A slightly refined by still ugly solution (in Python3.5):
def linear_merge(list1: list, list2: list):
result = []
while len(list1) and len(list2):
result.append((list1 if list1[-1] > list2[-1] else list2).pop(-1))
result += list1 if len(list1) else list2
return result[-1::-1]

def linear_merge(list1, list2):
a= list1 + list2
a.sort()
return a

python: how to know the index when you randomly select an element from a sequence with random.choice(seq)

I know very well how to select a random item from a list with random.choice(seq) but how do I know the index of that element?

import random
l = ['a','b','c','d','e']
i = random.choice(range(len(l)))
print i, l[i]

You could first choose a random index, then get the list element at that location to have both the index and value.
>>> import random
>>> a = [1, 2, 3, 4, 5]
>>> index = random.randint(0,len(a)-1)
>>> index
0
>>> a[index]
1

You can do it using randrange function from random module
import random
l = ['a','b','c','d','e']
i = random.randrange(len(l))
print i, l[i]

The most elegant way to do so is random.randrange:
index = random.randrange(len(MY_LIST))
value = MY_LIST[index]
One can also do this in python3, less elegantly (but still better than .index) with random.choice on a range object:
index = random.choice(range(len(MY_LIST)))
value = MY_LIST[index]
The only valid solutions are this solution and the random.randint solutions.
The ones which use list.index not only are slow (O(N) per lookup rather than O(1); gets really bad if you do this for each element, you'll have to do O(N^2) comparisons) but ALSO you will have skewed/incorrect results if the list elements are not unique.
One would think that this is slow, but it turns out to only be slightly slower than the other correct solution random.randint, and may be more readable. I personally consider it more elegant because one doesn't have to do numerical index fiddling and use unnecessary parameters as one has to do with randint(0,len(...)-1), but some may consider this a feature, though one needs to know the randint convention of an inclusive range [start, stop].
Proof of speed for random.choice: The only reason this works is that the range object is OPTIMIZED for indexing. As proof, you can do random.choice(range(10**12)); if it iterated through the entire list your machine would be slowed to a crawl.
edit: I had overlooked randrange because the docs seemed to say "don't use this function" (but actually meant "this function is pythonic, use it"). Thanks to martineau for pointing this out.
You could of course abstract this into a function:
def randomElement(sequence):
index = random.randrange(len(sequence))
return index,sequence[index]
i,value = randomElement(range(10**15)) # try THAT with .index, heh
# (don't, your machine will die)
# use xrange if using python2
# i,value = (268840440712786, 268840440712786)

If the values are unique in the sequence, you can always say: list.index(value)

Using randrage() as has been suggested is a great way to get the index. By creating a dictionary created via comprehension you can reduce this code to one line as shown below. Note that since this dictionary only has one element, when you call popitem() you get the combined index and value in a tuple.
import random
letters = "abcdefghijklmnopqrstuvwxyz"
# dictionary created via comprehension
idx, val = {i: letters[i] for i in [random.randrange(len(letters))]}.popitem()
print("index {} value {}" .format(idx, val))

We can use sample() method also.
If you want to randomly select n elements from list
import random
l, n = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 2
index_list = random.sample(range(len(l)), n)
index_list will have unique indexes.
I prefer sample() over choices() as sample() does not allow duplicate elements in a sequence.