We Keep Coding

what's the difference between filter and comprehention with if?

def anagramwordchecker(z,w):
if sorted([x for x in w])==sorted([x for x in z]):return True
return False
def anagramlistchecker(l,w):
d={}
for x in w:
d.update({x:w.count(x)})
l=list(filter(lambda x:anagramwordchecker(x,w),l))
return l
print(anagramlistchecker(['bcda', 'abce', 'cbda', 'cbea', 'adcb'],'abcd'))
trying to check which words are anagram.
using both of this it will print the same:
l=[x for x in l if anagramwordchecker(x,w)]
l=list(filter(lambda x:anagramwordchecker(x,w),l))
and it will be:
['bcda', 'cbda', 'adcb']
then what's the difference? any advantage using filter? cause comprehension is easier.

If you print the results of the following example, you will know which one is faster (Comments are results I got).
timeit.Timer('''[x for x in range(100) if x % 2 == 0]''' ).timeit(number=100000)
timeit.Timer('''list(filter(lambda x: x % 2 == 0, range(100)))''').timeit(number=100000)
# 0.3664856200000486
# 0.6642515319999802
So in your case, list comprehension would be faster. But let's see the following example.
timeit.Timer('''[x for x in range(100) if x % 2 == 0]''' ).timeit(number=100000)
timeit.Timer('''(x for x in range(100) if x % 2 == 0)''' ).timeit(number=100000)
timeit.Timer('''filter(lambda x: x % 2 == 0, range(100))''').timeit(number=100000)
# 0.5541256509999357
# 0.024836917000016
# 0.017953075000036733
The results show that casting an iterable to list takes much time and filter is faster than generator expression. So if your result does not really have to be a list, returning an iterable in a timely manner would be better.
As stated in here,
Note that filter(function, iterable) is equivalent to the generator expression (item for item in iterable if function(item)) if function is not None and (item for item in iterable if item) if function is None.
But list comprehension can do much more than simply filtering. If filter is given to the interpreter, it will knows it is a filter function. However, if a list comprehension is given to the interpreter, the interpreter does not know what it really is. After taking some time interpreting the list comprehension to something like a function, it would be a filter or filterfalse function in the end. Or, something else completely different.
filter with not condition can do what filterfalse does. But filterfalse is still there. Why? not operator does not need to be applied.
There is no magic. Human-friendly 1-for-many grammars are based on encapsulation. For them to be machine-executable binaries, they need to be decapsulated back and it takes time.
Go with a specific solution if it is enough than taking a more general solutions. Not only in coding, general solutions are usually for convenience, not for best results.

What is the most pythonic way to apply function to some items of a list?

Let's take this code that prints all positive integer of a list:
l = [1, -1, 1, 0, 2]
for i in l:
if i > 0:
print(i)
I can do it with a list comprehension but I guess this has the disadvantage to create a new useless list:
[print(i) for i in l if i > 0]
So my question: is there a more pythonic way to write this?

The most Pythonic way to apply a function to some subset of a list of elements is to use a for loop, just as you already have.
Within that for loop, there is an argument to be made for filtering the list before assigning any value to i; whether that constitutes an improvement is usually a matter of opinion and will depend on the context.
for i in filter(lambda x: x > 0, l):
print(i)
In this case, I think it's worse. But sometimes you have a predicate at hand, and filtering can be syntactically lighter. Compare
for i in some_list_of_strings:
if i.isdigit():
print(i)
with
for i in filter(str.isdigit, some_list_of_strings):
print(i)

The plain for-loop is perfectly Pythonic. You want to loop over the elements of the list, select the ones that are greater than zero, and print them, and that's exactly what it does - no more, no less.
The list comprehension is not Pythonic, mostly for the reason you gave: it creates a new useless list. Even if you were going to use the list, using a list comprehension for side effects is still bad practice.

I suppose that theoretically you could contrive to use a generator comprehension in order to avoid creating a large list in memory:
for _ in (print(i) for i in l if i > 0): pass
(maybe also using some function that does the consuming of values from the generator, so that any loop is hidden away inside that function).
However, not only is this less readable than the explicit for loop, the plain for loop is also quicker.
import time
l = [1, -1, 1, 0, 2]
# we don't really want lots of output for this timing test
def dont_really_print(i):
return i
t1 = time.time()
for x in range(1000000):
for i in l:
if i > 0:
dont_really_print(i)
t2 = time.time()
for x in range(1000000):
for _ in (dont_really_print(i) for i in l if i > 0):
pass
t3 = time.time()
print(f"generator comprehension {t3 - t2:.3f} "
f"explicit loop {t2 - t1:.3f}")
gives:
generator comprehension 0.629 explicit loop 0.423

Generally, whether Python or any other language, what you want to do is known as a "map."
Print is a weird function to map, so here's another function for better demonstration purposes.
def add_one(num):
return num + 1
foo = [1, 2, 3, 4]
new_list = map(add_one, foo)
print(list(new_list))
You can typically map in parallel, sometimes asynchronously, and a good number of parallel processing paradigms (including Python's multiprocessing) use map as a fundamental operation for using multiple cores to implement a function.

How to return a list that is made up of extracted elements from another list in python?

I am building a function to extract all negatives from a list called xs and I need it to add those extracted numbers into another list called new_home. I have come up with a code that I believe should work, however; it is only showing an empty list.
Example input/output:
xs=[1,2,3,4,0,-1,-2,-3,-4] ---> new_home=[1,2,3,4,0]
Here is my code that returns an empty list:
def extract_negatives(xs):
new_home=[]
for num in range(len(xs)):
if num <0:
new_home= new_home+ xs.pop(num)
return
return new_home

Why not use
[v for v in xs if v >= 0]

def extract_negatives(xs):
new_home=[]
for num in range(len(xs)):
if xs[num] < 0:
new_home.append(xs[num])
return new_home
for your code
But the Chuancong Gao solution is better:
def extract_negative(xs):
return [v for v in xs if v >= 0]

helper function filter could also help. Your function actually is
new_home = filter(lambda x: x>=0, xs)

Inside the loop of your code, the num variable doesn't really store the value of the list as you expect. The loop just iterates for len(xs) times and passes the current iteration number to num variable.
To access the list elements using loop, you should construct loop in a different fashion like this:
for element in list_name:
print element #prints all element.
To achieve your goal, you should do something like this:
another_list=[]
for element in list_name:
if(element<0): #only works for elements less than zero
another_list.append(element) #appends all negative element to another_list

Fortunately (or unfortunately, depending on how you look at it) you aren't examining the numbers in the list (xs[num]), you are examining the indexes (num). This in turn is because as a Python beginner you probably nobody haven't yet learned that there are typically easier ways to iterate over lists in Python.
This is a good (or bad, depending on how you look at it) thing, because had your code taken that branch you would have seen an exception occurring when you attempted to add a number to a list - though I agree the way you attempt it seems natural in English. Lists have an append method to put new elements o the end, and + is reserved for adding two lists together.
Fortunately ignorance is curable. I've recast your code a bit to show you how you might have written it:
def extract_negatives(xs):
out_list = []
for elmt in xs:
if elmt < 0:
out_list.append(elmt)
return out_list
As #ChuangongGoa suggests with his rather terse but correct answer, a list comprehension such as he uses is a much better way to perform simple operations of this type.

for-in loop's upper limit changing in each loop

How can I update the upper limit of a loop in each iteration? In the following code, List is shortened in each loop. However, the lenList in the for, in loop is not, even though I defined lenList as global. Any ideas how to solve this? (I'm using Python 2.sthg)
Thanks!
def similarity(List):
import difflib
lenList = len(List)
for i in range(1,lenList):
import numpy as np
global lenList
a = List[i]
idx = [difflib.SequenceMatcher(None, a, x).ratio() for x in List]
z = idx > .9
del List[z]
lenList = len(List)
X = ['jim','jimmy','luke','john','jake','matt','steve','tj','pat','chad','don']
similarity(X)

Looping over indices is bad practice in python. You may be able to accomplish what you want like this though (edited for comments):
def similarity(alist):
position = 0
while position < len(alist):
item = alist[position]
position += 1
# code here that modifies alist
A list will evaluate True if it has any entries, or False when it is empty. In this way you can consume a list that may grow during the manipulation of its items.
Additionally, if you absolutely have to have indices, you can get those as well:
for idx, item in enumerate(alist):
# code here, where items are actual list entries, and
# idx is the 0-based index of the item in the list.
In ... 3.x (I believe) you can even pass an optional parameter to enumerate to control the starting value of idx.

The issue here is that range() is only evaluated once at the start of the loop and produces a range generator (or list in 2.x) at that time. You can't then change the range. Not to mention that numbers and immutable, so you are assigning a new value to lenList, but that wouldn't affect any uses of it.
The best solution is to change the way your algorithm works not to rely on this behaviour.

The range is an object which is constructed before the first iteration of your loop, so you are iterating over the values in that object. You would instead need to use a while loop, although as Lattyware and g.d.d.c point out, it would not be very Pythonic.

What you are effectively looping on in the above code is a list which got generated in the first iteration itself.
You could have as well written the above as
li = range(1,lenList)
for i in li:
... your code ...
Changing lenList after li has been created has no effect on li

This problem will become quite a lot easier with one small modification to how your function works: instead of removing similar items from the existing list, create and return a new one with those items omitted.
For the specific case of just removing similarities to the first item, this simplifies down quite a bit, and removes the need to involve Numpy's fancy indexing (which you weren't actually using anyway, because of a missing call to np.array):
import difflib
def similarity(lst):
a = lst[0]
return [a] + \
[x for x in lst[1:] if difflib.SequenceMatcher(None, a, x).ratio() > .9]
From this basis, repeating it for every item in the list can be done recursively - you need to pass the list comprehension at the end back into similarity, and deal with receiving an empty list:
def similarity(lst):
if not lst:
return []
a = lst[0]
return [a] + similarity(
[x for x in lst[1:] if difflib.SequenceMatcher(None, a, x).ratio() > .9])
Also note that importing inside a function, and naming a variable list (shadowing the built-in list) are both practices worth avoiding, since they can make your code harder to follow.

python : list index out of range error while iteratively popping elements

I have written a simple python program
l=[1,2,3,0,0,1]
for i in range(0,len(l)):
if l[i]==0:
l.pop(i)
This gives me error 'list index out of range' on line if l[i]==0:
After debugging I could figure out that i is getting incremented and list is getting reduced.
However, I have loop termination condition i < len(l). Then why I am getting such error?

You are reducing the length of your list l as you iterate over it, so as you approach the end of your indices in the range statement, some of those indices are no longer valid.
It looks like what you want to do is:
l = [x for x in l if x != 0]
which will return a copy of l without any of the elements that were zero (that operation is called a list comprehension, by the way). You could even shorten that last part to just if x, since non-zero numbers evaluate to True.
There is no such thing as a loop termination condition of i < len(l), in the way you've written the code, because len(l) is precalculated before the loop, not re-evaluated on each iteration. You could write it in such a way, however:
i = 0
while i < len(l):
if l[i] == 0:
l.pop(i)
else:
i += 1

The expression len(l) is evaluated only one time, at the moment the range() builtin is evaluated. The range object constructed at that time does not change; it can't possibly know anything about the object l.
P.S. l is a lousy name for a value! It looks like the numeral 1, or the capital letter I.

You're changing the size of the list while iterating over it, which is probably not what you want and is the cause of your error.
Edit: As others have answered and commented, list comprehensions are better as a first choice and especially so in response to this question. I offered this as an alternative for that reason, and while not the best answer, it still solves the problem.
So on that note, you could also use filter, which allows you to call a function to evaluate the items in the list you don't want.
Example:
>>> l = [1,2,3,0,0,1]
>>> filter(lambda x: x > 0, l)
[1, 2, 3]
Live and learn. Simple is better, except when you need things to be complex.

What Mark Rushakoff said is true, but if you iterate in the opposite direction, it is possible to remove elements from the list in the for-loop as well. E.g.,
x = [1,2,3,0,0,1]
for i in range(len(x)-1, -1, -1):
if x[i] == 0:
x.pop(i)
It's like a tall building that falls from top to bottom: even if it is in the middle of collapse, you still can "enter" into it and visit yet-to-be-collapsed floors.

I think the best way to solve this problem is:
l = [1, 2, 3, 0, 0, 1]
while 0 in l:
l.remove(0)
Instead of iterating over list I remove 0 until there aren't any 0 in list

List comprehension will lead you to a solution.
But the right way to copy a object in python is using python module copy - Shallow and deep copy operations.
l=[1,2,3,0,0,1]
for i in range(0,len(l)):
if l[i]==0:
l.pop(i)
If instead of this,
import copy
l=[1,2,3,0,0,1]
duplicate_l = copy.copy(l)
for i in range(0,len(l)):
if l[i]==0:
m.remove(i)
l = m
Then, your own code would have worked.
But for optimization, list comprehension is a good solution.

The problem was that you attempted to modify the list you were referencing within the loop that used the list len(). When you remove the item from the list, then the new len() is calculated on the next loop.
For example, after the first run, when you removed (i) using l.pop(i), that happened successfully but on the next loop the length of the list has changed so all index numbers have been shifted. To a certain point the loop attempts to run over a shorted list throwing the error.
Doing this outside the loop works, however it would be better to build and new list by first declaring and empty list before the loop, and later within the loop append everything you want to keep to the new list.
For those of you who may have come to the same problem.

I am using python 3.3.5. The above solution of using while loop did not work for me. Even if i put print (i) after len(l) it gave me an error. I ran the same code in command line (shell)[ window that pops up when we run a function] it runs without error. What i did was calculated len(l) outside the function in main program and passed the length as a parameter. It worked. Python is weird sometimes.

I think most solutions talk here about List Comprehension, but if you'd like to perform in place deletion and keep the space complexity to O(1); The solution is:
i = 0
for j in range(len(arr)):
if (arr[j] != 0):
arr[i] = arr[j]
i +=1
arr = arr[:i]

x=[]
x = [int(i) for i in input().split()]
i = 0
while i < len(x):
print(x[i])
if(x[i]%5)==0:
del x[i]
else:
i += 1
print(*x)

Code:
while True:
n += 1
try:
DATA[n]['message']['text']
except:
key = DATA[n-1]['message']['text']
break
Console :
Traceback (most recent call last):
File "botnet.py", line 82, in <module>
key =DATA[n-1]['message']['text']
IndexError: list index out of range

I recently had a similar problem and I found that I need to decrease the list index by one.
So instead of:
if l[i]==0:
You can try:
if l[i-1]==0:
Because the list indices start at 0 and your range will go just one above that.