I'm trying to convert this working nested forloop into a single line list comprehension & i cannot seem to get it to work. The pseudo-code is as follows:
result = []
for x in something:
for y in x.address:
m = re.search("some pattern to match",y)
if m:
result += [m.group(1)]
Any pointers on how do i go about this ?
You'll need a generator expression..
matches = ( re.search(r'some pattern to match', y) for x in something
for y in x.address )
result = [ m.group(1) for m in matches if m ]
Nested loops are not really a problem for list comprehensions, as you can nest those there too:
lst = []
for y in z:
for x in y:
lst.append(f(x))
This translates into the following list comprehension:
[f(x) for y in z for x in y]
And you can easily continue that for multiple levels.
Conditions that decide on whether you want to add something to the list or not also work just fine:
lst = []
for x in y:
if t(x):
lst.append(f(x))
This translated into the following list comprehension with a filter:
[f(x) for x in y if t(x)]
Of course you can also combine that with multiple levels.
Now what is some kind of a problem though is when you want to execute something first, then filter on the result of that and append also something that depends on the result. The naive solution would be to move the function call inside and do it twice:
rexpr = re.compile('some pattern to match')
[rexpr.search(y).group(1) for x in something for y in x.address if rexpr.search(y)]
But this obviously runs the search twice which you generally want to avoid. At this point, you could use some hackish solutions which I generally wouldn’t recommend (as they harm readability). Since your result only depends on the result of the regular expression search, you could also solve this in two steps: First, you search on every element and map them to a match object, and then you filter on those matches and just return the valid ones:
[m.group(1) for m in (rexpr.search(y) for x in something for y in x.address) if m]
Note that I’m using generator expressions here: Those are essentially the same as list comprehensions, but don’t create the full result as a list but only yield on element at a time. So it’s more efficient if you only want to consume this one by one (which is the case here). After all, you’re only interested in the result from the list comprehension, so the comprehension will consume the generator expression.
I would do something like this:
# match function
def match(x):
m = re.search("some pattern to match",x)
if m:
return m.group(1)
else:
return None
#list comprehension
results = [match(y) for x in something for y in x.address if match(y)]
Related
RTX_number = [int(x) for x in input().split()]
Could someone explain this line of code to me step by step?
I am having great difficulty understanding it.
As far as I know, .split creates spaces between elements?
I saw this code on a forum and I am trying to get a better understanding of it because I think it might be helpful for a simulation project.
I heard this is called a list comprehension, but I am kind of lost as of for now.
input().split()
Reads a line and breaks it where ever there is a space into a list of strings.
for x in input().split()
Takes this list, runs over it item by item, and binds this item to x.
int(x) for ...
Takes this x we bound, and runs int(x) on it and returns it.
[int(x) for x in input().split()]
Takes all these results and puts them into a list.
The short version is that this:
RTX_number = [int(x) for x in input().split()]
is a short-form of this:
RTX_number = []
for x in input().split():
RTX_number.append(int(x))
where input().split() returns the list of strings that you get from separating whatever input() returned on each whitespace (for example, "Hello World" becomes ["Hello", "World"].
The str.split() function can also be given an argument, such as ',', which it will split on instead of whitespace.
The general syntax of a comprehension is
(expression) for (element) in (iterable) if (condition)
For every element element in the iterable, if the condition resolves to True (note that the condition can be omitted entirely) the expression is evaluated and added to the returned list.
We usually use comprehensions as shorthand for full loops, since they often save space and complexity.
Note that list comprehensions aren't the only kind of comprehension - they can be used to make several different data structures:
# list comprehension - produces a list
[expression for element in iterable]
# set comprehension - produces a set instead of a list
{expression for element in iterable}
# dict comprehension - produces a dict with key-value pairs
{key:value for element in iterable}
# generator comprehension - like a list comprehension, but each expression is not
# actually evaluated until something tries to read it.
# The parentheses are technically optional, if the syntax isn't ambiguous
(expression for element in iterable)
This code is equivalent to:
# ask user input (it expected something like "1 2 34 5")
input_text = input()
# split on spaces
split_text_list = input_text.split()
list_of_integers = []
# for each string item in list
for item in split_text_list:
# convert to integer
number = int(item)
# add to list
list_of_integers.append(number)
But of course it avoids having all the unnecessary intermediate variables, so it is shorter. Also faster as it doesn't require to store the intermediate values.
def anagramwordchecker(z,w):
if sorted([x for x in w])==sorted([x for x in z]):return True
return False
def anagramlistchecker(l,w):
d={}
for x in w:
d.update({x:w.count(x)})
l=list(filter(lambda x:anagramwordchecker(x,w),l))
return l
print(anagramlistchecker(['bcda', 'abce', 'cbda', 'cbea', 'adcb'],'abcd'))
trying to check which words are anagram.
using both of this it will print the same:
l=[x for x in l if anagramwordchecker(x,w)]
l=list(filter(lambda x:anagramwordchecker(x,w),l))
and it will be:
['bcda', 'cbda', 'adcb']
then what's the difference? any advantage using filter? cause comprehension is easier.
If you print the results of the following example, you will know which one is faster (Comments are results I got).
timeit.Timer('''[x for x in range(100) if x % 2 == 0]''' ).timeit(number=100000)
timeit.Timer('''list(filter(lambda x: x % 2 == 0, range(100)))''').timeit(number=100000)
# 0.3664856200000486
# 0.6642515319999802
So in your case, list comprehension would be faster. But let's see the following example.
timeit.Timer('''[x for x in range(100) if x % 2 == 0]''' ).timeit(number=100000)
timeit.Timer('''(x for x in range(100) if x % 2 == 0)''' ).timeit(number=100000)
timeit.Timer('''filter(lambda x: x % 2 == 0, range(100))''').timeit(number=100000)
# 0.5541256509999357
# 0.024836917000016
# 0.017953075000036733
The results show that casting an iterable to list takes much time and filter is faster than generator expression. So if your result does not really have to be a list, returning an iterable in a timely manner would be better.
As stated in here,
Note that filter(function, iterable) is equivalent to the generator expression (item for item in iterable if function(item)) if function is not None and (item for item in iterable if item) if function is None.
But list comprehension can do much more than simply filtering. If filter is given to the interpreter, it will knows it is a filter function. However, if a list comprehension is given to the interpreter, the interpreter does not know what it really is. After taking some time interpreting the list comprehension to something like a function, it would be a filter or filterfalse function in the end. Or, something else completely different.
filter with not condition can do what filterfalse does. But filterfalse is still there. Why? not operator does not need to be applied.
There is no magic. Human-friendly 1-for-many grammars are based on encapsulation. For them to be machine-executable binaries, they need to be decapsulated back and it takes time.
Go with a specific solution if it is enough than taking a more general solutions. Not only in coding, general solutions are usually for convenience, not for best results.
I am building a function to extract all negatives from a list called xs and I need it to add those extracted numbers into another list called new_home. I have come up with a code that I believe should work, however; it is only showing an empty list.
Example input/output:
xs=[1,2,3,4,0,-1,-2,-3,-4] ---> new_home=[1,2,3,4,0]
Here is my code that returns an empty list:
def extract_negatives(xs):
new_home=[]
for num in range(len(xs)):
if num <0:
new_home= new_home+ xs.pop(num)
return
return new_home
Why not use
[v for v in xs if v >= 0]
def extract_negatives(xs):
new_home=[]
for num in range(len(xs)):
if xs[num] < 0:
new_home.append(xs[num])
return new_home
for your code
But the Chuancong Gao solution is better:
def extract_negative(xs):
return [v for v in xs if v >= 0]
helper function filter could also help. Your function actually is
new_home = filter(lambda x: x>=0, xs)
Inside the loop of your code, the num variable doesn't really store the value of the list as you expect. The loop just iterates for len(xs) times and passes the current iteration number to num variable.
To access the list elements using loop, you should construct loop in a different fashion like this:
for element in list_name:
print element #prints all element.
To achieve your goal, you should do something like this:
another_list=[]
for element in list_name:
if(element<0): #only works for elements less than zero
another_list.append(element) #appends all negative element to another_list
Fortunately (or unfortunately, depending on how you look at it) you aren't examining the numbers in the list (xs[num]), you are examining the indexes (num). This in turn is because as a Python beginner you probably nobody haven't yet learned that there are typically easier ways to iterate over lists in Python.
This is a good (or bad, depending on how you look at it) thing, because had your code taken that branch you would have seen an exception occurring when you attempted to add a number to a list - though I agree the way you attempt it seems natural in English. Lists have an append method to put new elements o the end, and + is reserved for adding two lists together.
Fortunately ignorance is curable. I've recast your code a bit to show you how you might have written it:
def extract_negatives(xs):
out_list = []
for elmt in xs:
if elmt < 0:
out_list.append(elmt)
return out_list
As #ChuangongGoa suggests with his rather terse but correct answer, a list comprehension such as he uses is a much better way to perform simple operations of this type.
This question already has answers here:
Explanation of how nested list comprehension works?
(11 answers)
Closed 6 years ago.
Can someone please explain the meaning the syntax behind the following line of code:
temp3 = [x for x in temp1 if x not in s]
I understand it's for finding the differences between 2 lists, but what does the 'x' represent here? Each individual element in the list that is being compared? I understand that temp1 and s are lists. Also, does x for x have to have the same variable or could it be x for y?
[x for x in temp1 if x not in s]
It may help to re-order it slightly, so you can read the whole thing left to right. Let's move the first x to the end.
[for x in temp1 if x not in s yield x]
I've added a fake yield keyword so it reads naturally as English. If we then add some colons it becomes even clearer.
[for x in temp1: if x not in s: yield x]
Really, this is the order that things get evaluated in. The x variable comes from the for loop, that's why you can refer to it in the if and yield clauses. But the way list comprehensions are written is to put the value being yielding at the front. So you end up using a variable name that's not yet defined.
In fact, this final rewrite is exactly how you'd write an explicit generator function.
def func(temp1, s):
for x in temp1:
if x not in s:
yield x
If you call func(temp1, s) you get a generator equivalent to the list. You could turn it into that list with list(func(temp1, s)).
It iterates through each element in temp1 and checks to see if it is not in s before including it in temp3.
It is a shorter and more pythonic way of writing
temp3 = []
for item in temp1:
if item not in s:
temp3.append(item)
Where temp1 and s are the two lists you are comparing.
As for your second question, x for y will work, but probably not in the way you intend to, and certainly not in a very useful way. It will assign each item in temp1 to the variable name y, and then search for x in the scope outside of the list comprehension. Assuming x is defined previously (otherwise you will get NameError or something similar), the condition if x not in s will evaluate to the same thing for every item in temp1, which is why it’s not terribly useful. And if that condition is true, your resulting temp3 will be populated with xs; the y values are unused.
Do not take this as saying that using different variables in a list comprehension is never useful. In fact list comprehensions like [a if condition(x) else b for x in original_sequence] are often very useful. A list comprehension like [a for x in original_sequence if condition(x)] can also be useful for constructing a list containing exactly as many instances of a as the number of items in original_sequence that satisfy condition().
Try yourself:
arr = [1,2,3]
[x+5 for x in arr]
This should give you [6, 7, 8] that are the values on the [1,2,3] list plus 5. This syntax is know as list comprehension (or mapping). It applies the same instructions to all elements on a list. Would be the same as doing this:
for x in arr:
arr += 5
X is same variable and it is not y. It works same as below code
newList = []
for x in temp1:
if x not in s:
newList.append(x)
So x for x, here first is x which is inside append in code and x after for is same as for x in temp1.
I have a function which looks like that:
def roulette(self):
sum = 0
lst = []
for x in self.drinkList:
sum += x.fitness
lst.append(sum)
return lst
Can it be replaced with list comprehension expression or something more efficient than for loop?
PS: it apperars that if I do random.randrange(0), it raises an exception ValueError: empty range for randrange(). Is there a way to avoid it without using if test?
It's actually possible to 'peek' at the list being built in a list comprehension. the outermost list has the name _[1], which of course is not a valid python identifier, so it must be accessed in another way:
def roulette(self):
return [drink.fitness + (locals()['_[1]'][-1] if locals()['_[1]'] else 0)
for drink
in self.drinkList]
But just because you can doesn't mean you should; go with your for loop, it looks like exactly what it does, and also doesn't rely on an undocumented python feature.
Your roulette function is computing the partial sums of the list
of x.fitness elements.
You can reach the same result by defining a closure and using map on
a generator expression.
sum = 0
def partial_sum(x):
sum += x
return sum
lst = map(partial_sum, (x.fitness for x in self.drinkList))
This is certainly less readable than a for loop; it could be faster
but you'll have to experiment: map is generally faster than for,
but function calls are slow. (Substituting a list comprehension for
the generator expression might speed things up at the expense of
memory.)
[sum(x.fitness for x in self.drinklist[:i+1]) for i in range(len(self.drinklist))]
But this would be O(n^2), while yours is O(n).
I don't know if it is more efficient, but this has a list comprehension and less lines:
def roulette(self):
lst = [self.drinkList.pop(0).fitness]
[ lst.append(x.fitness + lst[-1]) for x in self.drinkList]
return lst
.
Edit
As nothing authorizes me to modify the list self.drinkList , I rewrite:
def roulette(self):
lst = [0]
[ lst.append(x.fitness + lst[-1]) for x in self.drinkList]
return lst[1:]