Related
when I create a list, I use the one-liner
new_list = [func(item) for item in somelist]
Is there a simple way to write the following iteration in one line?
new_list = [0]
for _ in range(N):
new_list.append(func(new_list[-1]))
or even
new_list = [0]
for t in range(N):
new_list.append(func(t, new_list[-1]))
i.e. each item is calculated based on the previous item with a specific initializer.
itertools.accumulate() exists exactly for that:
from itertools import accumulate
new_list = list(accumulate(range(N), func)) # == [0, func(0, 1), func(func(0, 1), 2), ...]
If you wish to dump the N, just use accumulate like so:
from itertools import accumulate
new_list = list(accumulate(range(N), lambda x, y: func(x))) # == [0, func(0), func(fnc(0)), ...]
You can use an assignment expression to store the returning value of the last call to func, but since you also want the initial value 0 to be in the list, you would have to join it separately:
new_list = [i := 0] + [i := func(i) for _ in range(N)]
or even:
new_list = [i := 0] + [i := func(t, i) for t in range(N)]
I have a list xs containing a mixture of strings and None values. How can I use a list comprehension to call a function on each string, but convert the None values to '' (rather than passing them to the function)?
I tried:
[f(x) for x in xs if x is not None else '']
but it gives a SyntaxError. What is the correct syntax?
See List comprehension with condition if you are trying to make a list comprehension that omits values based on a condition.
If you need to consider more than two conditional outcomes, beware that Python's conditional expressions do not support elif. Instead, it is necessary to nest if/else conditionals. See `elif` in list comprehension conditionals for details.
You can totally do that. It's just an ordering issue:
[f(x) if x is not None else '' for x in xs]
In general,
[f(x) if condition else g(x) for x in sequence]
And, for list comprehensions with if conditions only,
[f(x) for x in sequence if condition]
Note that this actually uses a different language construct, a conditional expression, which itself is not part of the comprehension syntax, while the if after the for…in is part of list comprehensions and used to filter elements from the source iterable.
Conditional expressions can be used in all kinds of situations where you want to choose between two expression values based on some condition. This does the same as the ternary operator ?: that exists in other languages. For example:
value = 123
print(value, 'is', 'even' if value % 2 == 0 else 'odd')
Let's use this question to review some concepts. I think it's good to first see the fundamentals so you can extrapolate to different cases.
Other answers provide the specific answer to your question. I'll first give some general context and then I'll answer the question.
Fundamentals
if/else statements in list comprehensions involve two things:
List comprehensions
Conditional expressions (Ternary operators)
1. List comprehensions
They provide a concise way to create lists.
Its structure consists of: "brackets containing an expression followed by a for clause, then zero or more for or if clauses".
Case 1
Here we have no condition. Each item from the iterable is added to new_list.
new_list = [expression for item in iterable]
new_list = [x for x in range(1, 10)]
> [1, 2, 3, 4, 5, 6, 7, 8, 9]
Case 2
Here we have one condition.
Example 1
Condition: only even numbers will be added to new_list.
new_list = [expression for item in iterable if condition == True]
new_list = [x for x in range(1, 10) if x % 2 == 0]
> [2, 4, 6, 8]
Example 2
Condition: only even numbers that are multiple of 3 will be added to new_list.
new_list = [expression for item in iterable if condition == True]
new_list = [x for x in range(1, 10) if x % 2 == 0 if x % 3 == 0]
> [6]
But howcome we have one condition if we use two if in new_list?
The prior expression could be written as:
new_list = [x for x in range(1, 10) if x % 2 and x % 3 == 0]
> [6]
We only use one if statement.
This is like doing:
new_list = []
for x in range(1, 10):
if x % 2 == 0 and x % 3 == 0:
new_list.append(x)
> [6]
Example 3
Just for the sake of argument, you can also use or.
Condition: even numbers or numbers multiple of 3 will be added to new_list.
new_list = [x for x in range(1, 10) if x % 2 == 0 or x % 3 == 0]
> [2, 3, 4, 6, 8, 9]
Case 3
More than one condition:
Here we need the help of conditional expressions (Ternary operators).
2.Conditional Expressions
What are conditional expressions? What the name says: a Python expression that has some condition.
<Exp1> if condition else <Exp2>
First the condition is evaluated. If condition is True, then <Exp1> is evaluated and returned. If condition is False, then <Exp2> is evaluated and returned.
A conditional expression with more than one condition:
<Exp1> if condition else <Exp2> if condition else <Exp3>...
An example from Real Python:
age = 12
s = 'minor' if age < 21 else 'adult'
> minor
The value of s is conditioned to age value.
3.List Comprehensions with Conditionals
We put list comprehensions and conditionals together like this.
new_list = [<Conditional Expression> for <item> in <iterable>]
new_list = [<Exp1> if condition else <Exp2> if condition else <Exp3> for <item> in <iterable>]
Condition: even numbers will be added as 'even', the number three will be added as 'number three' and the rest will be added as 'odd'.
new_list = ['even' if x % 2 == 0 else 'number three' if x == 3 else 'odd'
for x in range(1, 10)]
> ['odd', 'even', 'number three', 'even', 'odd', 'even', 'odd', 'even', 'odd']
The answer to the question
[f(x) for x in xs if x is not None else '']
Here we have a problem with the structure of the list: for x in xs should be at the end of the expression.
Correct way:
[f(x) if x is not None else '' for x in xs]
Further reading:
Does Python have a ternary conditional operator?
The specific problem has already been solved in previous answers, so I will address the general idea of using conditionals inside list comprehensions.
Here is an example that shows how conditionals can be written inside a list comprehension:
X = [1.5, 2.3, 4.4, 5.4, 'n', 1.5, 5.1, 'a'] # Original list
# Extract non-strings from X to new list
X_non_str = [el for el in X if not isinstance(el, str)] # When using only 'if', put 'for' in the beginning
# Change all strings in X to 'b', preserve everything else as is
X_str_changed = ['b' if isinstance(el, str) else el for el in X] # When using 'if' and 'else', put 'for' in the end
Note that in the first list comprehension for X_non_str, the order is:
expression for item in iterable if condition
and in the last list comprehension for X_str_changed, the order is:
expression1 if condition else expression2 for item in iterable
I always find it hard to remember that expression1 has to be before if and expression2 has to be after else. My head wants both to be either before or after.
I guess it is designed like that because it resembles normal language, e.g. "I want to stay inside if it rains, else I want to go outside"
In plain English the two types of list comprehensions mentioned above could be stated as:
With only if:
extract_apple for apple in apple_box if apple_is_ripe
and with if/else
mark_apple if apple_is_ripe else leave_it_unmarked for apple in apple_box
One way:
def change(x):
if x is None:
return f(x)
else:
return ''
result = [change(x) for x in xs]
Although then you have:
result = map(change, xs)
Or you can use a lambda inline.
Here is another illustrative example:
>>> print(", ".join(["ha" if i else "Ha" for i in range(3)]) + "!")
Ha, ha, ha!
It exploits the fact that if i evaluates to False for 0 and to True for all other values generated by the function range(). Therefore the list comprehension evaluates as follows:
>>> ["ha" if i else "Ha" for i in range(3)]
['Ha', 'ha', 'ha']
[f(x) if x != None else '' for x in xs]
Syntax for list comprehension:
[item if condition else item for item in items]
[f(item) if condition else value for item in items]
[item if condition for item in items]
[value if condition else value1 if condition1 else value2]
The other solutions are great for a single if / else construct. However, ternary statements within list comprehensions are arguably difficult to read.
Using a function aids readability, but such a solution is difficult to extend or adapt in a workflow where the mapping is an input. A dictionary can alleviate these concerns:
xs = [None, 'This', 'is', 'a', 'filler', 'test', 'string', None]
d = {None: '', 'filler': 'manipulated'}
res = [d.get(x, x) for x in xs]
print(res)
['', 'This', 'is', 'a', 'manipulated', 'test', 'string', '']
It has to do with how the list comprehension is performed.
Keep in mind the following:
[ expression for item in list if conditional ]
Is equivalent to:
for item in list:
if conditional:
expression
Where the expression is in a slightly different format (think switching the subject and verb order in a sentence).
Therefore, your code [x+1 for x in l if x >= 45] does this:
for x in l:
if x >= 45:
x+1
However, this code [x+1 if x >= 45 else x+5 for x in l] does this (after rearranging the expression):
for x in l:
if x>=45: x+1
else: x+5
Make a list from items in an iterable
It seems best to first generalize all the possible forms rather than giving specific answers to questions. Otherwise, the reader won't know how the answer was determined. Here are a few generalized forms I thought up before I got a headache trying to decide if a final else' clause could be used in the last form.
[expression1(item) for item in iterable]
[expression1(item) if conditional1 for item in iterable]
[expression1(item) if conditional1 else expression2(item) for item in iterable]
[expression1(item) if conditional1 else expression2(item) for item in iterable if conditional2]
The value of item doesn't need to be used in any of the conditional clauses. A conditional3 can be used as a switch to either add or not add a value to the output list.
For example, to create a new list that eliminates empty strings or whitespace strings from the original list of strings:
newlist = [s for s in firstlist if s.strip()]
There isn't any need for ternary if/then/else. In my opinion your question calls for this answer:
row = [unicode((x or '').strip()) for x in row]
You can combine conditional logic in a comprehension:
ps = PorterStemmer()
stop_words_english = stopwords.words('english')
best = sorted(word_scores.items(), key=lambda x: x[1], reverse=True)[:10000]
bestwords = set([w for w, s in best])
def best_word_feats(words):
return dict([(word, True) for word in words if word in bestwords])
# with stemmer
def best_word_feats_stem(words):
return dict([(ps.stem(word), True) for word in words if word in bestwords])
# with stemmer and not stopwords
def best_word_feats_stem_stop(words):
return dict([(ps.stem(word), True) for word in words if word in bestwords and word not in stop_words_english])
# coding=utf-8
def my_function_get_list():
my_list = [0, 1, 2, 3, 4, 5]
# You may use map() to convert each item in the list to a string,
# and then join them to print my_list
print("Affichage de my_list [{0}]".format(', '.join(map(str, my_list))))
return my_list
my_result_list = [
(
number_in_my_list + 4, # Condition is False : append number_in_my_list + 4 in my_result_list
number_in_my_list * 2 # Condition is True : append number_in_my_list * 2 in my_result_list
)
[number_in_my_list % 2 == 0] # [Condition] If the number in my list is even
for number_in_my_list in my_function_get_list() # For each number in my list
]
print("Affichage de my_result_list [{0}]".format(', '.join(map(str, my_result_list))))
(venv) $ python list_comp.py
Affichage de my_list [0, 1, 2, 3, 4, 5]
Affichage de my_result_list [0, 5, 4, 7, 8, 9]
So, for you:
row = [('', unicode(x.strip()))[x is not None] for x in row]
I am new to Python and wanted to explore it's pseudo-code like syntax to solve the following problem:
# x is 0, 1 or 2
arr = [0, 1, 2]
I want to return any element in arr that is not equal to x
My intuition:
return x if x != element for element in arr
I have tried to complete the conditional with an else clause. Still, the syntax is invalid
What is my mistake? What is a correct one-line solution (if any exists)
Thanks!
return [element for element in arr if element != x]
https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions
If you want to return one element from your list matching a certain condition (in this case the condition is !=x), you can use next.
return next(item for item in arr if item!=x)
You're pretty close. Just put the if last. This will return a generator:
def f(ls, e):
return (x for x in ls if x != e)
You can also return a list instead:
def f(ls, e):
return [x for x in ls if x != e]
You can use ternary if statement as given below:
def f(x,arr):
return x if x in arr else None
Or you can use list comprehension if you want return all elements contains in arr, as you can see below:
def f(x,arr):
return [y for y in arr if y in x] #here x is a list
I would use a list comprehension:
return [a for a in arr if a != x]
I'm currently writing a little script in python (2.x), and there's a portion of the code that I'd like to improve without knowing how to do so.
I have a list of lists that looks like the following:
my_list = [["abc",1,2,"def"],["ghi",4,5,"klm"],["nop",6,7,"qrs"]]
I need to get the sum of all the integers at the index 1 and the sum of all the integers at the index 2. To do so, I currently have:
sum1, sum2 = 0, 0
for i in my_list:
sum1 += i[1]
sum2 += i[2]
What could be a more pythonic way to do that? Maybe using reduce and a lambda function or something?
A more pythonic way to do that would be using the sum function along with the for ... in ... generator and do all the work in a single line, like this:
sum1, sum2 = sum(x[1] for x in my_list), sum(x[2] for x in my_list)
The most Pythonic would probably be list comprehensions:
my_list = [["abc",1,2,"def"],["ghi",4,5,"klm"],["nop",6,7,"qrs"]]
Summations:
sum1 = sum(l[1] for l in my_list)
sum2 = sum(l[2] for l in my_list)
Which returns:
sum1 = 11
sum2 = 14
You could do sum(x[1] for x in in my_list), sum(x[2] for x in my_list) if you don't mind looping twice.
or reduce(lambda acc, l: (acc[0] + l[1], acc[1] + l[2]), my_list, (0, 0)) if you want to do it both at once. This will return a tuple with the sum of [1] on the first element, and [2] on the second
You can use zip and list comprehension
>>> lst = [["abc",1,2,"def"],["ghi",4,5,"klm"],["nop",6,7,"qrs"]]
>>> [sum(i) for i in list(zip(*lst))[1:3]]
[11, 14]
Or use zip and islice class from itertools
>>> from itertools import islice
>>> [sum(i) for i in islice(zip(*lst), 1, 3)]
[11, 14]
Here's a one liner that doesn't use a generator expression. Use zip plus unpacking to transpose the list, then run sum on all the numeric columns using map.
>>> map(sum, zip(*my_list)[1:-1])
[11, 14]
Unfortunately it's a little wordier in 3.X since you can't slice a zip object.
a,b = map(sum, list(zip(*my_list))[1:-1])
You can use reduce but, since the elements in your sequence are lists, you'll need to set an initial value of 0. In Example:
reduce(lambda total, list: total+list[1], my_list, 0) # integers at index 1
reduce(lambda total, list: total+list[2], my_list, 0) # integers at index 2
I have a list xs containing a mixture of strings and None values. How can I use a list comprehension to call a function on each string, but convert the None values to '' (rather than passing them to the function)?
I tried:
[f(x) for x in xs if x is not None else '']
but it gives a SyntaxError. What is the correct syntax?
See List comprehension with condition if you are trying to make a list comprehension that omits values based on a condition.
If you need to consider more than two conditional outcomes, beware that Python's conditional expressions do not support elif. Instead, it is necessary to nest if/else conditionals. See `elif` in list comprehension conditionals for details.
You can totally do that. It's just an ordering issue:
[f(x) if x is not None else '' for x in xs]
In general,
[f(x) if condition else g(x) for x in sequence]
And, for list comprehensions with if conditions only,
[f(x) for x in sequence if condition]
Note that this actually uses a different language construct, a conditional expression, which itself is not part of the comprehension syntax, while the if after the for…in is part of list comprehensions and used to filter elements from the source iterable.
Conditional expressions can be used in all kinds of situations where you want to choose between two expression values based on some condition. This does the same as the ternary operator ?: that exists in other languages. For example:
value = 123
print(value, 'is', 'even' if value % 2 == 0 else 'odd')
Let's use this question to review some concepts. I think it's good to first see the fundamentals so you can extrapolate to different cases.
Other answers provide the specific answer to your question. I'll first give some general context and then I'll answer the question.
Fundamentals
if/else statements in list comprehensions involve two things:
List comprehensions
Conditional expressions (Ternary operators)
1. List comprehensions
They provide a concise way to create lists.
Its structure consists of: "brackets containing an expression followed by a for clause, then zero or more for or if clauses".
Case 1
Here we have no condition. Each item from the iterable is added to new_list.
new_list = [expression for item in iterable]
new_list = [x for x in range(1, 10)]
> [1, 2, 3, 4, 5, 6, 7, 8, 9]
Case 2
Here we have one condition.
Example 1
Condition: only even numbers will be added to new_list.
new_list = [expression for item in iterable if condition == True]
new_list = [x for x in range(1, 10) if x % 2 == 0]
> [2, 4, 6, 8]
Example 2
Condition: only even numbers that are multiple of 3 will be added to new_list.
new_list = [expression for item in iterable if condition == True]
new_list = [x for x in range(1, 10) if x % 2 == 0 if x % 3 == 0]
> [6]
But howcome we have one condition if we use two if in new_list?
The prior expression could be written as:
new_list = [x for x in range(1, 10) if x % 2 and x % 3 == 0]
> [6]
We only use one if statement.
This is like doing:
new_list = []
for x in range(1, 10):
if x % 2 == 0 and x % 3 == 0:
new_list.append(x)
> [6]
Example 3
Just for the sake of argument, you can also use or.
Condition: even numbers or numbers multiple of 3 will be added to new_list.
new_list = [x for x in range(1, 10) if x % 2 == 0 or x % 3 == 0]
> [2, 3, 4, 6, 8, 9]
Case 3
More than one condition:
Here we need the help of conditional expressions (Ternary operators).
2.Conditional Expressions
What are conditional expressions? What the name says: a Python expression that has some condition.
<Exp1> if condition else <Exp2>
First the condition is evaluated. If condition is True, then <Exp1> is evaluated and returned. If condition is False, then <Exp2> is evaluated and returned.
A conditional expression with more than one condition:
<Exp1> if condition else <Exp2> if condition else <Exp3>...
An example from Real Python:
age = 12
s = 'minor' if age < 21 else 'adult'
> minor
The value of s is conditioned to age value.
3.List Comprehensions with Conditionals
We put list comprehensions and conditionals together like this.
new_list = [<Conditional Expression> for <item> in <iterable>]
new_list = [<Exp1> if condition else <Exp2> if condition else <Exp3> for <item> in <iterable>]
Condition: even numbers will be added as 'even', the number three will be added as 'number three' and the rest will be added as 'odd'.
new_list = ['even' if x % 2 == 0 else 'number three' if x == 3 else 'odd'
for x in range(1, 10)]
> ['odd', 'even', 'number three', 'even', 'odd', 'even', 'odd', 'even', 'odd']
The answer to the question
[f(x) for x in xs if x is not None else '']
Here we have a problem with the structure of the list: for x in xs should be at the end of the expression.
Correct way:
[f(x) if x is not None else '' for x in xs]
Further reading:
Does Python have a ternary conditional operator?
The specific problem has already been solved in previous answers, so I will address the general idea of using conditionals inside list comprehensions.
Here is an example that shows how conditionals can be written inside a list comprehension:
X = [1.5, 2.3, 4.4, 5.4, 'n', 1.5, 5.1, 'a'] # Original list
# Extract non-strings from X to new list
X_non_str = [el for el in X if not isinstance(el, str)] # When using only 'if', put 'for' in the beginning
# Change all strings in X to 'b', preserve everything else as is
X_str_changed = ['b' if isinstance(el, str) else el for el in X] # When using 'if' and 'else', put 'for' in the end
Note that in the first list comprehension for X_non_str, the order is:
expression for item in iterable if condition
and in the last list comprehension for X_str_changed, the order is:
expression1 if condition else expression2 for item in iterable
I always find it hard to remember that expression1 has to be before if and expression2 has to be after else. My head wants both to be either before or after.
I guess it is designed like that because it resembles normal language, e.g. "I want to stay inside if it rains, else I want to go outside"
In plain English the two types of list comprehensions mentioned above could be stated as:
With only if:
extract_apple for apple in apple_box if apple_is_ripe
and with if/else
mark_apple if apple_is_ripe else leave_it_unmarked for apple in apple_box
One way:
def change(x):
if x is None:
return f(x)
else:
return ''
result = [change(x) for x in xs]
Although then you have:
result = map(change, xs)
Or you can use a lambda inline.
Here is another illustrative example:
>>> print(", ".join(["ha" if i else "Ha" for i in range(3)]) + "!")
Ha, ha, ha!
It exploits the fact that if i evaluates to False for 0 and to True for all other values generated by the function range(). Therefore the list comprehension evaluates as follows:
>>> ["ha" if i else "Ha" for i in range(3)]
['Ha', 'ha', 'ha']
[f(x) if x != None else '' for x in xs]
Syntax for list comprehension:
[item if condition else item for item in items]
[f(item) if condition else value for item in items]
[item if condition for item in items]
[value if condition else value1 if condition1 else value2]
The other solutions are great for a single if / else construct. However, ternary statements within list comprehensions are arguably difficult to read.
Using a function aids readability, but such a solution is difficult to extend or adapt in a workflow where the mapping is an input. A dictionary can alleviate these concerns:
xs = [None, 'This', 'is', 'a', 'filler', 'test', 'string', None]
d = {None: '', 'filler': 'manipulated'}
res = [d.get(x, x) for x in xs]
print(res)
['', 'This', 'is', 'a', 'manipulated', 'test', 'string', '']
It has to do with how the list comprehension is performed.
Keep in mind the following:
[ expression for item in list if conditional ]
Is equivalent to:
for item in list:
if conditional:
expression
Where the expression is in a slightly different format (think switching the subject and verb order in a sentence).
Therefore, your code [x+1 for x in l if x >= 45] does this:
for x in l:
if x >= 45:
x+1
However, this code [x+1 if x >= 45 else x+5 for x in l] does this (after rearranging the expression):
for x in l:
if x>=45: x+1
else: x+5
Make a list from items in an iterable
It seems best to first generalize all the possible forms rather than giving specific answers to questions. Otherwise, the reader won't know how the answer was determined. Here are a few generalized forms I thought up before I got a headache trying to decide if a final else' clause could be used in the last form.
[expression1(item) for item in iterable]
[expression1(item) if conditional1 for item in iterable]
[expression1(item) if conditional1 else expression2(item) for item in iterable]
[expression1(item) if conditional1 else expression2(item) for item in iterable if conditional2]
The value of item doesn't need to be used in any of the conditional clauses. A conditional3 can be used as a switch to either add or not add a value to the output list.
For example, to create a new list that eliminates empty strings or whitespace strings from the original list of strings:
newlist = [s for s in firstlist if s.strip()]
There isn't any need for ternary if/then/else. In my opinion your question calls for this answer:
row = [unicode((x or '').strip()) for x in row]
You can combine conditional logic in a comprehension:
ps = PorterStemmer()
stop_words_english = stopwords.words('english')
best = sorted(word_scores.items(), key=lambda x: x[1], reverse=True)[:10000]
bestwords = set([w for w, s in best])
def best_word_feats(words):
return dict([(word, True) for word in words if word in bestwords])
# with stemmer
def best_word_feats_stem(words):
return dict([(ps.stem(word), True) for word in words if word in bestwords])
# with stemmer and not stopwords
def best_word_feats_stem_stop(words):
return dict([(ps.stem(word), True) for word in words if word in bestwords and word not in stop_words_english])
# coding=utf-8
def my_function_get_list():
my_list = [0, 1, 2, 3, 4, 5]
# You may use map() to convert each item in the list to a string,
# and then join them to print my_list
print("Affichage de my_list [{0}]".format(', '.join(map(str, my_list))))
return my_list
my_result_list = [
(
number_in_my_list + 4, # Condition is False : append number_in_my_list + 4 in my_result_list
number_in_my_list * 2 # Condition is True : append number_in_my_list * 2 in my_result_list
)
[number_in_my_list % 2 == 0] # [Condition] If the number in my list is even
for number_in_my_list in my_function_get_list() # For each number in my list
]
print("Affichage de my_result_list [{0}]".format(', '.join(map(str, my_result_list))))
(venv) $ python list_comp.py
Affichage de my_list [0, 1, 2, 3, 4, 5]
Affichage de my_result_list [0, 5, 4, 7, 8, 9]
So, for you:
row = [('', unicode(x.strip()))[x is not None] for x in row]