I was trying some different ways to run some for...in loops. Consider a list of lists:
list_of_lists = []
list = [1, 2, 3, 4, 5]
for i in range(len(list)):
list_of_lists.append(list) # each entry in l_o_l will now be list
Now let's say I want to have the first "column" of l_o_l be included in a separate list, a.
There are several ways I can go about this. For example:
a = [list[0] for list in list_of_lists] # this works (a = [1, 1, 1, 1, 1])
OR
a=[]
for list in list_of_lists:
a.append(hit[0]) #this also works
For the second example, however, I would imagine the "full" expansion to be equivalent to
a=[]
a.append(list[0] for list in list_of_lists) #but this, obviously, produces a generator error
The working "translation" is, in fact,
a=[]
a.append([list[0] for list in list_of_lists]) #this works
My question is on interpretation and punctuation, then. How come Python "knows" to append/does append the list brackets around the "list[0] for list in list_of_lists" expansion (and thus requires it in any rewrite)?
The issue here is that list comprehensions and generator expressions are not just loops, they are more than that.
List comprehensions are designed to be an easy way to build up a list from an iterable, as you have shown.
Your latter two examples both don't work - in both cases you are appending the wrong thing to the list - in the first case, a generator, the second appends a list inside your existing list. Neither of these are what you want.
You are trying to do something in two different ways at the same time, and it doesn't work. Just use the list comprehension - it does what you want to do in the most efficient and readable way.
Your main problem is you seem to have taken list comprehensions and generator expressions and not understood what they are and what they are trying to do. I suggest you try to understand them further before using them.
My question is on interpretation and punctuation, then. How come
Python "knows" to append/does append the list brackets around the
"hit[0] for list in list_of_lists" expansion (and thus requires it in
any rewrite)?
Not sure what that is supposed to mean. I think you might be unaware that in addition to list comprehensions [i*2 for i in range(0,3)] there are also generator expressions (i*2 for i in range(0,3)).
Generator expressions are a syntax for creating generators that perform a mapping, just like a list comprehension (but as a generator). Any list comprehension [c] can be rewritten list(c). The reason why there is a naked c inside list() is because where generator expressions appear as a parameter to a call, it is permitted to drop the brackets. This is what you are seeing in a.append(hit[0] for list in list_of_lists).
Related
This following page [see below] has a syntax description for a Python list comprehension which says that the output expression is "Optional." I haven't seen this "optional" designation elsewhere and it doesn't seem that a list comprehension would work without it. EG.
>>> llist = [1, 2, 3]
# list comprehension with output expression works
>>> listc = [num for num in llist]
# list comprehension without output expression fails
>>> listc2 =[for num in llist]
File "<stdin>", line 1
listc2 =[for num in llist]
^
SyntaxError: invalid syntax
** Here is the page:**
https://python-reference.readthedocs.io/en/latest/docs/comprehensions/list_comprehension.html
and here is the description from that page:
[expression(variable) for variable in input_set [predicate][, …]]
expression
Optional. An output expression producing members of the new set from members of the input set that satisfy the predicate expression.
variable
Required. Variable representing members of an input set.
input_set
Required. Represents the input set.
predicate
Optional. Expression acting as a filter on members of the input set.
[, …]]
Optional. Another nested comprehension.
Possibly they are trying to say that you can start a list comprehension with a bare variable, but that is still an expression, correct?
Looks like the doc is a bit unclear. You do need something on the left hand side. Otherwise the comprehension doesn't make much sense.
The page you have linked in an unofficial reference for Python, and as you can see on the GitHub page, it hasn't been updated in about 4 years. If you would like the up to date, and more importantly, correct information, go to the official documentation
Here is the link relevent to list comprehensions and their syntax
https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions
A list comprehension consists of brackets containing an expression
followed by a for clause, then zero or more for or if clauses.
The page you mentioned is quite clear about the expression. It's true, that in [expression(variable) for variable in input_set [predicate][, …]] the expression is optional, you may leave the variable as is, and it will still work. Moreover, you may leave there anything you like, not even remotely connected to the variable, like a number 42, and it still works.
Probably the original intention of the authors of the original page was to explain that you don't have to do anything with the loop variable if you don't want to an leave it as is.
I've been seeing many solutions with list comprehensions and i'm wondering if it's possible to convert it into a for loop notation.
For example, if i have a list comprehension notation:
radix = [radix_sort(i,0) for i in lst]
if i write it in:
for i in lst:
radix_sort(i,0)
do i get the same output? What differentiates both? Is it more efficient for list comprehension to be applied rather than conventional for loops?
A list comprehension creates a list—that's the whole point of it. But your loop doesn't create anything at all. So no, you're not going to get the same "output".
The equivalent loop is:
radix = []
for i in lst:
radix.append(radix_sort(i,0))
The list comprehension is defined to mean almost exactly the same thing as this. It may run a bit faster, at least in CPython,* but it will have the same effect.
If that radix_sort returns a copy of the list in sorted order, your loop was doing a lot of work for no effect. But now, as with the list comprehension, you're saving the result of all that work.
If, on the other hand, that radix_sort sorts the list in-place and returns nothing, then both the list comprehension and the explicit loop with append are highly misleading, and you should just use the loop without append.
* For example, in a comprehension, you don't have any way to access radix until the looping is done, so the compiler can make some assumptions and use a faster way of appending to the list.
I wrote a bit of code to iterate over a list to see if a line contained one or more keywords:
STRINGS_TO_MATCH = [ "Foo",
"Bar",
"Oggle" ]
for string in STRINGS_TO_MATCH
if string in line:
val_from_line = line.split(' ')[-1]
Does anyone happen to know if there is a way to make this more readable? Would a list comprehension be a better fit here?
The thing to remember here is that comprehensions are expressions, whose purpose is to create a value - list comprehensions create lists, dict comprehensions create dicts, and set comprehensions create sets. They are unlikely to help in this case, because you aren't creating any such object.
Your code sample is incomplete, because it doesn't do anything with the val_from_line values that it extracts. I am presuming that you want to extract the last "word" from a line which contains any of the strings in STRINGS_TO_MATCH, but it's difficult to work with such incomplete information so this answer might, for all I know, be totally useless.
Assuming I'm correct, the easiest way to find out if line contains any of the STRINGS_TO_MATCH is to use the expression
any(s in line for s in STRINGS_TO_MATCH)
This uses a so-called generator expression, which is similar to a list comprehension - the interpreter can iterate over it to produce a sequence of values - but it doesn't go as far as creating a list of the values, it just creates them as the client code (in this case the any built-in function) requests them. So I might rewrite your code as
if any(s in line for s in STRINGS_TO_MATCH):
val_from_line = line.split(' ')[-1]
I'll leave you to decide what you actually want to do after that, with the warning note that after this code executes val_from_line may or may not exist (depending on whether or not the condition was true), which is never an entirely comfortable situation.
This question already has answers here:
Generator expressions vs. list comprehensions
(13 answers)
Closed 7 years ago.
I come from a Java background and just started to work on Python. Most of the things are fairly easy to pick up but I am having hard time to understand one thing in the language which I just found out that is called list comprehension. What is this list comprehension in Python? How does this compare with language constructs found in Java? The problem is it's everywhere, nearly all the examples I found here and there use it.
For the following example, allow me to understand how this works.
[x**2 for x in range(10)]
And then there this.
[j + k for j in 'abc' for k in 'def']
Beyond that I also have seen things like this somewhere on Stackoverflow.
(x for x in (0,1,2,3,4))
Also things like this.
total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)
This started to get messy, could you please help me?
What is this list comprehension in Python?
First let’s start with the basic definition taken from the official Python documentation.
A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. The result will be a new list resulting from evaluating the expression in the context of the for and if clauses which follow it.
The problem is it's everywhere, nearly all the examples I found here and there use it.
List comprehension is a very flexible concept. It allows us to define lists as we know from the mathematics. Say we have a set S in which each element of S is a square of a number x and x could only takes values ranging from 0 to 10.
See the above definition. It took a paragraph to describe it. But there is a better way to describe it.
S = {x² : x in {0 ... 10}}
That’s why I love math, it is always to the point. Now remember your first example?
S = [x**2 for x in range(10)]
That’s the set we just defined. Neat right? That’s why it is used so much. (Don’t get confused with the x**2 syntax, because Python follows a weird syntax here which you probably might be familiar as x^2 from other languages.)
In Python you can iterate over pretty much anything. Which basically means you can easily iterate over each character of a string. Let’s look at the second example. It just iterates over the words ‘abc’ and ‘def’ and creates a list out of them.
lst = [j + k for j in 'abc' for k in 'def']
Notice that we assigned this to a list named lst. It was no coincidence. Python is the most humane programming language I have ever laid eyes on. So she will help you when you get stuck. Like this.
help(lst)
You can now see what you can do with lst. Ever got confused what lst is? You can check what it is via type.
print type(lst)
Before we move forward let’s talk a little bit about iterators and generators.
Iterators are objects that you can call next() method on. Like this.
iterator = iter([1,2,3,4])
Now we can print the first element of our iterator, like this.
print iterator.next()
Now we can talk about the generators. They are functions that generate iterators. There is however one other concept called generator expressions.
(x for x in (0,1,2,3,4))
This is a generator expression. A generator expressions is like a shortcut to build generators out of expressions similar to that of list comprehensions.
total = sum(x+y for x in (0,1,2,3) for y in (0,1,2,3) if x < y)
What above line does is to first create a generator using a generator expression and iterate over each element and sum those elements.
This question already has answers here:
Using while in list comprehension or generator expressions
(2 answers)
Closed 7 years ago.
On several occasions I've wanted python syntax for short-circuiting list comprehensions or generator expressions.
Here is a simple list comprehension, and the equivalent for loop in python:
my_list = [1, 2, 3, 'potato', 4, 5]
[x for x in my_list if x != 'potato']
result = []
for element in my_list:
if element != 'potato':
result.append(element)
There isn't support in the language for a comprehension which short-circuits. Proposed syntax, and equivalent for loop in python:
[x for x in my_list while x != 'potato']
# --> [1, 2, 3]
result = []
for element in my_list:
if element != 'potato':
result.append(element)
else:
break
It should work with arbitrary iterables, including infinite sequences, and be extendible to generator expression syntax. I am aware of list(itertools.takewhile(lambda x: x != 'potato'), my_list) as an option, but:
it's not particularly pythonic - not as readable as a while comprehension
it probably can't be as efficient or fast as a CPython comprehension
it requires an additional step to transform the output, whereas that can be put into a comprehension directly, e.g. [x.lower() for x in mylist]
even the original author doesn't seem to like it much.
My question is, was there any theoretical wrinkle about why it's not a good idea to extend the grammar to this use case, or is it just not possible because python dev think it would be rarely useful? It seems like a simple addition to the language, and a useful feature, but I'm probably overlooking some hidden subtleties or complications.
Related: this and this
Turns out, as #Paul McGuire noted, that it had been proposed in PEP 3142 and got rejected by Guido:
I didn't know there was a PEP for that. I hereby reject it. No point
wasting more time on it.
He doesn't give explanations, though. In the mailing list, some of the points against it are:
"[comprehension are] a carefully designed 1 to 1 transformation between multiple nested statements and a single expression. But this proposal ignores and breaks that. (here)." That is, the while keyword does not correspond to a while in the explicit loop - it is only a break there.
"It makes Python harder to learn because it adds one more thing to learn." (cited here)
It adds another difference between generator expressions and list-comprehension. Seems like adding this syntax to list comprehension too is absolutely out of the question.
I think one basic difference from the usual list comprehension is that while is inherently imperative, not declarative. It depends and dictates an order of execution, which is not guaranteed by the language (AFAIK). I guess this is the reason it is not included in Haskell's comprehensions, from which Python stole the idea.
Of course, generator expressions do have direction, but their elements may be precomputed - again, AFAIK. The PEP mentioned did propose it only for generator expressions - which makes some sense.
Of course, Python is an imperative language anyway, but it will raise problems.
What about choosing out of a non-ordered collection?
[x for x in {1,2,3} while x!=2]
You don't have it in simple for loops too, but that's something you can't enforce by the language. takewhile answers this question, but it is an arbitrary answer.
One last point, note that for consistency you will want support for dropwhile. something like
[x for x in my_list from x != 'potato']
Which is even less related to the equivalent for construct, and this time it is not possibly short circuit if my_list is just an iterable.