This is probably simple, but, I cannot find it for some reason. For example:
def fnc(dt):
print dt
return;
#[(fnc(y)) for y in range(5) for x in range(5)]
for x in range(0, 5):
fnc(x)
for y in range(0, 5):
fnc(y)
I would like the commented out line to have similar behaviour with the double nested loop bellow it. Is this possible? I give up, I cannot find it! Thanks for any input.
You have to used nested list comprehensions to achieve the same result:
[(fnc(x),[fnc(y) for y in range(5)]) for x in range(5)]
I used a tuple (fnc(x), [...]) to output x before performing the list comprehension for y.
P.S.: Don't actually use this. Stick to your loops.
You don't need a list comprehension here. List comprehensions are for building lists not for side effects, as you have in your for loop. Any solution that provides the same result using a list comp. (like the one below) will produce a useless list of Nones:
[fnc(y) for x in range(5) if fnc(x) or 1 for y in range(5)]
The code is unpythonic and unreadable. You should never use it. fnc(x) is always evaluated while evaluating the if, and the branch is always taken because it is short-circuited with a truthy value using or, so that the nested loop always executes ∀ x.
The Pythonic way is to use a vanilla for like you've done.
What you can do is probably technically possible (I'm thinking of a class with an overridden iterator that calls func() in the iteration, although I'm not sure if this is actually implementable).
The implementation, however, would be an aberration.
List comprehensions are intended as a fast way to filter, combine and/or process data in a list to generate another one. You should think of them as a way to quickly apply a function to all the data in the list, appending each time the function results to the output list.
This is why there's no syntax to, say, do assignments or external function calls in comprehensions. If you need to call a function in the inner loop before processing the data, you're better off with the nested loop approach (which is also much more readable than anything equivalent hacked to work in a comprehension)
Related
Think about a function that I'm calling for its side effects, not return values (like printing to screen, updating GUI, printing to a file, etc.).
def fun_with_side_effects(x):
...side effects...
return y
Now, is it Pythonic to use list comprehensions to call this func:
[fun_with_side_effects(x) for x in y if (...conditions...)]
Note that I don't save the list anywhere
Or should I call this func like this:
for x in y:
if (...conditions...):
fun_with_side_effects(x)
Which is better and why?
It is very anti-Pythonic to do so, and any seasoned Pythonista will give you hell over it. The intermediate list is thrown away after it is created, and it could potentially be very, very large, and therefore expensive to create.
You shouldn't use a list comprehension, because as people have said that will build a large temporary list that you don't need. The following two methods are equivalent:
consume(side_effects(x) for x in xs)
for x in xs:
side_effects(x)
with the definition of consume from the itertools man page:
def consume(iterator, n=None):
"Advance the iterator n-steps ahead. If n is none, consume entirely."
# Use functions that consume iterators at C speed.
if n is None:
# feed the entire iterator into a zero-length deque
collections.deque(iterator, maxlen=0)
else:
# advance to the empty slice starting at position n
next(islice(iterator, n, n), None)
Of course, the latter is clearer and easier to understand.
List comprehensions are for creating lists. And unless you are actually creating a list, you should not use list comprehensions.
So I would got for the second option, just iterating over the list and then call the function when the conditions apply.
Second is better.
Think of the person who would need to understand your code. You can get bad karma easily with the first :)
You could go middle between the two by using filter(). Consider the example:
y=[1,2,3,4,5,6]
def func(x):
print "call with %r"%x
for x in filter(lambda x: x>3, y):
func(x)
Depends on your goal.
If you are trying to do some operation on each object in a list, the second approach should be adopted.
If you are trying to generate a list from another list, you may use list comprehension.
Explicit is better than implicit.
Simple is better than complex. (Python Zen)
You can do
for z in (fun_with_side_effects(x) for x in y if (...conditions...)): pass
but it's not very pretty.
Using a list comprehension for its side effects is ugly, non-Pythonic, inefficient, and I wouldn't do it. I would use a for loop instead, because a for loop signals a procedural style in which side-effects are important.
But, if you absolutely insist on using a list comprehension for its side effects, you should avoid the inefficiency by using a generator expression instead. If you absolutely insist on this style, do one of these two:
any(fun_with_side_effects(x) and False for x in y if (...conditions...))
or:
all(fun_with_side_effects(x) or True for x in y if (...conditions...))
These are generator expressions, and they do not generate a random list that gets tossed out. I think the all form is perhaps slightly more clear, though I think both of them are confusing and shouldn't be used.
I think this is ugly and I wouldn't actually do it in code. But if you insist on implementing your loops in this fashion, that's how I would do it.
I tend to feel that list comprehensions and their ilk should signal an attempt to use something at least faintly resembling a functional style. Putting things with side effects that break that assumption will cause people to have to read your code more carefully, and I think that's a bad thing.
I am experiencing a bit of confusion with how to place a for loop at the end of a line in python, for instance
for i in x:
print i
produces the expected result but if I run
print i for i in x
I get a syntax error. Could someone explain a little more about how one goes about putting your for loops at the end of a line like this.
In earlier versions of Python, the idea of list comprehensions was introduced to neaten up these two patterns of code:
# Go through each item in a list and run a test on them
# build a new list containing only items that pass the test
results = []
for item in somelist:
if sometest(item):
results.add(item)
and
# build a new list by changing every item in a list in the
# same way
results = []
for item in somelist:
results.add(2 * item)
by adding a new syntax that includes doing all three things in one - changing the items and/or testing them to only include some in the result, and creating the list of the results:
results = [2 * item for item in somelist if sometest(item)]
# results is a list
This feature uses the [] syntax that indicates "list" in Python, and it builds a list right away in memory.
Sometimes you don't want or need the entire list built in memory right away, so later versions of Python introduced generator expressions - the same idea, but they save memory by quickly returning a generator which you can iterate over as if it was a list, and it builds the list as and when you use it.
But they can't have the same syntax and be a direct swap out because they behave slightly differently, so they use () instead of [], e.g.:
somelist = [1,2,3,4,5]
results = (2 * item for item in somelist if sometest(item))
# results is a generator
If you try and call a function with this, you get two layers of parentheses:
function((2 * item for item in somelist))
Which looks silly, so you can leave one out:
function(2 * item for item in somelist)
Which appears to be a standalone backwards for loop on its own, but it actually isn't.
So with parentheses you can write this:
>>> print (item for item in [1,2,3])
<generator object <genexpr> at 0x7fe31b8663c0>
Which is the closest thing to what you wrote that is valid syntax in Python 2.x, but it doesn't do what you expect, and ^^ is what it does and why.
Or this:
>>> print [item for item in [1,2,3]]
[1,2,3]
Which generates a list and prints the list, the other close thing to what you wrote, but still not what you expected.
-- (there really isn't much point in me posting now a bunch other answers have appeared while I was writing this, but eh).
expression for i in x
doesn't mean anything by itself in Python. You can have
(expression for i in x)
[expression for i in x]
But you can't do
[print i for i in x]
because print isn't an expression, it's a statement.
First of all:
this is considered bad style—you're essentially just abusing the list comprehension syntax to get what's effectively a different notation for imperative for loops.
this only works in Python 3 where print is a function by default, or when you do from __future__ import print_function in Python 2.x
However, if you insist on putting the for ... part after the print i part, you can do:
[print(i) for i in x]
(but I'm writing that example for purely "academic" purposes)
P.S. if you let us know what you want to do, we might be able to provide a suitable overall solution, but if you're just asking for the sake of it, then that's it.
You can't. You've probably seen a generator expression, which takes the form x for x in iter. a for loop is slightly different, though you can definitely see for x in iter inside the genexp.
In Python3, you can do:
print(*(i for i in x))
And as #wim points out in the comments, you can make it more "for loopy" by doing
print(*(i for i in x), sep='\n')
You can of course do arbitrary changes since this is a genexp, so e.g. i**2 for i in x will give you the square of each item in x, i for i in x if i%2 will give all odd numbers in x, etc.
This will create a generator of each item in x, then pass each one in turn (using the * assignment the same way *args is built and etc) as separate arguments to print
The only thing I can think of that is equivalent in python 2.x would be this:
print '\n'.join(str(i) for i in x)
But don't do that. The for loop as you had it is much clearer.
If you are just fooling around in interactive interpreter and want to one-liner it for some reason, this will work:
for i in x: print i
But it's violating pep8 so I wouldn't write a line like that into a script.
Consider the following example code. The example is given just to highlight the different functionality between map and itertools.imap. What I really want to do cannot be
done with a list comprehension because in my real problem I am not creating a list but filling a larger numpy array with small arrays. So in response to the code below, please do not suggest: [f(x) for x in range(3)]
Example:
g = []
def f(x):
g.append(x)
The results I get with map and itertools.map:
map(f, range(3)) # Results in g = [0,1,2]
itertools.imap(f, range(3)) # Does not change g
I want the g to change as done by the map function.
However, I have heard that map will (or does) behave like
itertools.imap in Python 3. Even though I am using Python 2.7 I want to learn the correct way of using an iterator version of the map. How do I use itertools.imap to achieve
the same outcome as I get with map?
I can do:
b = itertools.imap(f, range(3))
list(b) # This gives g = [0,1,2] if starting with an empty g.
Is this the correct way, or is there a better way?
Thanks in advance.
itertools functions return generators; they only operate when iterated over. So itertools.imap(f, range(3)) won't actually do anything until you run it to completion e.g. with list.
Per http://docs.python.org/2/library/itertools.html#recipes, the most efficient way to consume an iterator is with a zero-length deque:
collections.deque(itertools.imap(f, range(3)), maxlen=0)
However, if you're calling a function for its side effects you should be using an imperative as opposed to functional syntax:
for i in range(3):
f(i)
You are using map() incorrectly. It should not be used for the side effects of the function call, but to transform an iterable.
You can use list(itertools.imap(f, range(3))) to get the behavior you want, but I would argue that you should change your entire approach to use a normal for loop:
for i in range(3):
f(i)
This makes it clear that the return values from the f() calls are not being used.
The difference is that map in Python 2.x is the equivalent of list(map(...)) in Python 3. That's it - nothing special...
Think about a function that I'm calling for its side effects, not return values (like printing to screen, updating GUI, printing to a file, etc.).
def fun_with_side_effects(x):
...side effects...
return y
Now, is it Pythonic to use list comprehensions to call this func:
[fun_with_side_effects(x) for x in y if (...conditions...)]
Note that I don't save the list anywhere
Or should I call this func like this:
for x in y:
if (...conditions...):
fun_with_side_effects(x)
Which is better and why?
It is very anti-Pythonic to do so, and any seasoned Pythonista will give you hell over it. The intermediate list is thrown away after it is created, and it could potentially be very, very large, and therefore expensive to create.
You shouldn't use a list comprehension, because as people have said that will build a large temporary list that you don't need. The following two methods are equivalent:
consume(side_effects(x) for x in xs)
for x in xs:
side_effects(x)
with the definition of consume from the itertools man page:
def consume(iterator, n=None):
"Advance the iterator n-steps ahead. If n is none, consume entirely."
# Use functions that consume iterators at C speed.
if n is None:
# feed the entire iterator into a zero-length deque
collections.deque(iterator, maxlen=0)
else:
# advance to the empty slice starting at position n
next(islice(iterator, n, n), None)
Of course, the latter is clearer and easier to understand.
List comprehensions are for creating lists. And unless you are actually creating a list, you should not use list comprehensions.
So I would got for the second option, just iterating over the list and then call the function when the conditions apply.
Second is better.
Think of the person who would need to understand your code. You can get bad karma easily with the first :)
You could go middle between the two by using filter(). Consider the example:
y=[1,2,3,4,5,6]
def func(x):
print "call with %r"%x
for x in filter(lambda x: x>3, y):
func(x)
Depends on your goal.
If you are trying to do some operation on each object in a list, the second approach should be adopted.
If you are trying to generate a list from another list, you may use list comprehension.
Explicit is better than implicit.
Simple is better than complex. (Python Zen)
You can do
for z in (fun_with_side_effects(x) for x in y if (...conditions...)): pass
but it's not very pretty.
Using a list comprehension for its side effects is ugly, non-Pythonic, inefficient, and I wouldn't do it. I would use a for loop instead, because a for loop signals a procedural style in which side-effects are important.
But, if you absolutely insist on using a list comprehension for its side effects, you should avoid the inefficiency by using a generator expression instead. If you absolutely insist on this style, do one of these two:
any(fun_with_side_effects(x) and False for x in y if (...conditions...))
or:
all(fun_with_side_effects(x) or True for x in y if (...conditions...))
These are generator expressions, and they do not generate a random list that gets tossed out. I think the all form is perhaps slightly more clear, though I think both of them are confusing and shouldn't be used.
I think this is ugly and I wouldn't actually do it in code. But if you insist on implementing your loops in this fashion, that's how I would do it.
I tend to feel that list comprehensions and their ilk should signal an attempt to use something at least faintly resembling a functional style. Putting things with side effects that break that assumption will cause people to have to read your code more carefully, and I think that's a bad thing.
I was writing a python function that looked something like this
def foo(some_list):
for i in range(0, len(some_list)):
bar(some_list[i], i)
so that it was called with
x = [0, 1, 2, 3, ... ]
foo(x)
I had assumed that index access of lists was O(1), but was surprised to find that for large lists this was significantly slower than I expected.
My question, then, is how are python lists are implemented, and what is the runtime complexity of the following
Indexing: list[x]
Popping from the end: list.pop()
Popping from the beginning: list.pop(0)
Extending the list: list.append(x)
For extra credit, splicing or arbitrary pops.
there is a very detailed table on python wiki which answers your question.
However, in your particular example you should use enumerate to get an index of an iterable within a loop. like so:
for i, item in enumerate(some_seq):
bar(item, i)
The answer is "undefined". The Python language doesn't define the underlying implementation. Here are some links to a mailing list thread you might be interested in.
It is true that Python's lists have
been implemented as contiguous
vectors in the C implementations of
Python so far.
I'm not saying that the O()
behaviours of these things should be
kept a secret or anything. But you
need to interpret them in the context
of how Python works generally.
Also, the more Pythonic way of writing your loop would be this:
def foo(some_list):
for item in some_list:
bar(item)
Lists are indeed O(1) to index - they are implemented as a vector with proportional overallocation, so perform much as you'd expect. The likely reason you were finding this code slower than you expected is the call to "range(0, len(some_list))".
range() creates a new list of the specified size, so if some_list has 1,000,000 items, you will create a new million item list up front. This behaviour changes in python3 (range is an iterator), to which the python2 equivalent is xrange, or even better for your case, enumerate
if you need index and value then use enumerate:
for idx, item in enumerate(range(10, 100, 10)):
print idx, item
Python list actually nothing but arrays. Thus,
indexing takes O(1)
for pop and append again it should be O(1) as per the docs
Check out following link for details:
http://dustycodes.wordpress.com/2012/03/31/pythons-data-structures-complexity-analysis/