Normally, list comprehensions are used to derive a new list from an existing list. Eg:
>>> a = [1, 2, 3, 4, 5]
>>> [i for i in a if i > 2]
[3, 4, 5]
Should we use them to perform other procedures? Eg:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> [b.append(i) for i in a]
[None, None, None, None, None]
>>> print b
[1, 2, 3, 4, 5]
or should I avoid the above and use the following instead?:
for i in a:
b.append(i)
You should indeed avoid using list comprehensions (along with dictionary comprehensions, set comprehensions and generator expressions) for side effects. Apart from the fact that they'd accumulate a bogus list and thus waste memory, it's also confusing. I expect a list comprehension to generate a (meaningful) value, and many would agree. Loops, on the other hand, are clearly a sequence of statements. They are expected to kick off side effects and generate no result value - no surprise.
From python documentation:
List comprehensions provide a concise way to create lists. Common
applications are to make new lists
Perhaps you want to learn more about reduce(), filter() and map() functions.
In the example you give it would make the most sense to do:
b = [i for i in a]
if for some reason you wanted to create b. In general, there is some common sense that must be employed. If using a comprehension makes your code unreadable, don't use it. Otherwise go for it.
Only use list comprehensions if you plan to use the created list. Otherwise you create it just for the GC to throw it again without ever being used.
So instead of [b.append(i) for i in a] you should use a proper for loop:
for i in a:
b.append(i)
Another solution would be through a generator expression:
b += (i for i in a)
However, if you want to append the whole list, you can simply do
b += a
And if you just need to apply a function to the elements before adding them to the list, you can always use map:
b += map(somefunc, a)
b = []
a = [1, 2, 3, 4, 5]
b.extend (a)
Related
My goal
My question is about a list comprehension that does not puts elements in the resulting list as they are (which would results in a nested list), but extends the results into a flat list. So my question is not about flattening a nested list, but how to get a flat list while avoiding to make a nested list in the first place.
Example
Consider a have class instances with attributes that contains a list of integers:
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
Now I want to have a list of all integers in all instances of Foo. My best solution using extend is:
result = []
for f in list_of_foos:
result.extend(f.l)
As desired, result is now [1, 2, 3, 4, 5].
Is there something better? For example list comprehensions?
Since I expect list comprehension to be faster, I'm looking for pythonic way get the desired result with a list comprehension. My best approach is to get a list of lists ('nested list') and flatten this list again - which seems quirky:
result = [item for sublist in [f.l for f in list_of_foos] for item in sublist]
What functionaly I'm looking for
result = some_module.list_extends(f.l for f in list_of_foos)
Questions and Answers I read before
I was quite sure there is an answer to this problem, but during my search, I only found list.extend and list comprehension where the reason why a nested list occurs is different; and python list comprehensions; compressing a list of lists? where the answers are about avoiding the nested list, or how to flatten it.
You can use multiple fors in a single comprehension:
result = [
n
for foo in list_of_foos
for n in foo.l
]
Note that the order of fors is from the outside in -- same as if you wrote a nested for-loop:
for foo in list_of_foos:
for n in foo.l:
print(n)
If you want to combine multiple lists, as if they were all one list, I'd immediately think of itertools.chain. However, you have to access an attribute on each item, so we're also going to need operator.attrgetter. To get those together, I used map and itertools.chain.from_iterable()
https://docs.python.org/3/library/itertools.html#itertools.chain.from_iterable
from itertools import chain
from operator import attrgetter
class Foo:
def __init__(self, l):
self.l = l
foo_0 = Foo([1, 2, 3])
foo_1 = Foo([4, 5])
list_of_foos = [foo_0, foo_1]
for item in chain.from_iterable(map(attrgetter('l'), list_of_foos)):
print(item)
That demonstrates iterating through iterators with chain, as if they were one. If you don't specifically need to keep the list around, don't. But in case you do, here is the comprehension:
final = [item for item in chain.from_iterable(map(attrgetter('l'), list_of_foos))]
print(final)
[1, 2, 3, 4, 5]
In a list, you can make good use to + operator to concatenate two or more list together. It acts like an extend function to your list.
foo_0.l + foo_1.l
Out[7]: [1, 2, 3, 4, 5]
or you can use sum to perform this operation
sum([foo_0.l, foo_1.l], [])
Out[15]: [1, 2, 3, 4, 5]
In fact, it's in one of the post you have read ;)
I tried:
a_list = [1,2,3]
b_list = [4,5]
...
call_function(a_list + iter(b_list)) # TypeError
Is there a better code than this:
a_list = [1,2,3]
b_list = [4,5]
...
new_list = a_list[:]
new_list += iter(b_list) # no TypeError?
call_function(new_list)
Consider any iterator, I'm using islice in place of iter.
In python-3.5, you can use iterable unpacking:
call_function([*a_list, *iter(b_list)])
This works since:
>>> [*a_list, *iter(b_list)]
[1, 2, 3, 4, 5]
Notice the asterisk (*) in front of both a_list and iter(b_list). Furthermore a_list only has to be a finite iterable/iterator. So you can simply construct a list that concatenates finite iterables together.
You can generally use itertools.chain to join iterables:
from itertools import chain
new_list = list(chain(a_list, iter(b_list)))
print(new_list)
# [1, 2, 3, 4, 5]
The existing answers already address the workaround. Additionally, this line:
new_list += iter(b_list)
Does not throw an error, because it calls list.__iadd__ which supports the addition of iterators.
You can use __iadd__() which is the real function trigger by the syntactic sugar += (That's why it doesn't create an error).
call_function(a_list.__iadd__(iter(b_list)))
Produce
>>> a_list.__iadd__(iter(b_list))
[1, 2, 3, 4, 5]
This is fun but not really good in term of readability to be honest. Prefer other answers :)
EDIT:
Of course to produce new_list, you have to make a copy of list_a has you did in your question.
a_list[:].__iadd__(iter(b_list))
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 5 months ago.
Working my way into Python (2.7.1)
But failing to make sense (for hours) of this:
>>> a = [1, 2]
>>> b = [3, 4]
>>>
>>> a.extend([b[0]])
>>> a
[1, 2, 3]
>>>
>>> a.extend([b[1]])
>>> a
[1, 2, 3, 4]
>>>
>>> m = [a.extend([b[i]]) for i in range(len(b))] # list of lists
>>> m
[None, None]
The first two extends work as expected, but when compacting the same in a list comprehension it fails.
What am i doing wrong?
extend modifies the list in-place.
>>> [a + b[0:i] for i in range(len(b)+1)]
[[1, 2], [1, 2, 3], [1, 2, 3, 4]]
list.extend() extends a list in place. Python standard library methods that alter objects in-place always return None (the default); your list comprehension executed a.extend() twice and thus the resulting list consists of two None return values.
Your a.extend() calls otherwise worked just fine; if you were to print a it would show:
[1, 2, 3, 4, 3, 4]
You don't see the None return value in the Python interpreter, because the interpreter never echoes None results. You could test for that explicitly:
>>> a = []
>>> a.extend(['foo', 'bar']) is None
True
>>> a
['foo', 'bar']
the return value of extend is None.
extend function extends the list with the value you've provided in-place and returns None. That's why you have two None values in your list. I propose you rewrite your comprehension like so:
a = [1, 2]
b = [3, 4]
m = [a + [v] for v in b] # m is [[1,2,3],[1,2,4]]
For python lists, methods that change the list work in place and return None. This applies to extendas well as to append, remove, insert, ...
In reply to an older question, I sketched an subclass of list that would behave as you expected list to work.
Why does [].append() not work in python?
This is intended as educational. For pros and cons.. look at the comments to my answer.
I like this for the ability of chaining methods and working in a fluent style, e.g. then something like
li = FluentList()
li.extend([1,4,6]).remove(4).append(7).insert(1,10).reverse().sort(key=lambda x:x%2)
would be possible.
a.extend() returns None.
You probably want one of these:
>>> m = a + b
>>> m
[1, 2, 3, 4]
>>> a.extend(b)
>>> a
[1, 2, 3, 4]
Aside from that, if you want to iterate over all elements of a list, you just can do it like that:
m = [somefunction(element) for element in somelist]
or
for element in somelist:
do_some_thing(element)
In most cases there is no need to go over the indices.
And if you want to add just one element to a list, you should use somelist.append(element) instead of `somelist.extend([element])
This question already has answers here:
Why can't I iterate twice over the same iterator? How can I "reset" the iterator or reuse the data?
(5 answers)
Closed last month.
I would like to use itertools.chain for efficient concatenation of lists (memoization), but I need to be able to read (or map, etc.) the result multiple times. This example illustrates the problem:
import itertools
a = itertools.chain([1, 2], [3, 4])
print list(a) # => [1, 2, 3, 4]
print list(a) # => []
What is the best way to avoid this problem?
As with all generators, you'll need to convert it to a list and store that result instead:
a = list(a)
This is a fundamental principle of generators, they are expected to produce their sequence only once.
Moreover, you cannot simply store a generator for memoization purposes, as the underlying lists could change. In almost all memoization use-cases, you should store the list instead; a generator is usually only a means of efficiently transforming or filtering the underlying sequences, and does not represent the data you want to memoize itself. It's as if you are storing a function, not it's output. In your specific case, if all what you are doing is using chain() to concatenate existing lists, store those lists directly instead.
Note that this enables generators to produce endless sequences, so be careful with that you convert to a list.
Try itertools.tee:
import itertools
a = itertools.chain([1, 2], [3, 4])
a, b = itertools.tee(a)
print list(b) # => [1, 2, 3, 4]
a, b = itertools.tee(a)
print list(b) # => [1, 2, 3, 4]
Sometime I write code like this:
a,temp,b = s.partition('-')
I just need to pick the first and 3rd elements. temp would never be used. Is there a better way to do this?
In other terms, is there a better way to pick distinct elements to make a new list?
For example, I want to make a new list using the elements 0,1,3,7 from the old list. The
code would be like this:
newlist = [oldlist[0],oldlist[1],oldlist[3],oldlist[7]]
It's pretty ugly, isn't it?
Be careful using
a, _, b = s.partition('-')
sometimes _ is use for internationalization (gettext), so you wouldn't want to accidentally overwrite it.
Usually I would do this for partition rather than creating a variable I don't need
a, b = s.partition('-')[::2]
and this in the general case
from operator import itemgetter
ig0137 = itemgetter(0, 1, 3, 7)
newlist = ig0137(oldlist)
The itemgetter is more efficient than a list comprehension if you are using it in a loop
For the first there's also this alternative:
a, b = s.partition('-')[::2]
For the latter, since there's no clear interval there is no way to do it too clean. But this might suit your needs:
newlist = [oldlist[k] for k in (0, 1, 3, 7)]
You can use Python's extended slicing feature to access a list periodically:
>>> a = range(10)
>>> # Pick every other element in a starting from a[1]
>>> b = a[1::2]
>>> print b
>>> [1, 3, 5, 7, 9]
Negative indexing works as you'd expect:
>>> c = a[-1::-2]
>>> print c
>>> [9, 7, 5, 3, 1]
For your case,
>>> a, b = s.partition('-')[::2]
the common practice in Python to pick 1st and 3rd values is:
a, _, b = s.partition('-')
And to pick specified elements in a list you can do :
newlist = [oldlist[k] for k in (0, 1, 3, 7)]
If you don't need to retain the middle field you can use split (and similarly rsplit) with the optional maxsplit parameter to limit the splits to the first (or last) match of the separator:
a, b = s.split('-', 1)
This avoids a throwaway temporary or additional slicing.
The only caveat is that with split, unlike partition, the original string is returned if the separator is not found. The attempt to unpack will fail as a result. The partition method always returns a 3-tuple.