I want this:
[foo() for _ in xrange (100)]
but beautifuller. ?
You can write a generator repeat like this:
def repeat(times, func, *args, **kwargs):
for _ in xrange(times):
yield func(*args, **kwargs)
Then:
list(repeat(100, foo))
It also accepts arguments to be passed on to the function, so you can:
from random import randint
list(repeat(100, randint, 1, 100)) # 100 random ints between 1 and 100
Since it's a generator, you can pipe it into any kind of iterable, be it a list (as here) or a tuple or a set, or use it in a comprehension or a loop.
I'm afraid you're not gonna get it any prettier than that in Python, except that some people would advise against _ for an "anonymous" variable. This is the Pythonic idiom for doing what you want.
(The _ can be considered confusing to novices because it can be mistaken for special syntax. I use it, but only in the "expert parts" of my code. I also encounter it more and more often, but opinion still seems a bit divided on this one.)
Depending on your definition of "beautifuller", you may prefer this:
map(lambda x: foo(), xrange(100))
Although what you have already is much nicer IMO.
Depending on what it does, you can make foo() a generator.
Your list comprehension is already beatiful and effective but if you need several options to do the same things then i think you can use map here. In case you need to call a certain function the specified number of times use:
# in case your func looks like
def func():
# do something
#then
map(func(), xrange(numberOfTimes))
In case your function need value from range then you can use map with lambda:
# in case your func looks like
def func(value):
# do something with value
#then
map(lambda val: func(val), xrange(numberOfTimes))
Or in case you need to use data from several lists of the same length:
# in case your func looks like
def func(value1, value2):
# do something with values
#then
map(lambda val: func(*val), zip(xrange(10), xrange(10,20)))
And so on...
In case foo() always returns the same result, you could use
[foo()]*100
This has the advantage that foo() is only called once.
Edit: As #larsmans points out this only makes sense though if foo() returns an immutable result.
In all other cases, your solution is fine!
Related
In my python project, im many places appears the functionality strict equivalent to:
def map_or_single(item,func, iterative_classes, *args ):
if isinstance(item, iterative_classes):
for k in item:
func(k, *args)
return
func(item, *args)
but i am sure there must be some Python (3 ?) built-in function for this trivial task, i dont want to invent a bycicle and use this crap in the project. Anyone know the proper build-in function for that ?
Don't try to write this at all. Instead of
map_or_single(item, func, ...)
just write
def mapper(func, items, *args):
for k in items:
func(k, *args)
mapper always takes a list (or some other iterable); it's up to the caller to provide one if necessary.
mapper(f, [1,2,3,4])
mapper(f, [1]) # Not mapper(f, 1)
mapper(f, some_tree_object.depth_first_iterator())
mapper(f, some_tree_object.breadth_first_iterator())
This is a clear antipattern, therefore there is no "proper" or "correct" way to do this in Python (or most, if not all other languages). You most certainly can come up with a wide variety of brittle solutions, but this is the type of thing that leads to bugs in the long term for very little benefit.
This problem is solved by avoiding this pattern altogether and requiring user input to conform to one pattern instead of two or more.
Instead of this interface:
map_or_single(single, func, ...)
map_or_single(iterable, func, ...)
You have this interface:
map_or_single([single], func, ...)
map_or_single(iterable, func, ...)
Requiring single values to be wrapped is a small price to pay to avoid all the potential headaches that can easily result from this pattern.
And obviously if the situation permits:
func(single)
map(func, iterable)
I'm pretty sure that there's no one-liner builtin for this. I have this helper function in my project;
def any2list(val):
if isinstance(val, (str, bytes)):
val = [val]
try:
iter(val)
except TypeError:
val = [val]
return list(val)
The goal of this function is to convert any input to a list. If a single element is given, it returns a list of this one element.
I use this in other functions like so,
def map_or_single(container, func, *args):
container = any2list(container)
for k in container:
func(k, *args)
and now the container argument may be a proper container type (like list) or it might be a single element (scalar) like an int. I find this strategy pretty clean as I do not explicitly worry about the single element case.
Note: I choose to treat strs and bytes specially, so that any2list does not split a word or sentence into individual letters, which is almost always what I want. Your use cases might be different.
You can use functions map() and partial():
from functools import partial
from operator import add
l = [1, 2, 3]
result = map(partial(add, 2), l)
print(list(result))
# [3, 4, 5]
Oftentimes the case arises where one would need to loop indefinitely until a certain condition has been attained. For example, if I want keep collecting random integers until I find a number == n, following which I break. I'd do this:
import random
rlist = []
n = ...
low, high = ..., ...
while True:
num = random.randint(low, high)
if num == n:
break
rlist.append(num)
And this works, but is quite clunky. There is a much more pythonic alternative using iter:
iter(o[, sentinel])
Return an iterator object. The first argument is
interpreted very differently depending on the presence of the second
argument. [...] If the second argument,
sentinel, is given, then o must be a callable object. The iterator
created in this case will call o with no arguments for each call to
its next() method; if the value returned is equal to sentinel,
StopIteration will be raised, otherwise the value will be returned.
The loop above can be replaced with
import random
from functools import partial
f = partial(random.randint, low, high)
rlist = list(iter(f, 10))
To extend this principle to lists that have already been created, a slight change is needed. I'll need to define a partial function like this:
f = partial(next, iter(x)) # where x is some list I want to keep taking items from until I hit a sentinel
The rest remains the same, but the main caveat with this approach versus the while loop is I cannot apply generic boolean conditions.
For example, I cannot apply a "generate numbers until the first even number greater than 1000 is encountered".
The bottom line is this: Is there another alternative to the while loop and iter that supports a callback sentinel?
If you want generic boolean conditions, then iter(object, sentinel) is insufficiently expressive for your needs. itertools.takewhile(), in contrast, seems to be more or less what you want: It takes an iterator, and cuts it off once a given predicate stops being true.
rlist = list(itertools.takewhile(lambda x: x >= 20, inputlist))
Incidentally, partial is not very Pythonic, and neither is itertools. GvR is on record as disliking higher-order functional-style programming (note the downgrading of reduce from built-in to a module member in 3.0). Attributes like "elegant" and "readable" are in the eye of the beholder, but if you're looking for Pythonic in the purest sense, you want the while loop.
The following python code produces [(0, 0), (0, 7)...(0, 693)] instead of the expected list of tuples combining all of the multiples of 3 and multiples of 7:
multiples_of_3 = (i*3 for i in range(100))
multiples_of_7 = (i*7 for i in range(100))
list((i,j) for i in multiples_of_3 for j in multiples_of_7)
This code fixes the problem:
list((i,j) for i in (i*3 for i in range(100)) for j in (i*7 for i in range(100)))
Questions:
The generator object seems to play the role of an iterator instead of providing an iterator object each time the generated list is to be enumerated. The later strategy seems to be adopted by .Net LINQ query objects. Is there an elegant way to get around this?
How come the second piece of code works? Shall I understand that the generator's iterator is not reset after looping through all multiples of 7?
Don't you think that this behavior is counter intuitive if not inconsistent?
A generator object is an iterator, and therefore one-shot. It's not an iterable which can produce any number of independent iterators. This behavior is not something you can change with a switch somewhere, so any work around amounts to either using an iterable (e.g. a list) instead of an generator or repeatedly constructing generators.
The second snippet does the latter. It is by definition equivalent to the loops
for i in (i*3 for i in range(100)):
for j in (i*7 for i in range(100)):
...
Hopefully it isn't surprising that here, the latter generator expression is evaluated anew on each iteration of the outer loop.
As you discovered, the object created by a generator expression is an iterator (more precisely a generator-iterator), designed to be consumed only once. If you need a resettable generator, simply create a real generator and use it in the loops:
def multiples_of_3(): # generator
for i in range(100):
yield i * 3
def multiples_of_7(): # generator
for i in range(100):
yield i * 7
list((i,j) for i in multiples_of_3() for j in multiples_of_7())
Your second code works because the expression list of the inner loop ((i*7 ...)) is evaluated on each pass of the outer loop. This results in creating a new generator-iterator each time around, which gives you the behavior you want, but at the expense of code clarity.
To understand what is going on, remember that there is no "resetting" of an iterator when the for loop iterates over it. (This is a feature; such a reset would break iterating over a large iterator in pieces, and it would be impossible for generators.) For example:
multiples_of_2 = iter(xrange(0, 100, 2)) # iterator
for i in multiples_of_2:
print i
# prints nothing because the iterator is spent
for i in multiples_of_2:
print i
...as opposed to this:
multiples_of_2 = xrange(0, 100, 2) # iterable sequence, converted to iterator
for i in multiples_of_2:
print i
# prints again because a new iterator gets created
for i in multiples_of_2:
print i
A generator expression is equivalent to an invoked generator and can therefore only be iterated over once.
The real issue as I found out is about single versus multiple pass iterables and the fact that there is currently no standard mechanism to determine if an iterable single or multi pass: See Single- vs. Multi-pass iterability
If you want to convert a generator expression to a multipass iterable, then it can be done in a fairly routine fashion. For example:
class MultiPass(object):
def __init__(self, initfunc):
self.initfunc = initfunc
def __iter__(self):
return self.initfunc()
multiples_of_3 = MultiPass(lambda: (i*3 for i in range(20)))
multiples_of_7 = MultiPass(lambda: (i*7 for i in range(20)))
print list((i,j) for i in multiples_of_3 for j in multiples_of_7)
From the point of view of defining the thing it's a similar amount of work to typing:
def multiples_of_3():
return (i*3 for i in range(20))
but from the point of view of the user, they write multiples_of_3 rather than multiples_of_3(), which means the object multiples_of_3 is polymorphic with any other iterable, such as a tuple or list.
The need to type lambda: is a bit inelegant, true. I don't suppose there would be any harm in introducing "iterable comprehensions" to the language, to give you what you want while maintaining backward compatibility. But there are only so many punctuation characters, and I doubt this would be considered worth one.
I would like to implement a map-like function which preserves the type of input sequence. map does not preserve it:
map(str, (8, 9)) # input is a tuple
=> ['8', '9'] # output is a list
One way I came up with is this:
def map2(f, seq):
return type(seq)( f(x) for x in seq )
map2(str, (1,2))
=> ('1', '2')
map2(str, [3,4])
=> ['3', '4']
map2(str, deque([5,6]))
=> deque(['5', '6'])
However, this does not work if seq is an iterator/generator. imap works in this case.
So my questions are:
Is there a better way to implement map2, which supports list, tuple, and many others?
Is there an elegant way to extend map2 to also support generators (like imap does)? Clearly, I'd like to avoid: try: return map2(...) except TypeError: return imap(...)
The reason I'm looking for something like that is that I'm writing a function-decorator which converts the return value, from type X to Y. If the original function returns a sequence (let's assume a sequence can only be a list, a tuple, or a generator), I assume it is a sequence of X's, and I want to convert it to the corresponding sequence of Y's (while preserving the type of the sequence).
As you probably realize, I'm using python 2.7, but python 3 is also of interest.
Your formalism also doesn't work for map(str,'12') either.
Ultimately, you don't know what arguments the type of the iterable will actually take in the constructor/initializer, so there's no way to do this in general. Also note that imap doesn't give you the same type as a generator:
>>> type(x for x in range(10))
<type 'generator'>
>>> type(imap(str,range(10)))
<type 'itertools.imap'>
>>> isinstance((x for x in range(10)),type(imap(str,range(10))))
False
You might be thinking to yourself "surely with python's introspection, I could inspect the arguments to the initializer" -- And you'd be right! However, even if you know how many arguments go to the initializer, and what their names are, you still can't get any information on what you're actually supposed to pass to them. I suppose you could write some sort of machine learning algorithm to figure it out from the docstrings ... but I think that's well beyond the scope of this question (and it assumes the author was behaving nicely and creating good docstrings to begin with).
First, type(seq)( f(x) for x in seq ) is really just type(seq)(imap(f, seq)). Why not just use that?
Second, what you're trying to do doesn't make sense in general. map takes any iterable, not just a sequence. The difference is, basically, that a sequence has a len and is randomly-accessible.
There is no rule that an iterable of type X can be constructed from values of type Y by calling type(X)(y_iter). In fact, while it's generally true for sequences, there are very few other examples for which it is true.
If what you want is to handle a few special types specially, you can do that:
def map2(f, seq):
it = imap(f, seq)
if isinstance(seq, (tuple, list)):
return type(seq)(it)
else:
return it
Or, if you want to assume that all sequences can be constructed this way (which is true for most built-in sequences, but consider, e.g. xrange—which wasn't designed as a sequence but does meet the protocol—and of course there are no guarantees beyond what's built in):
def map2(f, seq):
it = imap(f, seq)
try:
len(seq)
except:
return it
else:
return type(seq)(it)
You could assume that any iterable type that can be constructed from an iterable is a sequence (as you suggested in your question)… but this is likely to lead to more false positives than benefits, so I wouldn't. Again, remember that len is part of the definition of being a sequence, while "constructible from an iterator" is not, and there are perfectly reasonable iterable types that will do something completely different when given an iterator.
Whatever you do is going to be a hack, because the very intention is a hack, and goes against the explicit design wishes of the Python developers. The whole point of the iterator/iterable protocol is that you should care about the type of the iterable as rarely as possible. That's why Python 3.x has gone further and replaced the list-based functions like map and filter with iterator-based functions instead.
So, how do we turn one of these transformations into a decorator?
Well, first, let's skip the decorator bit and just write a higher-order function that takes an imap-like function and returns an equivalent function with this transformation applied to it:
def sequify(func):
def wrapped(f, seq):
it = func(f, seq)
try:
len(seq)
except:
return it
else:
return type(seq)(it)
return wrapped
So:
>>> seqmap = sequify(itertools.imap)
>>> seqmap(int, (1.2, 2.3))
(1, 2)
>>> sequify(itertools.ifilter)(lambda x: x>0, (-2, -1, 0, 1, 2))
(1, 2)
Now, how do we turn that into a decorator? Well, a function that returns a function already is a decorator. You probably want to add in functools.wraps (although you may want that even in the non-decorator case), but that's the only change. For example, I can write a generator that acts like imap, or a function that returns an iterator, and automatically transform either into a seqmap-like function:
#sequify
def map_and_discard_none(func, it):
for elem in imap(func, it):
if elem is not None:
yield elem
Now:
>>> map_and_discard_none(lambda x: x*2 if x else x, (1, 2, None))
(2, 4)
This, of course, only works for functions with map-like syntax—that is, they take a function and an iterable. (Well, it will accidentally work for functions that take various kinds of wrong types—e.g., you can call sequify(itertools.count(10, 5)) and it will successfully detect that 5 isn't a sequence and therefore just pass the iterator back untouched.) To make it more general, you could do something like:
def sequify(func, type_arg=1):
def wrapped(*args, **kwargs):
it = func(f, seq)
try:
len(args[type_arg])
except:
return it
else:
return type(seq)(it)
return wrapped
And now, you can go crazy with sequify(itertools.combinations, 0) or whatever you prefer. In this case, to make it a useful decorator, you probably want to go a step further:
def sequify(type_arg=1):
def wrapper(func):
def wrapped(*args, **kwargs):
it = func(f, seq)
try:
len(args[type_arg])
except:
return it
else:
return type(seq)(it)
return wrapped
return wrapper
So you can do this:
#sequify(3)
def my_silly_function(pred, defval, extrastuff, main_iterable, other_iterable):
Your question boils down to:
Given a sequence (by which you seem to mean any python object which supports iteration, not the same sequence python docs lay down) and a transformation, is there a general way to apply the transformation to each element and create a new sequence of the exact same type?
The answer is no. There is no guarantee that the iterable type will support creating a new instance from an iterable. Some objects support this inherently in their constructors; some do not. An iterable type makes no guarantees about supporting the opposite operation. You would need to special case all types you were aware of that would not work with the simple iterable as an argument to the initialization case.
Is there a more elegant way to write the following piece of Python?
[foo() for i in range(10)]
I want to accumulate the results of foo() in a list, but I don't need the iterator i.
One way to do this is to use _:
[foo() for _ in range(10)]
This means exactly the same thing, but by convention the use of _ indicates to the reader that the index isn't actually used for anything.
Presumably foo() returns something different every time you call it. If it doesn't, and it returns the same thing each time, then you can:
[foo()] * 10
to replicate the result of calling foo() once, 10 times into a list.
map would be nice if foo() took an argument, but it doesn't. So instead, create a dummy lambda that takes an integer argument, but just calls foo():
map(lambda i:foo(), range(10))
If you are on Python 3.x, map returns an iterator instead of a list - just construct a list with it:
list(map(lambda i:foo(), range(10)))
By no means more elegant, but:
[x() for x in [foo]*10]
I think beyond that you have to go to Ruby ;)
map(lambda _ : foo(), range(10))
although this trades your problem with a meaningless iterator i with a new meaningless argument to the lambda expression.