So I am trying to figure out how Python's map() function works as a way to speed up my program a little bit. From my basic understanding it looks like you can use map() to replace certain instances where you'd use a for loop. What I'm curious about is can you change something like:
loopNum = 25
for i in range (loopNum):
self.doSomething()
To:
loopNum = 25
map(self.doSomething(), range(loopNum))
Additionally, in the above example, would I be able to forego that loopNum variable, and in the map just have map(something, 25)?
No, you can't as map(function, iterable) applies function to each element of the iterable. If you simply want to execute some functionn times, just use a loop.
Note that iterable must be (surprise!) an iterable, not a number.
map is roughly the equivalent of this for loop:
# my_map(func, iter1, iterN...)
def my_map(func, *iteables)
for x, y,... in zip(iter1, iter2,iterN...):
yield func(x,y,...)
What you're doing in your code is just like this:
my_map(self.doSomething(), range(loopNum))
self.dSomething() must return a function or a callable object or this obviously doesn't work. This is because, whatever object you pass into the func argument of my_map function will be called then in addition to passing the right number of arguments to func, func must be a callable object as well :-)
*You must iterate over the iterable returned by map to obtain the results, otherwise, you would just get an iterable object without tangible work.
I want to add to this question that map() is virtually never the right tool in Python. Our BDFL himself, wanted to remove it from Python 3, together with lambdas and, most forcefully reduce.
In almost all cases where you feel tempted to use map() take a step back and try to rewrite your code using list comprehension instead. For your current example, that would be:
my_list = [self.doSomething() for i in range(loopNum)]
or
my_generator = (self.doSomething() for i in range(loopNum))
to make it a generator instead.
But for this to make any sense at all, self.doSomething() should probably take the variable i as an input. After all you are trying to map the values in your range to something else.
Related
What i have understood so far from the map function is that it applies a given function to each item in an iterable and returns the list of the result.
def square(num):
return num**2
list_num = [1,2,3,4]
considering the above function and the list of numbers as an example.
if we: list(map(square,list_num)) we will get as an output [1,4,9,16].
now comes the part that i am not able to find a sensible explenation of, if i
print(map(square,list_num)) i will get as an output <map object at 0x038B0290>.
My question is, Why am i getting the memory location and not a list when i use the print() function or when use map(square,list_num).
map doesn't return a list. It returns a map object that lazily produces results as needed:
print(type(map(int, [1])))
<class 'map'>
It also doesn't override the stringify method which would produce pretty-printed results. That may be because that would require forcing the entire list to be evaluated, and if that's acceptable in your program, you probably shouldn't be using map in the first place; unless you're debugging, in which case use of list is probably fine unless you're dealing with an infinite list.
If you want a full-element print out, explicitly force it by placing it in a list before printing as you saw, or use a strict list production method like a list comprehension.
I have a "best practices" question here. I am using map in a way that it may not be intended to be used - using the elements of a list to change the state of a different object. the final list output is not actually changed. Is this appropriate?
For example:
class ToBeChanged(object):
def __init__(self):
self.foo_lst = [1,2,3,4]
def mapfunc(self, arg):
if arg in ['foo', 'bar']:
self.foo_lst.append(arg)
else:
pass
test = ToBeChanged()
list_to_map = [1,2,37,'foo']
map(lambda x: test.mapfunc(x), list_to_map)
It is not appropriate. In Python 2, you'll be creating a new list of the same length as list_to_map, and immediately discarding it; waste! And the lambda even makes it more complicated.
Better to use a for loop:
for x in list_to_map:
test.mapfunc(x)
More concise and readable.
And if you're still thinking of using this in Python 3 (by forcing the lazy object to be evaluated in some way), consider those who will maintain your code; map gives the impression you want to create a new iterable from the list.
map is the worst.
Because if you tried to run the code in python3 it wouldn't even perform the calls since in python3 map is lazy.
In any case both calling map or a list-comprehension are expressions and expressions should be as side-effect free as possible, their purpose is to return a value.
So if you don't have a value to return you should just use the plain statements: i.e. explicit for
What is considered to be a better programming practice when dealing with more object at time (but with the option to process just one object)?
A: LOOP INSIDE FUNCTION
Function can be called with one or more objects and it is iterating inside function:
class Object:
def __init__(self, a, b):
self.var_a = a
self.var_b = b
var_a = ""
var_b = ""
def func(obj_list):
if type(obj_list) != list:
obj_list = [obj_list]
for obj in obj_list:
# do whatever with an object
print(obj.var_a, obj.var_b)
obj_list = [Object("a1", "a2"), Object("b1", "b2")]
obj_alone = Object("c1", "c2")
func(obj_list)
func(obj_alone)
B: LOOP OUTSIDE FUNCTION
Function is dealing with one object only and when it is dealing with more objects in must be called multiple times.
class Object:
def __init__(self, a, b):
self.var_a = a
self.var_b = b
var_a = ""
var_b = ""
def func(obj):
# do whatever with an object
print(obj.var_a, obj.var_b)
obj_list = [Object("a1", "a2"), Object("b1", "b2")]
obj_alone = Object("c1", "c2")
for obj in obj_list:
func(obj)
func(obj_alone)
I personally like the first one (A) more, because for me it makes cleaner code when calling the function, but maybe it's not the right approach. Is there some method generally better than the other? And if not, what are the cons and pros of each method?
A function should have a defined input and output and follow the single responsibility principle. You need to be able to clearly define your function in terms of "I put foo in, I get bar back". The more qualifiers you need to make in this statement to properly describe your function probably means your function is doing too much. "I put foo in and get bar back, unless I put baz in then I also get bar back, unless I put a foo-baz in then it'll error".
In this particular case, you can pass an object or a list of objects. Try to generalise that to a value or a list of values. What if you want to pass a list as a value? Now your function behaviour is ambiguous. You want the single list object to be your value, but the function treats it as multiple arguments instead.
Therefore, it's trivial to adapt a function which takes one argument to work on multiple values in practice. There's no reason to complicate the function's design by making it adaptable to multiple arguments. Write the function as simple and clearly as possible, and if you need it to work through a list of things then you can loop it through that list of things outside the function.
This might become clearer if you try to give an actual useful name to your function which describes what it does. Do you need to use plural or singular terms? foo_the_bar(bar) does something else than foo_the_bars(bars).
Move loops outside functions (when possible)
Generally speaking, keep loops that do nothing but iterate over the parameter outside of functions. This gives the caller maximum control and assumes the least about how the client will use the function.
The rule of thumb is to use the most minimal parameter complexity that the function needs do its job.
For example, let's say you have a function that processes one item. You've anticipated that a client might conceivably want to process multiple items, so you changed the parameter to an iterable, baked a loop into the function, and are now returning a list. Why not? It could save the client from writing an ugly loop in the caller, you figure, and the basic functionality is still available -- and then some!
But this turns out to be a serious constraint. Now the caller needs to pack (and possibly unpack, if the function returns a list of results in addition to a list of arguments) that single item into a list just to use the function. This is confusing and potentially expensive on heap memory:
>>> def square(it): return [x ** 2 for x in it]
...
>>> square(range(6)) # you're thinking ...
[0, 1, 4, 9, 16, 25]
>>> result, = square([3]) # ... but the client just wants to square 1 number
>>> result
9
Here's a much better design for this particular function, intuitive and flexible:
>>> def square(x): return x ** 2
...
>>> square(3)
9
>>> [square(x) for x in range(6)]
[0, 1, 4, 9, 16, 25]
>>> list(map(square, range(6)))
[0, 1, 4, 9, 16, 25]
>>> (square(x) for x in range(6))
<generator object <genexpr> at 0x00000166D122CBA0>
>>> all(square(x) % 2 for x in range(6))
False
This brings me to a second problem with the functions in your code: they have a side-effect, print. I realize these functions are just for demonstration, but designing functions like this makes the example somewhat contrived. Functions typically return values rather than simply produce side-effects, and the parameters and return values are often related, as in the above example -- changing the parameter type bound us to a different return type.
When does it make sense to use an iterable argument? A good example is sort -- the smallest unit of operation for a sorting function is an iterable, so the problem of packing and unpacking in the square example above is a non-issue.
Following this logic a step further, would it make sense for a sort function to accept a list (or variable arguments) of lists? No -- if the caller wants to sort multiple lists, they should loop over them explicitly and call sort on each one, as in the second square example.
Consider variable arguments
A nice feature that bridges the gap between iterables and single arguments is support for variable arguments, which many languages offer. This sometimes gives you the best of both worlds, and some functions go so far as to accept either args or an iterable:
>>> max([1, 3, 2])
3
>>> max(1, 3, 2)
3
One reason max is nice as a variable argument function is that it's a reduction function, so you'll always get a single value as output. If it were a mapping or filtering function, the output is always a list (or generator) so the input should be as well.
To take another example, a sort routine wouldn't make much sense with varargs because it's a classically in-place algorithm that works on lists, so you'd need to unpack the list into the arguments with the * operator pretty much every time you invoke the function -- not cool.
There's no real need for a call like sort(1, 3, 4, 2) as there is with max, where the parameters are just as likely to be loose variables as they are a packed iterable. Varargs are usually used when you have a small number of arguments, or the thing you're unpacking is a small pair or tuple-type element, as often the case with zip.
There's definitely a "feel" to when to offer parameters as varargs, an iterable, or a single value (i.e. let the caller handle looping), but as long as you follow the rule of avoiding iterables unless they're essential to the function, it's hard to go wrong.
As a final tip, try to write your functions with similar contracts to the library functions in your language or the tools you use frequently. These are pretty much always designed well; mimic good design.
If you implement B then you will make it harder for yourself to achieve A.
If you implement A then it isn't too difficult to achieve B. You also have many tools already available to apply this function to a list of arguments (the loop method you described, using something like map, or even a multiprocessing approach if needed)
Therefore I would choose to implement A, and if it makes things neater or easier in a given case you can think about also implementing B (using A) also so that you have both.
I have a method that returns either a list or a tuple. What is the most pythonic way of denoting the return type in the argument?
def names(self, section, as_type=()):
return type(as_type)(([m[0] for m in self.items(section)]))
The pythonic way would be not to care about the type at all. Return a tuple, and if the calling function needs a list, then let it call list() on the result. Or vice versa, whichever makes more sense as a default type.
Even better, have it return a generator expression:
def names(self, section):
return (m[0] for m in self.items(section))
Now the caller gets an iterable that is evaluated lazily. He then can decide to iterate over it:
for name in obj.names(section):
...
or create a list or tuple from it from scratch - he never has to change an existing list into a tuple or vice versa, so this is efficient in all cases:
mylist = list(obj.names(section))
mytuple = tuple(obj.names(section))
Return whatever the caller will want most of the time. If they will want to be able to sort, remove or delete items, etc. then use a list. If they will want to use it as a dictionary key, use a tuple. If the primary use will be iteration, return an iterator. If it doesn't matter to the caller, which it won't more often than you might think, then return whatever makes the code the most straightforward. Usually this will be a list or an iterator.
Don't provide your own way to convert the output to a given type. Python has a perfectly simple way to do this already and any programmer using your function will be familiar with it. Look at the standard Python library. Do any of those routines do this? No, because there's no reason to.
Exception: sometimes there's a way to get an iterator or a list, even though it is easy to convert an iterator to a list. Usually this capability is provided as two separate functions or methods. Maybe you might want to follow suit sometimes, especially if you could implement the two alternatives using different algorithms that provide some clear benefit to callers who want one or another.
Keep it simple:
def names(self, section):
"""Returns a list of names."""
return [m[0] for m in self.items(section)]
If the caller wants a tuple instead of a list, he does this:
names = tuple(obj.names(section))
In python I have the following function:
def is_a_nice_element(element, parameter):
#do something
return True or False
Now I would like to filter a list with this function as predicate, giving a fixed parameter. Python has the itertools.ifilter function, but I can't figure out how to pass the parameter. Is this possible? If not, how can I solve this problem?
I like functools.partial much better than lambda.
itertools.ifilter( partial(is_a_nice_element, parameter=X), iterable )
Wrap it in a lambda:
itertools.ifilter(lambda e: is_a_nice_element(e, 42), iterable)
42 is your extra argument, or whatever else you want it to be.
The solutions use lambda and functools.partial are quite correct and directly answer your question, but I don't think they are really the ideal solution.
Using filter/itertools.ifilter/map/itertools.imap with an anonymous function is prettymuch always going to be less clear than using a generator expression or list comprehension. For example, in this case I would write the generator expression
(item for item in iterable if is_a_nice_element(item, constant))
instead of using ifilter. In this case the ifilter solution is still fairly readable, but in many cases trying to go through lots of trouble to define just the right function is not going to be as worth while as just performing the operation.
If you are using ifilter then parameter would need to be constant in which case you could use a default argument parameter=something. If you want parameter to vary, you'd need to use another method to take a dyadic predicate.
If you already have the list in hand, ifilter is a bit of overkill relative to the built-in filter.
Describing what you want to accomplish would help in making a more concrete answer.