Related
I can find lots of stuff showing me what a lambda function is, and how the syntax works and what not. But other than the "coolness factor" (I can make a function in middle a call to another function, neat!) I haven't seen something that's overwelmingly compelling to say why I really need/want to use them.
It seems to be more of a stylistic or structual choice in most examples I've seen. And kinda breaks the "Only one correct way to do something" in python rule. How does it make my programs, more correct, more reliable, faster, or easier to understand? (Most coding standards I've seen tend to tell you to avoid overly complex statements on a single line. If it makes it easier to read break it up.)
Here's a good example:
def key(x):
return x[1]
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key)
versus
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=lambda x: x[1])
From another angle: Lambda expressions are also known as "anonymous functions", and are very useful in certain programming paradigms, particularly functional programming, which lambda calculus provided the inspiration for.
http://en.wikipedia.org/wiki/Lambda_calculus
The syntax is more concise in certain situations, mostly when dealing with map et al.
map(lambda x: x * 2, [1,2,3,4])
seems better to me than:
def double(x):
return x * 2
map(double, [1,2,3,4])
I think the lambda is a better choice in this situation because the def double seems almost disconnected from the map that is using it. Plus, I guess it has the added benefit that the function gets thrown away when you are done.
There is one downside to lambda which limits its usefulness in Python, in my opinion: lambdas can have only one expression (i.e., you can't have multiple lines). It just can't work in a language that forces whitespace.
Plus, whenever I use lambda I feel awesome.
For me it's a matter of the expressiveness of the code. When writing code that people will have to support, that code should tell a story in as concise and easy to understand manner as possible. Sometimes the lambda expression is more complicated, other times it more directly tells what that line or block of code is doing. Use judgment when writing.
Think of it like structuring a sentence. What are the important parts (nouns and verbs vs. objects and methods, etc.) and how should they be ordered for that line or block of code to convey what it's doing intuitively.
Lambda functions are most useful in things like callback functions, or places in which you need a throwaway function. JAB's example is perfect - It would be better accompanied by the keyword argument key, but it still provides useful information.
When
def key(x):
return x[1]
appears 300 lines away from
[(1,2), (3,1), (5,10), (11,-3)].sort(key)
what does key do? There's really no indication. You might have some sort of guess, especially if you're familiar with the function, but usually it requires going back to look. OTOH,
[(1,2), (3,1), (5,10), (11,-3)].sort(lambda x: x[1])
tells you a lot more.
Sort takes a function as an argument
That function takes 1 parameter (and "returns" a result)
I'm trying to sort this list by the 2nd value of each of the elements of the list
(If the list were a variable so you couldn't see the values) this logic expects the list to have at least 2 elements in it.
There's probably some more information, but already that's a tremendous amount that you get just by using an anonymous lambda function instead of a named function.
Plus it doesn't pollute your namespace ;)
Yes, you're right — it is a structural choice. It probably does not make your programs more correct by just using lambda expressions. Nor does it make them more reliable, and this has nothing to do with speed.
It is only about flexibility and the power of expression. Like list comprehension. You can do most of that defining named functions (possibly polluting namespace, but that's again purely stylistic issue).
It can aid to readability by the fact, that you do not have to define a separate named function, that someone else will have to find, read and understand that all it does is to call a method blah() on its argument.
It may be much more interesting when you use it to write functions that create and return other functions, where what exactly those functions do, depends on their arguments. This may be a very concise and readable way of parameterizing your code behaviour. You can just express more interesting ideas.
But that is still a structural choice. You can do that otherwise. But the same goes for object oriented programming ;)
Ignore for a moment the detail that it's specifically anonymous functions we're talking about. functions, including anonymous ones, are assignable quantities (almost, but not really, values) in Python. an expression like
map(lambda y: y * -1, range(0, 10))
explicitly mentions four anonymous quantities: -1, 0, 10 and the result of the lambda operator, plus the implied result of the map call. it's possible to create values of anonymous types in some languages. so ignore the superficial difference between functions and numbers. the question when to use an anonymous function as opposed to a named one is similar to a question of when to put a naked number literal in the code and when to declare a TIMES_I_WISHED_I_HAD_A_PONY or BUFFER_SIZE beforehand. there are times when it's appropriate to use a (numeric, string or function) literal, and there are times when it's more appropriate to name such a thing and refer to it through its name.
see eg. Allen Holub's provocative, thought-or-anger-provoking book on Design Patterns in Java; he uses anonymous classes quite a bit.
Lambda, while useful in certain situations, has a large potential for abuse. lambda's almost always make code more difficult to read. And while it might feel satisfying to fit all your code onto a single line, it will suck for the next person who has to read your code.
Direct from PEP8
"One of Guido's key insights is that code is read much more often than it is written."
It is definitely true that abusing lambda functions often leads to bad and hard-to-read code. On the other hand, when used accurately, it does the opposite. There are already great answers in this thread, but one example I have come across is:
def power(n):
return lambda x: x**n
square = power(2)
cubic = power(3)
quadruple = power(4)
print(square(10)) # 100
print(cubic(10)) # 1000
print(quadruple(10)) # 10000
This simplified case could be rewritten in many other ways without the use of lambda. Still, one can infer how lambda functions can increase readability and code reuse in perhaps more complex cases and functions with this example.
Lambdas are anonymous functions (function with no name) that can be assigned to a variable or that can be passed as an argument to another function. The usefulness of lambda will be realized when you need a small piece of function that will be run once in a while or just once. Instead of writing the function in global scope or including it as part of your main program you can toss around few lines of code when needed to a variable or another function. Also when you pass the function as an argument to another function during the function call you can change the argument (the anonymous function) making the function itself dynamic. Suppose if the anonymous function uses variables outside its scope it is called closure. This is useful in callback functions.
One use of lambda function which I have learned, and where is not other good alternative or at least looks for me best is as default action in function parameter by
parameter=lambda x: x
This returns the value without change, but you can supply one function optionally to perform a transformation or action (like printing the answer, not only returning)
Also often it is useful to use in sorting as key:
key=lambda x: x[field]
The effect is to sort by fieldth (zero based remember) element of each item in sequence. For reversing you do not need lambda as it is clearer to use
reverse=True
Often it is almost as easy to do new real function and use that instead of lambda. If people has studied much Lisp or other functional programming, they also have natural tendency to use lambda function as in Lisp the function definitions are handled by lambda calculus.
Lambdas are objects, not methods, and they cannot be invoked in the same way that methods are.
for e.g
succ = ->(x){ x+1 }
succ mow holds a Proc object, which we can use like any other:
succ.call(2)
gives us an output = 3
I want to point out one situation other than list-processing where the lambda functions seems the best choice:
from tkinter import *
from tkinter import ttk
def callback(arg):
print(arg)
pass
root = Tk()
ttk.Button(root, text = 'Button1', command = lambda: callback('Button 1 clicked')).pack()
root.mainloop()
And if we drop lambda function here, the callback may only execute the callback once.
ttk.Button(root, text = 'Button1', command = callback('Button1 clicked')).pack()
Another point is that python does not have switch statements. Combining lambdas with dicts can be an effective alternative. e.g.:
switch = {
'1': lambda x: x+1,
'2': lambda x: x+2,
'3': lambda x: x+3
}
x = starting_val
ans = expression
new_ans = switch[ans](x)
In some cases it is much more clear to express something simple as a lambda. Consider regular sorting vs. reverse sorting for example:
some_list = [2, 1, 3]
print sorted(some_list)
print sorted(some_list, lambda a, b: -cmp(a, b))
For the latter case writing a separate full-fledged function just to return a -cmp(a, b) would create more misunderstanding then a lambda.
Lambdas allow you to create functions on the fly. Most of the examples I've seen don't do much more than create a function with parameters passed at the time of creation rather than execution. Or they simplify the code by not requiring a formal declaration of the function ahead of use.
A more interesting use would be to dynamically construct a python function to evaluate a mathematical expression that isn't known until run time (user input). Once created, that function can be called repeatedly with different arguments to evaluate the expression (say you wanted to plot it). That may even be a poor example given eval(). This type of use is where the "real" power is - in dynamically creating more complex code, rather than the simple examples you often see which are not much more than nice (source) code size reductions.
you master lambda, you master shortcuts in python.Here is why:
data=[(lambda x:x.text)(x.extract()) for x in soup.findAll('p') ]
^1 ^2 ^3 ^4
here we can see 4 parts of the list comprehension:
1: i finally want this
2: x.extract will perform some operation on x, here it pop the element from soup
3: x is the list iterable which is passed to the input of lambda at 2 along with extract operation
4: some arbitary list
i had found no other way to use 2 statements in lambda, but with this
kind of pipe-lining we can exploit the infinite potential of lambda.
Edit: as pointed out in the comments, by juanpa, its completely fine to use x.extract().text but the point was explaining the use of lambda pipe, ie passing the output of lambda1 as input to lambda2. via (lambda1 y:g(x))(lambda2 x:f(x))
I have some arguments taken from the user and passed along function to function (each function in a different class), until it eventually gets to a function that does some processing and then the solution is returned up the chain. Up the chain, the functions become more and more abstract merging results from multiple runs of the lower functions.
Where should I use *args and **kwargs?
I think *args and *kwargs can be used for every function where the function doesn't use the arguments explicitly. But, the actual arguments need to be defined at the top_level so that the user knows what the function expects.
Where should I define what the inputs mean?
I think they should be defined at the top_level because that's the one the end-user might want to see the documentation for.
Where should I define what default values?
Again, I think they should be defined at the top_level because that's the one the end-user interacts with.
This is a simple example to demonstrate the passing of the arguments, where I haven't shown how the functions become more and more abstract or how they interact with different classes, as I felt it was unnecessary detail.
def top_level(a=1, b=1, c=1, d=1, e=1):
""" Compute sum of five numbers.
:param a: int, a
:param b: int, b
:param c: int, c
:param d: int, d
:param e: int, e
:return: int, sum
"""
return mid_level(a, b, c, d, e)
def mid_level(*args, **kwargs):
return bottom_level(*args, **kwargs)
def bottom_level(a, b, c, d, e):
return a + b + c + d + e
print top_level(1, 2, 3)
8
Is there a Python convention for passing arguments like this?
I'm not going to answer your question because it would be like answering the question "what's the best way to use a screwdriver to tighten a nut?". I.e. I do not believe that the tools you are asking for guidance with (*args and **kwargs) are designed to solve the problem you want to solve.
Instead I'll answer this question: "how do I associate some data with a set of functions?", and the answer to that is clearly Use Classes.
Welcome to object-oriented programming. I think you're going to enjoy it!
This is a very basic example of what I mean, but it was hard to know exactly what you wanted from your example since it was simple, but the basic principle is encapsulate your data in a class, and then operate on it using the class's methods.
You can then call between methods in the class without needing to pass loads of arguments around all the time (such as the .calculate() method below), which you don't know whether the top layer will need or a bottom layer.
You can just document the parameters in one place, the __init__ method.
You can customize through subclassing transparently to the code (because if you override a method in a subclass, it can still be used by the more generic superclass), as I've done for the .reduce(x, y) method below.
Example:
class ReductionCalculator:
def __init__(self, *args):
self.args = args
def calculate(self):
start = self.args[0]
for arg in self.args[1:]:
start = self.reduce(start, arg)
return start
class Summer(ReductionCalculator):
def reduce(self, x, y):
return x + y
class Multiplier(ReductionCalculator):
def reduce(self, x, y):
return x * y
summer = Summer(1, 2, 4)
print('sum: %d' % (summer.calculate(),))
multiplier = Multiplier(1, 2, 4)
print('sum: %d' % (multiplier.calculate(),))
How about this approach: create a class, call it AllInputs, that represents the collection of all the "arguments taken from the user." The only purpose of this class is to serve as a container for a set of values. One instance of this class gets initialized, of course, at the top level of the program.
class AllInputs:
def __init__(self,a=1, b=1, c=1, d=1, e=1):
""" Compute sum of five numbers.
:param a: int, a
:param b: int, b
:param c: int, c
:param d: int, d
:param e: int, e
"""
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
This object, call it all_inputs, is now passed as the single argument to all of the functions in your example. If a function doesn't use any of the fields in the object, that's fine; it just passes it along to the lower-level function where the real work gets done. To refactor your example, you would now have:
def top_level(all_inputs):
""" Compute sum of all inputs
:return: int, sum
"""
return mid_level(all_inputs)
def mid_level(all_inputs):
return bottom_level(all_inputs)
def bottom_level(all_inputs):
return (all_inputs.a + all_inputs.b + all_inputs.c +
all_inputs.d + all_inputs.e)
all_inputs = AllInputs(1, 2, 3)
print top_level(all_inputs)
8
I don't know if this is "Pythonic" or "non-Pythonic" and I don't care. I think it's a good programming idea to group together the data that the program will use. The initialization process, which combines default values with others taken from the user, is centralized in one place where it's easy to understand. It's reasonably self-documenting. You say the function calls are distributed across several classes, and that's no problem. The function calls are clean and the program flow is easy to follow. There is potential for optimization by placing some of the calculation inside AllInputs so you can avoid duplicating code.
What I don't like in your example (and I think you don't like it either, or you probably wouldn't have asked the question in the first place) is how it uses the *args syntax. When I see that syntax, I take it as a hint that all the arguments have the same semantic meaning, like in the standard library function os.path.join. In your application, if I understand the question, the low-level functions require the argument list to be in a specific order and have specific meanings (your example doesn't reflect that but the text suggests it). It's confusing to see arguments that get passed into a function as *args and then, at a lower level, their specific names and meanings appear once again. Grouping them into a single object makes it clear what's going on.
This isn't the most common pattern, but I've seen it for command line programs that have levels of nested commands: sub-commands, sub-sub-commands and so on. That's a model where "upper" level functions may be more or less dispatchers and not have information about what parameters are needed by the sub-functions within a given route. The purest scenario for this model is when the sub-commands are plugins and the "upper" layers have literally no information about the sub-functions, other than a calling convention the plug-ins are expected to adhere to.
In these cases, I'd argue the pythonic way is to pass parameters from higher-level to lower-level functions, and let the worker level decide which are useful. The range of possible parameters would be defined in the calling convention. This is pythonic on the basis of DRY -- don't repeat yourself. If the low-level / worker function defines what inputs are required or optional, it would often make sense to not repeat this information at the higher levels.
The same could be said for any inversion-of-control flow design, not just CLI applications w/ plug-ins. There are many application designs where I wouldn't use this approach, but it works here.
An input's meaning must be set at the topmost level it can arise in -- as an interface spec to lower levels (a convention, not programmatic). Otherwise the inputs would have no semantic meaning.
If an input can be used by multiple sub-functions, i.e. there's a chaining or pipeline concept in the control flow, then an input's default will also need to be defined at the topmost level for the input.
I would argue that passing arguments down several levels of functions is not pythonic in itself.
From the Zen of Python:
Simple is better than complex
Flat is better than nested
Edit:
If there are a lot of arguments and the functions inbetween just pass them down, I would probably wrap them up in a tuple and unwrap them at the lowest level.
operator provides attrgetter to make a function that retrieves a field from an object.
Why isn't this included in operator (or somewhere else in the standard libraries)?
def attrsetter(name):
def setter( obj, val):
setattr(obj, name, val)
return setter
The only reason that I can think of is that there are edge cases where this straightforward approach will break. In which case, what are these edge cases so that I can try to trap/avoid them?
attrgetter is designed to be used in places where a function is required as a replacement for lambda. For example:
# equivalent
heads = map(attrgetter('head'), objs)
heads = map(lambda o: o.head, objs)
In other words, the point of attrgetter is to create a side-effect-free function that returns a useful value, and which can be used in expressions that require a function. An attrsetter, on the other hand, would only operate by side effect, and would need to return None by Python convention. Since attrsetter would not be at all useful as argument to map and similar, it is not provided. If you need attrsetter, simply write a normal for loop.
Also note that both of the above idioms are better expressed with a list comprehension:
heads = [o.head for o in objs]
attrgetter is rarely needed and has lost much of its appeal once it was decided that lambda would not be removed from Python 3 after all.
In Python, for a simple function foo(x, y) there are at least 3 ways that i know to bind the argument y to some value
# defining a nested function:
def foobar(x):
return foo(x, y=yval)
# using lambda
foobar = lambda x: foo(x, y=yval)
# using functools
from functools import partial
foobar = partial(foo, y=yval)
while i am doubtful that the list above is exhaustive, i also wonder which one should i go with? are they all equivalent in terms of performance, safety and namespace handling? or are there extra overheads and caveats with each method? why should functools define partial when the other methods are already there?
No, they're not all equivalent -- in particular, a lambda cannot be pickled and a functools.partial can, IIRC, be pickled only in recent Python versions (I can't find which exact version in the docs; it doesn't work in 2.6, but it does in 3.1). Neither can functions defined inside of other functions (neither in 2.6 nor 3.1).
The reason for partial's appearance in the library is that it gives you an explicit idiom to partially apply a function inline. A definition (def) cannot appear in the middle of an expression such as
map(partial(foo, y=yval), xs)
Also, from a definition or lambda, it's not immediately clear that partial application is what's going on, since you can put an arbitrary expression in a lambda and arbitrary statements in a definition.
I suggest you go with partial unless you have a reason not to use it.
[And indeed, the list is not exhaustive. The first alternative that comes to mind is a callable object:
class foobar:
def __init__(self, yval):
self.__yval = yval
def __call__(self, x):
return foo(x, self.__yval)
but those are heavy-weights for such a simple problem.]
Is there any reason that will make you use:
add2 = lambda n: n+2
instead of :
def add2(n): return n+2
I tend to prefer the def way but every now and then I see the lambda way being used.
EDIT :
The question is not about lambda as unnamed function, but about lambda as a named function.
There is a good answer to the same question here.
lambda is nice for small, unnamed functions but in this case it would serve no purpose other than make functionalists happy.
I usually use a lambda for throwaway functions I'm going to use only in one place, for example as an argument to a function. This keeps the logic together and avoids filling the namespace with function names I don't need.
For example, I think
map(lambda x: x * 2,xrange(1,10))
is neater than:
def bytwo(x):
return x * 2
map(bytwo,xrange(1,10))
Guido Van Rossum (the creator of Python) prefers def over lambdas (lambdas only made it to Python 3.0 at the last moment) and there is nothing you can't do with def that you can do with lambdas.
People who like the functional programming paradigm seem to prefer them. Some times using an anonymous function (that you can create with a lambda expression) gives more readable code.
I recall David Mertz writing that he preferred to use the form add2 = lambda n: n+2 because it emphasised that add2 is a pure function that has no side effects. However I think 99% of Python programmers would use the def statement and only use lambda for anonymous functions, since that is what it is intended for.
One place you can't use def is in expressions: A def is a statement. So that is about the only place I'd use lambda.