I want to dynamically inspect the body of a lambda that specialises another lambda at runtime. This would need to happen recursively until there are no more "parent" lambdas.
What I mean by that is the following example:
add_x_z = lambda x, z: x + z
add_x_1 = lambda x: add_x_z(x, 1)
Here add_x_1 is what I refer to as a "specialisation" of add_x_z (the "parent" lambda). The specialisation here is hardcoded (z = 1) but you can imagine cases where the fixed value comes from the runtime.
In this case, I am looking for the following string as an output:
"add_x_1 = lambda x: x + 1"
There are a lot of questions on Stack Overflow about inspecting the body of functions/lambdas, the most widely-accepted solution accross all those Q&As is the following:
add_x_z = lambda x, z: x + z
add_x_1 = lambda x: add_x_z(x, 1)
print(inspect.getsource(add_x_1))
# add_x_1 = lambda x: add_x_z(x, 1)
This doesn't do the trick for me because inspect under the hood only looks at the source files and doesn't care about the runtime. There are also various solutions floating around that suggest using packages such as uncompyle6, this also doesn't solve my issue:
import uncompyle6
add_x_z = lambda x, z: x + z
add_x_1 = lambda x: add_x_z(x, 1)
uncompyle6.deparse_code2str(add_x_1.__code__)
# return add_x_z(x, 1)
Is there any way to achieve this through Python internals and that doesn't involve writing some kind of parser?
Bonus points for an answer that doesn't make use of exec or eval
You might try using a debugger and stopping where you need to and inspect values. Most debuggers have some sort of command that allows you to show some sort of Python expression based on existing values. In trepan and other debuggers you can go into a python shell, and then create your own code based on existing variables and this may be useful for macro expanding lambdas.
And the trepan debugger allows its own kind of scriptable macros so that you can string simple debugger commands together to assist in piecing together macro expansion based on lambda parameters.
The other tool that I can think of that may be of help here might be x-python because (for simpler Python programs) you can step to where you want and then look at evaluation stack entries. There is a "trepan" debugger for that as well, so you can use this in conjunction with writing macros.
Or in x-python you could replace Python's "lambda" so that it also macro expands it and prints in addition to running it.
Note that this might not make use of exec or eval in the answer, unless some sort of debugger print or eval is done. And it feels like, for the kind the code in question, there may be some flavor of this.
In sum though, I have to say that if what you want is something general and complicated like this, you should expect to write some code including a parser or adapt one of the existing tools mentioned here (and not mentioned here).
I am trying to convert a code from Matlab to Python in which I came across a function named mynorm(x,y) in Matlab which I want to convert to Python. I searched equivalent for this function in Python but was not successful. So tried to find the implementation of this function on the net and found a small mynorm.m file which contains the function with just one-liner code which is as follows:
function L = mynorm(x,y)
%length of the vector [x,y]
L = sqrt(x^2 + y^2);
%note - if the input was a vector v, a better way to do this would be
%L = sqrt(dot(v,v))
%see help dot for the dot product. This would work for vectors of any size.
But when I looked into the call which is made to this function in matlab file, it is as follows:
feaNorm = mynorm(fea2, 1)
feaNorm = mynorm(iris(:,1:4),2);
which doesn't really look like length function as in the above implementation.
Thus, I am skeptical to use sqrt function in Python for this function call.
Can someone redirect me to the correct implementation or the equivalent python code?
I suggest you use the hypot trigonometric function available in the math module. You will find a description of the math module here.
I can find lots of stuff showing me what a lambda function is, and how the syntax works and what not. But other than the "coolness factor" (I can make a function in middle a call to another function, neat!) I haven't seen something that's overwelmingly compelling to say why I really need/want to use them.
It seems to be more of a stylistic or structual choice in most examples I've seen. And kinda breaks the "Only one correct way to do something" in python rule. How does it make my programs, more correct, more reliable, faster, or easier to understand? (Most coding standards I've seen tend to tell you to avoid overly complex statements on a single line. If it makes it easier to read break it up.)
Here's a good example:
def key(x):
return x[1]
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key)
versus
a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=lambda x: x[1])
From another angle: Lambda expressions are also known as "anonymous functions", and are very useful in certain programming paradigms, particularly functional programming, which lambda calculus provided the inspiration for.
http://en.wikipedia.org/wiki/Lambda_calculus
The syntax is more concise in certain situations, mostly when dealing with map et al.
map(lambda x: x * 2, [1,2,3,4])
seems better to me than:
def double(x):
return x * 2
map(double, [1,2,3,4])
I think the lambda is a better choice in this situation because the def double seems almost disconnected from the map that is using it. Plus, I guess it has the added benefit that the function gets thrown away when you are done.
There is one downside to lambda which limits its usefulness in Python, in my opinion: lambdas can have only one expression (i.e., you can't have multiple lines). It just can't work in a language that forces whitespace.
Plus, whenever I use lambda I feel awesome.
For me it's a matter of the expressiveness of the code. When writing code that people will have to support, that code should tell a story in as concise and easy to understand manner as possible. Sometimes the lambda expression is more complicated, other times it more directly tells what that line or block of code is doing. Use judgment when writing.
Think of it like structuring a sentence. What are the important parts (nouns and verbs vs. objects and methods, etc.) and how should they be ordered for that line or block of code to convey what it's doing intuitively.
Lambda functions are most useful in things like callback functions, or places in which you need a throwaway function. JAB's example is perfect - It would be better accompanied by the keyword argument key, but it still provides useful information.
When
def key(x):
return x[1]
appears 300 lines away from
[(1,2), (3,1), (5,10), (11,-3)].sort(key)
what does key do? There's really no indication. You might have some sort of guess, especially if you're familiar with the function, but usually it requires going back to look. OTOH,
[(1,2), (3,1), (5,10), (11,-3)].sort(lambda x: x[1])
tells you a lot more.
Sort takes a function as an argument
That function takes 1 parameter (and "returns" a result)
I'm trying to sort this list by the 2nd value of each of the elements of the list
(If the list were a variable so you couldn't see the values) this logic expects the list to have at least 2 elements in it.
There's probably some more information, but already that's a tremendous amount that you get just by using an anonymous lambda function instead of a named function.
Plus it doesn't pollute your namespace ;)
Yes, you're right — it is a structural choice. It probably does not make your programs more correct by just using lambda expressions. Nor does it make them more reliable, and this has nothing to do with speed.
It is only about flexibility and the power of expression. Like list comprehension. You can do most of that defining named functions (possibly polluting namespace, but that's again purely stylistic issue).
It can aid to readability by the fact, that you do not have to define a separate named function, that someone else will have to find, read and understand that all it does is to call a method blah() on its argument.
It may be much more interesting when you use it to write functions that create and return other functions, where what exactly those functions do, depends on their arguments. This may be a very concise and readable way of parameterizing your code behaviour. You can just express more interesting ideas.
But that is still a structural choice. You can do that otherwise. But the same goes for object oriented programming ;)
Ignore for a moment the detail that it's specifically anonymous functions we're talking about. functions, including anonymous ones, are assignable quantities (almost, but not really, values) in Python. an expression like
map(lambda y: y * -1, range(0, 10))
explicitly mentions four anonymous quantities: -1, 0, 10 and the result of the lambda operator, plus the implied result of the map call. it's possible to create values of anonymous types in some languages. so ignore the superficial difference between functions and numbers. the question when to use an anonymous function as opposed to a named one is similar to a question of when to put a naked number literal in the code and when to declare a TIMES_I_WISHED_I_HAD_A_PONY or BUFFER_SIZE beforehand. there are times when it's appropriate to use a (numeric, string or function) literal, and there are times when it's more appropriate to name such a thing and refer to it through its name.
see eg. Allen Holub's provocative, thought-or-anger-provoking book on Design Patterns in Java; he uses anonymous classes quite a bit.
Lambda, while useful in certain situations, has a large potential for abuse. lambda's almost always make code more difficult to read. And while it might feel satisfying to fit all your code onto a single line, it will suck for the next person who has to read your code.
Direct from PEP8
"One of Guido's key insights is that code is read much more often than it is written."
It is definitely true that abusing lambda functions often leads to bad and hard-to-read code. On the other hand, when used accurately, it does the opposite. There are already great answers in this thread, but one example I have come across is:
def power(n):
return lambda x: x**n
square = power(2)
cubic = power(3)
quadruple = power(4)
print(square(10)) # 100
print(cubic(10)) # 1000
print(quadruple(10)) # 10000
This simplified case could be rewritten in many other ways without the use of lambda. Still, one can infer how lambda functions can increase readability and code reuse in perhaps more complex cases and functions with this example.
Lambdas are anonymous functions (function with no name) that can be assigned to a variable or that can be passed as an argument to another function. The usefulness of lambda will be realized when you need a small piece of function that will be run once in a while or just once. Instead of writing the function in global scope or including it as part of your main program you can toss around few lines of code when needed to a variable or another function. Also when you pass the function as an argument to another function during the function call you can change the argument (the anonymous function) making the function itself dynamic. Suppose if the anonymous function uses variables outside its scope it is called closure. This is useful in callback functions.
One use of lambda function which I have learned, and where is not other good alternative or at least looks for me best is as default action in function parameter by
parameter=lambda x: x
This returns the value without change, but you can supply one function optionally to perform a transformation or action (like printing the answer, not only returning)
Also often it is useful to use in sorting as key:
key=lambda x: x[field]
The effect is to sort by fieldth (zero based remember) element of each item in sequence. For reversing you do not need lambda as it is clearer to use
reverse=True
Often it is almost as easy to do new real function and use that instead of lambda. If people has studied much Lisp or other functional programming, they also have natural tendency to use lambda function as in Lisp the function definitions are handled by lambda calculus.
Lambdas are objects, not methods, and they cannot be invoked in the same way that methods are.
for e.g
succ = ->(x){ x+1 }
succ mow holds a Proc object, which we can use like any other:
succ.call(2)
gives us an output = 3
I want to point out one situation other than list-processing where the lambda functions seems the best choice:
from tkinter import *
from tkinter import ttk
def callback(arg):
print(arg)
pass
root = Tk()
ttk.Button(root, text = 'Button1', command = lambda: callback('Button 1 clicked')).pack()
root.mainloop()
And if we drop lambda function here, the callback may only execute the callback once.
ttk.Button(root, text = 'Button1', command = callback('Button1 clicked')).pack()
Another point is that python does not have switch statements. Combining lambdas with dicts can be an effective alternative. e.g.:
switch = {
'1': lambda x: x+1,
'2': lambda x: x+2,
'3': lambda x: x+3
}
x = starting_val
ans = expression
new_ans = switch[ans](x)
In some cases it is much more clear to express something simple as a lambda. Consider regular sorting vs. reverse sorting for example:
some_list = [2, 1, 3]
print sorted(some_list)
print sorted(some_list, lambda a, b: -cmp(a, b))
For the latter case writing a separate full-fledged function just to return a -cmp(a, b) would create more misunderstanding then a lambda.
Lambdas allow you to create functions on the fly. Most of the examples I've seen don't do much more than create a function with parameters passed at the time of creation rather than execution. Or they simplify the code by not requiring a formal declaration of the function ahead of use.
A more interesting use would be to dynamically construct a python function to evaluate a mathematical expression that isn't known until run time (user input). Once created, that function can be called repeatedly with different arguments to evaluate the expression (say you wanted to plot it). That may even be a poor example given eval(). This type of use is where the "real" power is - in dynamically creating more complex code, rather than the simple examples you often see which are not much more than nice (source) code size reductions.
you master lambda, you master shortcuts in python.Here is why:
data=[(lambda x:x.text)(x.extract()) for x in soup.findAll('p') ]
^1 ^2 ^3 ^4
here we can see 4 parts of the list comprehension:
1: i finally want this
2: x.extract will perform some operation on x, here it pop the element from soup
3: x is the list iterable which is passed to the input of lambda at 2 along with extract operation
4: some arbitary list
i had found no other way to use 2 statements in lambda, but with this
kind of pipe-lining we can exploit the infinite potential of lambda.
Edit: as pointed out in the comments, by juanpa, its completely fine to use x.extract().text but the point was explaining the use of lambda pipe, ie passing the output of lambda1 as input to lambda2. via (lambda1 y:g(x))(lambda2 x:f(x))
If I have a string
equation = "1+2+3+4"
How do I change it into an int and equate the answer? I was thinking something like this but it would give an error.
answer = (int)equation
print(answer)
The equation could contain +-*/
If you are prepared to accept the risks, namely that you may be letting hostile users run whatever they like on your machine, you could use eval():
>>> equation = "1+2+3+4"
>>> eval(equation)
10
If your code will only ever accept input that you control, then this is the quickest and simplest solution. If you need to allow general input from users other than you, then you'd need to look for a more restrictive expression evaluator.
Update
Thanks to #roippi for pulling me up and pointing out that the code above executes in the environment of the calling code. So the expression would be evaluated with local and global variables available. Suppress that like so:
eval(equation, {'__builtins__': None})
This doesn't make the use of eval() secure. It just gives it a clean, empty environment in which to operate. A hostile user could still hose your system.
If we wanted to expose a calculator to the world, I'd be very interested to see if a cracker could work around this:
import string
def less_dangerous_eval(equation):
if not set(equation).intersection(string.ascii_letters + '{}[]_;\n'):
return eval(equation)
else:
print("illegal character")
return None
less_dangerous_eval('1*2/3^4+(5-6)')
returns:
3
I know that this can be broken by giving it bad syntax (fixable with a try/except block) or an operation that would take up all of the memory in the system (try/except catching this might be more iffy), but I am not currently aware of any way for someone to gain control.
This works if you only have digits and plusses:
answer = sum(float(i) for i in equation.split('+'))
or if you know they will only be integers
answer = sum(int(i) for i in equation.split('+'))
If you want to be able to evaluate more than that, I suggest you do your homework:
Look up the string module (which has string.digits)
the math module, which has your operations
create logic to perform the operations in proper order
Good luck!
I suggest you use eval and check the input:
def evaluate(s):
import string
for i in s:
if i not in "+-*/ " + string.digits:
return False # input is not valid
return eval(s)
As already mentioned, eval is unsafe, and very difficult to make safe. I remember seeing a very good blog post explaining this, does anyone know what it was? However ast.literal_eval is safe, it doesn't allow __import__ etc. I would strongly recomment using ast.literal_eval instead of eval whenever possible. Unfortunately, in this case it isn't possible. However, in a different case, e.g. one where you only need to support addition and multiplication, you could (and should) use literal_eval instead.
On the other hand, if this is homework with the intention for you to learn about parsing, then I would suggest doing this a different way. I know that if I was a teacher, I would respond to an answer using eval with "Very clever, but this won't help you pass a test on abstract syntax trees." (which are incidentally one thing that you should look at if you want to implement this "properly").
This seems like it should be easy, but I can't find the answer anywhere - nor able to derive one myself. How do you turn an unquoted python function/lambda into an AST?
Here is what I'd like to be able to do.
import ast
class Walker(ast.NodeVisitor):
pass
# ...
# note, this doesnt work as ast.parse wants a string
tree = ast.parse(lambda x,y: x+y)
Walker().visit(tree)
In general, you can't. For example, 2 + 2 is an expression -- but if you pass it to any function or method, the argument being passed is just the number 4, no way to recover what expression it was computed from. Function source code can sometimes be recovered (though not for a lambda), but "an unquoted Python expression" gets evaluated so what you get is just the object that's the expression's value.
What problem are you trying to solve? There may be other, viable approaches.
Edit: tx to the OP for clarifying. There's no way to do it for lambda or some other corner cases, but as I mention function source code can sometimes be recovered...:
import ast
import inspect
def f():
return 23
tree = ast.parse(inspect.getsource(f))
print ast.dump(tree)
inspect.getsource raises IOError if it can't get the source code for whatever object you're passing it. I suggest you wrap the parsing and getsource call into an auxiliary function that can accept a string (and just parses it) OR a function (and tries getsource on it, possibly giving better errors in the IOError case).
If you only get access to the function/lambda you only have the compiled python bytecode. The exact Python AST can't be reconstructed from the bytecode because there is information loss in the compilation process. But you can analyze the bytecode and create AST's for that. There is one such analyzer in GeniuSQL. I also have a small proof of concept that analyzes bytecode and creates SQLAlchemy clauseelements from this.
The process I used for analyzing is the following:
Split the code into a list of opcodes with potential arguments.
Find the basic blocks in the code by going through the opcodes and for every jump create a basic block boundary after the jump and before the jump target
Create a control flow graph from the basic blocks.
Go through all the basic blocks with abstract interpretation tracking stack and variable assignments in SSA form.
To create the output expression just get the calculated SSA return value.
I have pasted my proof of concept and example code using it. This is non-clean quickly hacked together code, but you're free to build on it if you like. Leave a note if you decide to make something useful from it.
The Meta library allows you to recover the source in many cases, with some exceptions such as comprehensions and lambdas.
import meta, ast
source = '''
a = 1
b = 2
c = (a ** b)
'''
mod = ast.parse(source, '<nofile>', 'exec')
code = compile(mod, '<nofile>', 'exec')
mod2 = meta.decompile(code)
source2 = meta.dump_python_source(mod2)
assert source == source2
You can't generate AST from compiled bytecode. You need the source code.
Your lambda expression is a function, which has lots of information, but I don't think it still has source code associated with. I'm not sure you can get what you want.