In python what is the equivalent of the quote operator? I am finding the need to delay evaluation. For example, suppose in the following lisp psuedocode I have:
a = '(func, 'g)
g = something
(eval a)
What I am doing is deferring evaluation of g till a later time. This is necessary because I want to define g later. What is the equivalent idea of this psuedocode in python?
a = lambda: func(g)
g = something
a()
This isn't quite the most literal translation - the most literal translation would use a string and eval - but it's probably the best fit. Quoting probably isn't what you wanted in Lisp anyway; you probably wanted to delay something or create a lambda. Note that func and g are closure variables in the lambda function, rather than symbols, so if you call a from an environment with different bindings for func or g, it'll still use the variables from a's environment of definition.
Using eval for delaying evaluation is bad, both in Lisp and Python.
in Python, and in Lisp, you can delay evaluation using a closure:
def print_it(x):
def f():
print g(x)
return f
f = print_it(42)
def g(x):
return x * x
f()
Please note that what is captured in a closure is not the value of a variable, but the variable itself and this is sometimes surprising:
fa = []
for x in range(10):
def g():
print x
fa.append(g)
for f in fa:
f() # all of them will print 9
x = 42
fa[0]() # the output will be 42
to solve this problem (that can also be present in Common Lisp) you may see things like:
for x in range(10):
def g(x = x):
print x
fa.append(g)
or (in CL) things like
(let ((a a))
(lambda () (print a)))
Python also has a lambda special form for anonymous functions, but they are limited to one single expression and cannot contain any statement. A locally def-ined function instead is a regular function without any limitations.
for x in range(10):
# print is a statement in Python 2.x and cannot be in a lambda
fa.append(lambda x=x: sys.stdout.write(str(x) + "\n"))
Finally Python 2.x has a syntax limitation and closed-over variables are read-only because if there is an assignment (or augmented-assignment) in a function there are only two possibilities:
The variable is a global (and has been previously declared so with global x)
The variable is a local
and in particular it's ruled out that a variable being assigned could be a local of an enclosing function scope.
Python 3.x removed this limitation by providing a new possible declaration nonlocal x and now the famous adder example can be implemented as
def adder(x):
def f(y):
nonlocal x
x += y
return x
return f
Related
When I run this code, I get this result:
15
15
I expect the output should be
15
17
but it is not. The question is: why?
def make_adder_and_setter(x):
def setter(n):
x = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)
You are setting a local variable x in the setter() function. Assignment to a name in a function marks it as a local, unless you specifically tell the Python compiler otherwise.
In Python 3, you can explicitly mark x as non-local using the nonlocal keyword:
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
Now x is marked as a free variable and looked up in the surrounding scope instead when assigned to.
In Python 2 you cannot mark a Python local as such. The only other option you have is marking x as a global. You'll have to resort to tricks where you alter values contained by a mutable object that lives in the surrounding scope.
An attribute on the setter function would work, for example; setter is local to the make_adder_and_setter() scope, attributes on that object would be visible to anything that has access to setter:
def make_adder_and_setter(x):
def setter(n):
setter.x = n
setter.x = x
return (lambda y: setter.x + y, setter)
Another trick is to use a mutable container, such as a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
In both cases you are not assigning to a local name anymore; the first example uses attribute assignment on the setter object, the second alters the x list, not assign to x itself.
Python 2.x has a syntax limitation that doesn't allow to capture a variable in read/write.
The reason is that if a variable is assigned in a function there are only two possibilities:
the variable is a global and has been declared so with global x
the variable is a local of the function
more specifically it's ruled out that the variable is a local of an enclosing function scope
This has been superseded in Python 3.x with the addition of nonlocal declaration. Your code would work as expected in Python 3 by changing it to
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
The python 2.x runtime is able to handle read-write closed over variable at a bytecode level, however the limitation is in the syntax that the compiler accepts.
You can see a lisp compiler that generates python bytecode directly that creates an adder closure with read-write captured state at the end of this video. The compiler can generate bytecode for Python 2.x, Python 3.x or PyPy.
If you need closed-over mutable state in Python 2.x a trick is to use a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
Your inner def setter(n) function defines its own local variable x. That hides the other x variable that was a parameter of make_adder_and_setter (makes a hole in the scope). So the setter function has no side effect. It just sets the value of an inner local variable and exits.
Maybe it will be clear for you if you try the code below. It does exactly the same thing, just uses the name z instead of x.
def make_adder_and_setter(x):
def setter(n):
z = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)
So today in computer science I asked about using a function as a variable. For example, I can create a function, such as returnMe(i) and make an array that will be used to call it. Like h = [help,returnMe] and then I can say h1 and it would call returnMe("Bob"). Sorry I was a little excited about this. My question is is there a way of calling like h.append(def function) and define a function that only exists in the array?
EDIT:
Here Is some code that I wrote with this!
So I just finished an awesome FizzBuzz with this solution thank you so much again! Here's that code as an example:
funct = []
s = ""
def newFunct(str, num):
return (lambda x: str if(x%num==0) else "")
funct.append(newFunct("Fizz",3))
funct.append(newFunct("Buzz",5))
for x in range(1,101):
for oper in funct:
s += oper(x)
s += ":"+str(x)+"\n"
print s
You can create anonymous functions using the lambda keyword.
def func(x,keyword='bar'):
return (x,keyword)
is roughly equivalent to:
func = lambda x,keyword='bar':(x,keyword)
So, if you want to create a list with functions in it:
my_list = [lambda x:x**2,lambda x:x**3]
print my_list[0](2) #4
print my_list[1](2) #8
Not really in Python. As mgilson shows, you can do this with trivial functions, but they can only contain expressions, not statements, so are very limited (you can't assign to a variable, for example).
This is of course supported in other languages: in Javascript, for example, creating substantial anonymous functions and passing them around is a very idiomatic thing to do.
You can create the functions in the original scope, assign them to the array and then delete them from their original scope. Thus, you can indeed call them from the array but not as a local variable. I am not sure if this meets your requirements.
#! /usr/bin/python3.2
def a (x): print (x * 2)
def b (x): print (x ** 2)
l = [a, b]
del a
del b
l [0] (3) #works
l [1] (3) #works
a (3) #fails epicly
You can create a list of lambda functions to increment by every number from 0 to 9 like so:
increment = [(lambda arg: (lambda x: arg + x))(i) for i in range(10)]
increment[0](1) #returns 1
increment[9](10) #returns 19
Side Note:
I think it's also important to note that this (function pointers not lambdas) is somewhat like how python holds methods in most classes, except instead of a list, it's a dictionary with function names pointing to the functions. In many but not all cases instance.func(args) is equivalent to instance.__dict__['func'](args) or type(class).__dict__['func'](args)
I'm wondering if it's possible for a closure in Python to manipulate variables in its namespace. You might call this side-effects because the state is being changed outside the closure itself. I'd like to do something like this
def closureMaker():
x = 0
def closure():
x+=1
print x
return closure
a = closureMaker()
a()
1
a()
2
Obviously what I hope to do is more complicated, but this example illustrates what I'm talking about.
You can't do exactly that in Python 2.x, but you can use a trick to get the same effect: use a mutable object such as a list.
def closureMaker():
x = [0]
def closure():
x[0] += 1
print x[0]
return closure
You can also make x an object with a named attribute, or a dictionary. This can be more readable than a list, especially if you have more than one such variable to modify.
In Python 3.x, you just need to add nonlocal x to your inner function. This causes assignments to x to go to the outer scope.
What limitations have closures in Python compared to language X closures?
nonlocal keyword in Python 2.x
Example:
def closureMaker():
x = 0
def closure():
nonlocal x
x += 1
print(x)
return closure
Very rarely I'll come across some code in python that uses an anonymous function which returns an anonymous function...?
Unfortunately I can't find an example on hand, but it usually takes the form like this:
g = lambda x,c: x**c lambda c: c+1
Why would someone do this? Maybe you can give an example that makes sense (I'm not sure the one I made makes any sense).
Edit: Here's an example:
swap = lambda a,x,y:(lambda f=a.__setitem__:(f(x,(a[x],a[y])),
f(y,a[x][0]),f(x,a[x][1])))()
You could use such a construct to do currying:
curry = lambda f, a: lambda x: f(a, x)
You might use it like:
>>> add = lambda x, y: x + y
>>> add5 = curry(add, 5)
>>> add5(3)
8
swap = lambda a,x,y:(lambda f=a.__setitem__:(f(x,(a[x],a[y])),
f(y,a[x][0]),f(x,a[x][1])))()
See the () at the end? The inner lambda isn't returned, its called.
The function does the equivalent of
def swap(a, x, y):
a[x] = (a[x], a[y])
a[y] = a[x][0]
a[x] = a[x][1]
But let's suppose that we want to do this in a lambda. We cannot use assignments in a lambda. However, we can call __setitem__ for the same effect.
def swap(a, x, y):
a.__setitem__(x, (a[x], a[y]))
a.__setitem__(y, a[x][0])
a.__setitem__(x, a[x][1])
But for a lambda, we can only have one expression. But since these are function calls we can wrap them up in a tuple
def swap(a, x, y):
(a.__setitem__(x, (a[x], a[y])),
a.__setitem__(y, a[x][0]),
a.__setitem__(x, a[x][1]))
However, all those __setitem__'s are getting me down, so let's factor them out:
def swap(a, x, y):
f = a.__setitem__
(f(x, (a[x], a[y])),
f(y, a[x][0]),
f(x, a[x][1]))
Dagnamit, I can't get away with adding another assignment! I know let's abuse default parameters.
def swap(a, x, y):
def inner(f = a.__setitem__):
(f(x, (a[x], a[y])),
f(y, a[x][0]),
f(x, a[x][1]))
inner()
Ok let's switch over to lambdas:
swap = lambda a, x, y: lambda f = a.__setitem__: (f(x, (a[x], a[y])), f(y, a[x][0]), f(x, a[x][1]))()
Which brings us back to the original expression (plus/minus typos)
All of this leads back to the question: Why?
The function should have been implemented as
def swap(a, x, y):
a[x],a[y] = a[y],a[x]
The original author went way out of his way to use a lambda rather then a function. It could be that he doesn't like nested function for some reason. I don't know. All I'll say is its bad code. (unless there is a mysterious justification for it.)
It can be useful for temporary placeholders. Suppose you have a decorator factory:
#call_logger(log_arguments=True, log_return=False)
def f(a, b):
pass
You can temporarily replace it with
call_logger = lambda *a, **kw: lambda f: f
It can also be useful if it indirectly returns a lambda:
import collections
collections.defaultdict(lambda: collections.defaultdict(lambda: collections.defaultdict(int)))
It's also useful for creating callable factories in the Python console.
And just because something is possible doesn't mean that you have to use it.
I did something like this just the other day to disable a test method in a unittest suite.
disable = lambda fn : lambda *args, **kwargs: None
#disable
test_method(self):
... test code that I wanted to disable ...
Easy to re-enable it later.
This can be used to pull out some common repetitive code (there are of course other ways to achieve this in python).
Maybe you're writing a a logger, and you need to prepend the level to the log string. You might write something like:
import sys
prefixer = lambda prefix: lambda message: sys.stderr.write(prefix + ":" + message + "\n")
log_error = prefixer("ERROR")
log_warning = prefixer("WARNING")
log_info = prefixer("INFO")
log_debug = prefixer("DEBUG")
log_info("An informative message")
log_error("Oh no, a fatal problem")
This program prints out
INFO:An informative message
ERROR:Oh no, a fatal problem
It is most oftenly - at least in code I come accross and that I myself write - used to "freeze" a variable with the value it has at the point the lambda function is created. Otherwise, nonlocals variable reference a variable in the scope they exist, which can lead to undesied results sometimes.
For example, if I want to create a list of ten functions, each one being a multiplier for a scalar from 0 to 9. One might be tempted to write it like this:
>>> a = [(lambda j: i * j) for i in range(10)]
>>> a[9](10)
90
Whoever, if you want to use any of the other factoried functions you get the same result:
>>> a[1](10)
90
That is because the "i" variable inside the lambda is not resolved when the lambda is created. Rather, Python keeps a reference to the "i" in the "for" statement - on the scope it was created (this reference is kept in the lambda function closure). When the lambda is executed, the variable is evaluated, and its value is the final one it had in that scope.
When one uses two nested lambdas like this:
>>> a = [(lambda k: (lambda j: k * j))(i) for i in range(10)]
The "i" variable is evaluated durint the execution of the "for" loop. Itś value is passed to "k" - and "k" is used as the non-local variable in the multiplier function we are factoring out. For each value of i, there will be a different instance of the enclosing lambda function, and a different value for the "k" variable.
So, it is possible to achieve the original intent :
>>> a = [(lambda k: (lambda j: k * j))(i) for i in range(10)]
>>> a[1](10)
10
>>> a[9](10)
90
>>>
It can be used to achieve a more continuation/trampolining style of programming,
See Continuation-passing style
Basically, with this you can modify functions instead of values
One example I stumbled with recently: To compute approximate derivatives (as functions) and use it as an input function in another place.
dx = 1/10**6
ddx = lambda f: lambda x: (f(x + dx) - f(x))/dx
f = lambda x: foo(x)
newton_method(func=ddx(f), x0=1, n=10)
This question already has answers here:
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
While I was investigating a problem I had with lexical closures in Javascript code, I came along this problem in Python:
flist = []
for i in xrange(3):
def func(x): return x * i
flist.append(func)
for f in flist:
print f(2)
Note that this example mindfully avoids lambda. It prints "4 4 4", which is surprising. I'd expect "0 2 4".
This equivalent Perl code does it right:
my #flist = ();
foreach my $i (0 .. 2)
{
push(#flist, sub {$i * $_[0]});
}
foreach my $f (#flist)
{
print $f->(2), "\n";
}
"0 2 4" is printed.
Can you please explain the difference ?
Update:
The problem is not with i being global. This displays the same behavior:
flist = []
def outer():
for i in xrange(3):
def inner(x): return x * i
flist.append(inner)
outer()
#~ print i # commented because it causes an error
for f in flist:
print f(2)
As the commented line shows, i is unknown at that point. Still, it prints "4 4 4".
Python is actually behaving as defined. Three separate functions are created, but they each have the closure of the environment they're defined in - in this case, the global environment (or the outer function's environment if the loop is placed inside another function). This is exactly the problem, though - in this environment, i is modified, and the closures all refer to the same i.
Here is the best solution I can come up with - create a function creater and invoke that instead. This will force different environments for each of the functions created, with a different i in each one.
flist = []
for i in xrange(3):
def funcC(j):
def func(x): return x * j
return func
flist.append(funcC(i))
for f in flist:
print f(2)
This is what happens when you mix side effects and functional programming.
The functions defined in the loop keep accessing the same variable i while its value changes. At the end of the loop, all the functions point to the same variable, which is holding the last value in the loop: the effect is what reported in the example.
In order to evaluate i and use its value, a common pattern is to set it as a parameter default: parameter defaults are evaluated when the def statement is executed, and thus the value of the loop variable is frozen.
The following works as expected:
flist = []
for i in xrange(3):
def func(x, i=i): # the *value* of i is copied in func() environment
return x * i
flist.append(func)
for f in flist:
print f(2)
Here's how you do it using the functools library (which I'm not sure was available at the time the question was posed).
from functools import partial
flist = []
def func(i, x): return x * i
for i in range(3):
flist.append(partial(func, i))
for f in flist:
print(f(2))
Outputs 0 2 4, as expected.
look at this:
for f in flist:
print f.func_closure
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
It means they all point to the same i variable instance, which will have a value of 2 once the loop is over.
A readable solution:
for i in xrange(3):
def ffunc(i):
def func(x): return x * i
return func
flist.append(ffunc(i))
What is happening is that the variable i is captured, and the functions are returning the value it is bound to at the time it is called. In functional languages this kind of situation never arises, as i wouldn't be rebound. However with python, and also as you've seen with lisp, this is no longer true.
The difference with your scheme example is to do with the semantics of the do loop. Scheme is effectively creating a new i variable each time through the loop, rather than reusing an existing i binding as with the other languages. If you use a different variable created external to the loop and mutate it, you'll see the same behaviour in scheme. Try replacing your loop with:
(let ((ii 1)) (
(do ((i 1 (+ 1 i)))
((>= i 4))
(set! flist
(cons (lambda (x) (* ii x)) flist))
(set! ii i))
))
Take a look here for some further discussion of this.
[Edit] Possibly a better way to describe it is to think of the do loop as a macro which performs the following steps:
Define a lambda taking a single parameter (i), with a body defined by the body of the loop,
An immediate call of that lambda with appropriate values of i as its parameter.
ie. the equivalent to the below python:
flist = []
def loop_body(i): # extract body of the for loop to function
def func(x): return x*i
flist.append(func)
map(loop_body, xrange(3)) # for i in xrange(3): body
The i is no longer the one from the parent scope but a brand new variable in its own scope (ie. the parameter to the lambda) and so you get the behaviour you observe. Python doesn't have this implicit new scope, so the body of the for loop just shares the i variable.
The problem is that all of the local functions bind to the same environment and thus to the same i variable. The solution (workaround) is to create separate environments (stack frames) for each function (or lambda):
t = [ (lambda x: lambda y : x*y)(x) for x in range(5)]
>>> t[1](2)
2
>>> t[2](2)
4
I'm still not entirely convinced why in some languages this works one way, and in some another way. In Common Lisp it's like Python:
(defvar *flist* '())
(dotimes (i 3 t)
(setf *flist*
(cons (lambda (x) (* x i)) *flist*)))
(dolist (f *flist*)
(format t "~a~%" (funcall f 2)))
Prints "6 6 6" (note that here the list is from 1 to 3, and built in reverse").
While in Scheme it works like in Perl:
(define flist '())
(do ((i 1 (+ 1 i)))
((>= i 4))
(set! flist
(cons (lambda (x) (* i x)) flist)))
(map
(lambda (f)
(printf "~a~%" (f 2)))
flist)
Prints "6 4 2"
And as I've mentioned already, Javascript is in the Python/CL camp. It appears there is an implementation decision here, which different languages approach in distinct ways. I would love to understand what is the decision, exactly.
The variable i is a global, whose value is 2 at each time the function f is called.
I would be inclined to implement the behavior you're after as follows:
>>> class f:
... def __init__(self, multiplier): self.multiplier = multiplier
... def __call__(self, multiplicand): return self.multiplier*multiplicand
...
>>> flist = [f(i) for i in range(3)]
>>> [g(2) for g in flist]
[0, 2, 4]
Response to your update: It's not the globalness of i per se which is causing this behavior, it's the fact that it's a variable from an enclosing scope which has a fixed value over the times when f is called. In your second example, the value of i is taken from the scope of the kkk function, and nothing is changing that when you call the functions on flist.
The reasoning behind the behavior has already been explained, and multiple solutions have been posted, but I think this is the most pythonic (remember, everything in Python is an object!):
flist = []
for i in xrange(3):
def func(x): return x * func.i
func.i=i
flist.append(func)
for f in flist:
print f(2)
Claudiu's answer is pretty good, using a function generator, but piro's answer is a hack, to be honest, as it's making i into a "hidden" argument with a default value (it'll work fine, but it's not "pythonic").
I didn't like how solutions above created wrappers in the loop. Note: python 3.xx
flist = []
def func(i):
return lambda x: x * i
for i in range(3):
flist.append(func(i))
for f in flist:
print f(2)