def cons(a, b):
def pair(f):
return f(a, b)
return pair
def car(f):
def left(a, b):
return a
return f(left)
def cdr(f):
def right(a, b):
return b
return f(right)
Found this python code on git.
Just want to know what is f(a,b) in cons definition is, and how does it work?
(Not a function I guess)
cons is a function, that takes two arguments, and returns a function that takes another function, which will consume these two arguments.
For example, consider the following function:
def add(a, b):
return a + b
This is just a function that adds the two inputs, so, for instance, add(2, 5) == 7
As this function takes two arguments, we can use cons to call this function:
func_caller = cons(2, 5) # cons receives two arguments and returns a function, which we call func_caller
result = func_caller(add) # func_caller receives a function, that will process these two arguments
print(result) # result is the actual result of doing add(2, 5), i.e. 7
This technique is useful for wrapping functions and executing stuff, before and after calling the appropriate functions.
For example, we can modify our cons function to actually print the values before and after calling add:
def add(a, b):
print('Adding {} and {}'.format(a, b))
return a + b
def cons(a, b):
print('Received arguments {} and {}'.format(a, b))
def pair(f):
print('Calling {} with {} and {}'.format(f, a, b))
result = f(a, b)
print('Got {}'.format(result))
return result
return pair
With this update, we get the following outputs:
func_caller = cons(2, 5)
# prints "Received arguments 2 and 5" from inside cons
result = func_caller(add)
# prints "Calling add with 2 and 5" from inside pair
# prints "Adding 2 and 5" from inside add
# prints "Got 7" from inside pair
This isn't going to make any sense to you until you know what cons, car, and cdr mean.
In Lisp, lists are stored as a very simple form of linked list. A list is either nil (like None) for an empty list, or it's a pair of a value and another list. The cons function takes a value and a list and returns you another list just by making a pair:
def cons(head, rest):
return (head, rest)
And the car and cdr functions (they stand for "Contents of Address|Data Register", because those are the assembly language instructions used to implement them on a particular 1950s computer, but that isn't very helpful) return the first or second value from a pair:
def car(lst):
return lst[0]
def cdr(lst):
return lst[1]
So, you can make a list:
lst = cons(1, cons(2, cons(3, None)))
… and you can get the second value from it:
print(car(cdr(lst))
… and you can even write functions to get the nth value:
def nth(lst, n):
if n == 0:
return car(lst)
return nth(cdr(lst), n-1)
… or print out the whole list:
def printlist(lst):
if lst:
print(car(lst), end=' ')
printlist(cdr(lst))
If you understand how these work, the next step is to try them on those weird definitions you found.
They still do the same thing. So, the question is: How? And the bigger question is: What's the point?
Well, there's no practical point to using these weird functions; the real point is to show you that everything in computer science can be written with just functions, no built-in data structures like tuples (or even integers; that just takes a different trick).
The key is higher-order functions: functions that take functions as values and/or return other functions. You actually use these all the time: map, sort with a key, decorators, partial… they’re only confusing when they’re really simple:
def car(f):
def left(a, b):
return a
return f(left)
This takes a function, and calls it on a function that returns the first of its two arguments.
And cdr is similar.
It's hard to see how you'd use either of these, until you see cons:
def cons(a, b):
def pair(f):
return f(a, b)
return pair
This takes two things and returns a function that takes another function and applies it to those two things.
So, what do we get from cons(3, None)? We get a function that takes a function, and applies it to the arguments 3 and None:
def pair3(f):
return f(3, None)
And if we call cons(2, cons(3, None))?
def pair23(f):
return f(2, pair3)
And what happens if you call car on that function? Trace through it:
def left(a, b):
return a
return pair23(left)
That pair23(left) does this:
return left(2, pair3)
And left is dead simple:
return 2
So, we got the first element of (2, cons(3, None)).
What if you call cdr?
def right(a, b):
return a
return pair23(right)
That pair23(right) does this:
return right(2, pair3)
… and right is dead simple, so it just returns pair3.
You can work out that if we call car(cdr(pair23)), we're going to get the 3 out of it.
And now you can write lst = cons(1, cons(2, cons(3, None))), write the recursive nth and printlist functions above, and trace through how they work on lst.
I mentioned above that you can even get rid of integers. How do you do that? Read about Church numerals. You define zero and successor functions. Then you can define one as successor(zero) and two as successor(one). You can even recursively define add so that add(x, zero) is x but add(x, successor(y)) is successor(add(x, y)), and go on to define mul, etc.
You also need a special function you can use as a value for nil.
Anyway, once you've done that, using all of the other definitions above, you can do lst = cons(zero(cons(one, cons(two, cons(three, nil)))), and nth(lst, two) will give you back one. (Of course writing printlist will be a bit trickier…)
Obviously, this is all going to be a lot slower than just using tuples and integers and so on. But theoretically, it’s interesting.
Consider this: we could write a tiny dialect of Python that has only three kinds of statements—def, return, and expression statements—and only three kinds of expressions—literals, identifiers, and function calls—and it could do everything normal Python does. (In fact, you could get rid of statements altogether just by having a function-defining expression, which Python already has.) That tiny language would be a pain to use, but it would a lot easier to write a program to reason about programs in that tiny language. And we even know how to translate code using tuples, loops, etc. into code in this tiny subset language, which means we can write a program that reasons about that real Python code.
In fact, with a couple more tricks (curried functions and/or static function types, and lazy evaluation), the compiler/interpreter could do that kind of reasoning on the fly and optimize our code for us. It’s easy to tell programmatically that car(cdr(cons(2, cons(3, None)) is going to return 3 without having to actually evaluate most of those function calls, so we can just skip evaluating them and substitute 3 for the whole expression.
Of course this breaks down if any function can have side effects. You obviously can’t just substitute None for print(3) and get the same results. So instead, you need some clever trick where IO is handled by some magic object that evaluates functions to figure out what it should read and write, and then the whole rest of the program, the part that users write, becomes pure and can be optimized however you want. With a couple more abstractions, we can even make IO something that doesn’t have to be magical to do that.
And then you can build a standard library that gives you back all those things we gave up, written in terms of defining and calling functions, so it’s actually usable—but under the covers it’s all just reducing pure function calls, which is simple enough for a computer to optimize. And then you’ve basically written Haskell.
Related
I recently had following code in mind and wondered what was wrong with it. Previously I used the .get method of dictionaries with success, but now i wanted to pass arguments too and this is where i noticed a somewhat weird behavior:
def string_encoder(nmstr):
return nmstr.encode('UTF-8')
def int_adder(nr_int):
return int(nr_int) + int(nr_int)
def selector(fun, val):
return {'str_en': string_encoder(val),
'nr_add': int_adder(val)}.get(fun, string_encoder(val))
selector('str_en', 'Test') -> ValueError
selector('str_en', 1) -> AttributeError
The above code will never run.
To inspect the issue i supplied a small piece of code:
def p1(pstr):
print('p1: ', pstr)
return pstr
def p2(pstr):
print('p2: ', pstr)
return pstr
def selector_2(fun, val):
return {'p1': p1(val),
'p2': p2(val)}.get(fun, p2(val))
selector_2('p1', 'Test')
Out[]: p1: Test
p2: Test
p2: Test
'Test'
I would expect the following .get('p1', 'test') to output 'p1: test' test.
But as it appears to me, every argument is evaluated, even if it is not selected. So my question is: Why is every argument evaluated with the .get method, or how can this behavior be explained?
dict creation is eager, as is argument evaluation. So before get even runs, you've called string_encoder twice, and int_adder once (and since the behaviors are largely orthogonal, you'll get an error for anything but a numeric str like "123").
You need to avoid calling the function until you know which one to call (and ideally, only call that function once).
The simplest solution is to have the dict and get call contain the functions themselves, rather than the result of calling them; you'll end up with whichever function wins, and you can then call that function. For example:
def selector(fun, val):
# Removed (val) from all mentions of functions
return {'str_en': string_encoder,
'nr_add': int_adder}.get(fun, string_encoder)(val) # <- But used it to call resulting function
Given string_encoder is your default, you could remove 'str_en' handling entirely to simplify to:
return {'nr_add': int_adder}.get(fun, string_encoder)(val)
which leads to the realization that you're not really getting anything out of the dict. dicts have cheap lookup, but you're rebuilding the dict every call, so you didn't save a thing. Given that you really only have two behaviors:
Call int_adder if fun is 'nr_add'
Otherwise, call string_encoder
the correct solution is just an if check which is more efficient, and easier to read:
def selector(fun, val):
if fun == 'nr_add':
return int_adder(val)
return string_encoder(val)
# Or if you love one-liners:
return int_adder(val) if fun == 'nr_add' else string_encoder(val)
If your real code has a lot of entries in the dict, not just two, one of which is unnecessary, then you can use a dict for performance, but build it once at global scope and reference it in the function so you're not rebuilding it every call (which loses all performance benefits of dict), e.g.:
# Built only once at global scope
_selector_lookup_table = {
'str_en': string_encoder,
'nr_add': int_adder,
'foo': some_other_func,
...
'baz': yet_another_func,
}
def selector(fun, val):
# Reused in function for each call
return _selector_lookup_table.get(fun, default_func)(val)
If you want to avoid evaluation of functions and only chooses the function, do this instead for your second block (the syntax will also work for your first block):
def selector_2(fun, val):
return {'p1': p1,
'p2': p2}.get(fun)(val)
Suppose I have a function that takes two arguments and performs some calculation on them:
def add(a, b):
return a + b
I want to call this function through a multiprocessing library which can only handle functions with a single argument. So, I change the function to take its argument as a single tuple instead:
def add2(ab):
a, b = ab
return a + b
However, this seems clunky to me. The variables essentially need to be defined (and documented) twice. If I were using a lambda function, I could just write the following and it will accept the tuple properly:
add3 = lambda (a, b): a + b
Unfortunately, my function is not trivial enough to implement as a lambda function. Is there any sort of syntactic sugar feature in python that would allow me to write a named function that accepts a tuple but treats each component of that tuple as a separate named argument? My attempts to search for solutions to this have mostly turned up references to the *args operator, but that does not apply here because I do not control the site where the function is called.
Here is an example of how the function is being called. Note that it is called via the multiprocessing library so I cannot pass more than one argument:
import multiprocessing
pool = multiprocessing.Pool(processes=4)
for result in pool.imap_unordered(add, [(1,2),(3,4)]):
print(result)
Answers for either python 2.7 or 3.x are welcome.
It's best not to alter the original function interface, making it less Pythonic.
In Python 2, write a wrapper function to use with multiprocessing.
def _add(args):
return add(*args)
In Python 3, just use Pool.starmap instead:
>>> def add(a, b):
... return a + b
...
>>> p = Pool()
>>> list(p.starmap(add, [(1, 2), ('hello', ' world')]))
[3, 'hello world']
If you are worried about repeating yourself (a and b appear too many times), simply give the incoming tuple a non-descriptive name.
def add(t):
a, b = t
return a + b
Or, in your specific case, you can avoid a and b altogether by indexing the tuple:
def add(addends):
return addends[0] + addends[1]
As an alternative, you could wrap your function so the source code has the familiar argument format, but the function in use has the tuple argument:
def tupleize(func):
def wrapper(tup):
return func(*tup)
return wrapper
#tupleize
def add(a, b):
return a+b
t = 1, 2
assert(add(t) == 3)
As I was writing this question, I found the way to do it in Python 2.7:
def add4((a, b)):
return a + b
However apparently this no longer works in Python 3, so additional answers regarding Python 3 would still be helpful.
You could use a decorator to extend the multiprocessing library function to take multiple arguments, do whatever you want to them, and then call it with a single argument.
For example, a simple decorator that takes any number of arguments, sums them together, then calls the original function with the total as a single argument:
import 3rdpartylib
def sum_args(func):
def inner(*args):
return func(sum(args))
return inner
# Replace imported function with decorated version
3rdpartylib = sum_args(3rdpartylib)
# Decorate your own libraries
#sum_args
def my_own_lib(number):
print("A:", number)
3rdpartylib(1,2,3,4)
my_own_lib(5,10,15)
The main advantage is that you can decorate/replace any number of methods with this same decorator function to achieve the same effect.
(disclaimed: not a Python kid, so please be gentle)
I am trying to compose functions using the following:
def compose(*functions):
return functools.reduce(lambda acc, f: lambda x: acc(f(x)), functions, lambda x: x)
which works as expected for scalar functions. I'd like to work with functions returning tuples and others taking multiple arguments, eg.
def dummy(name):
return (name, len(name), name.upper())
def transform(name, size, upper):
return (upper, -size, name)
# What I want to achieve using composition,
# ie. f = compose(transform, dummy)
transform(*dummy('Australia'))
=> ('AUSTRALIA', -9, 'Australia')
Since dummy returns a tuple and transform takes three arguments, I need to unpack the value.
How can I achieve this using my compose function above? If I try like this, I get:
f = compose(transform, dummy)
f('Australia')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in <lambda>
File "<stdin>", line 2, in <lambda>
TypeError: transform() takes exactly 3 arguments (1 given)
Is there a way to change compose such that it will unpack where needed?
This one works for your example but it wont handle just any arbitrary function - it will only works with positional arguments and (of course) the signature of any function must match the return value of the previous (wrt/ application order) one.
def compose(*functions):
return functools.reduce(
lambda f, g: lambda *args: f(*g(*args)),
functions,
lambda *args: args
)
Note that using reduce here, while certainly idiomatic in functional programming, is rather unpythonic. The "obvious" pythonic implementation would use iteration instead:
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
args = func(*args)
return args
return composed
Edit:
You ask "Is there a way to make have a compose function which will work in both cases" - "both cases" here meaning wether the functions returns an iterable or not (what you call "scalar functions", a concept that has no meaning in Python).
Using the iteration-based implementation, you could just test if the return value is iterable and wrap it in a tuple ie:
import collections
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
if not isinstance(args, collections.Iterable):
args = (args,)
args = func(*args)
return args
return composed
but this is not garanteed to work as expected - actually this is even garanteed to NOT work as expected for most use cases. There are a lot of builtin iterable types in Python (and even more user-defined ones) and just knowing an object is iterable doesn't say much about it's semantic.
For example a dict or str are iterable but in this case should obviously be considered a "scalar". A list is iterable too, and how it should be interpreted in this case is actually just undecidable without knowing exactly what it contains and what the "next" function in composition order expects - in some cases you will want to treat it as a single argument, in other cases ase a list of args.
IOW only the caller of the compose() function can really tell how each function result should be considered - actually you might even have cases where you want a tuple to be considered as a "scalar" value by the next function. So to make a long story short: no, there's no one-size-fits-all generic solution in Python. The best I could think of requires a combination of result inspection and manual wrapping of composed functions so the result is properly interpreted by the "composed" function but at this point manually composing the functions will be both way simpler and much more robust.
FWIW remember that Python is first and mostly a dynamically typed object oriented language so while it does have a decent support for functional programming idioms it's obviously not the best tool for real functional programming.
You might consider inserting a "function" (really, a class constructor) in your compose chain to signal the unpacking of the prior/inner function's results. You would then adjust your composer function to check for that class to determine if the prior result should be unpacked. (You actually end up doing the reverse: tuple-wrap all function results except those signaled to be unpacked -- and then have the composer unpack everything.) It adds overhead, it's not at all Pythonic, it's written in a terse lambda style, but it does accomplish the goal of being able to properly signal in a function chain when the composer should unpack a result. Consider the following generic code, which you can then adapt to your specific composition chain:
from functools import reduce
from operator import add
class upk: #class constructor signals composer to unpack prior result
def __init__(s,r): s.r = r #hold function's return for wrapper function
idt = lambda x: x #identity
wrp = lambda x: x.r if isinstance(x, upk) else (x,) #wrap all but unpackables
com = lambda *fs: ( #unpackable compose, unpacking whenever upk is encountered
reduce(lambda a,f: lambda *x: a(*wrp(f(*x))), fs, idt) )
foo = com(add, upk, divmod) #upk signals divmod's results should be unpacked
print(foo(6,4))
This circumvents the problem, as called out by prior answers/comments, of requiring your composer to guess which types of iterables should be unpacked. Of course, the cost is that you must explicitly insert upk into the callable chain whenever unpacking is required. In that sense, it is by no means "automatic", but it is still a fairly simple/terse way of achieving the intended result while avoiding unintended wraps/unwraps in many corner cases.
The compose function in the answer contributed by Bruno did do the job for functions with multiple arguments but didn't work any more for scalar ones unfortunately.
Using the fact that Python `unpacks' tuples into positional arguments, this is how I solved it:
import functools
def compose(*functions):
def pack(x): return x if type(x) is tuple else (x,)
return functools.reduce(
lambda acc, f: lambda *y: f(*pack(acc(*pack(y)))), reversed(functions), lambda *x: x)
which now works just as expected, eg.
#########################
# scalar-valued functions
#########################
def a(x): return x + 1
def b(x): return -x
# explicit
> a(b(b(a(15))))
# => 17
# compose
> compose(a, b, b, a)(15)
=> 17
########################
# tuple-valued functions
########################
def dummy(x):
return (x.upper(), len(x), x)
def trans(a, b, c):
return (b, c, a)
# explicit
> trans(*dummy('Australia'))
# => ('AUSTRALIA', 9, 'Australia')
# compose
> compose(trans, dummy)('Australia')
# => ('AUSTRALIA', 9, 'Australia')
And this also works with multiple arguments:
def add(x, y): return x + y
# explicit
> b(a(add(5, 3)))
=> -9
# compose
> compose(b, a, add)(5, 3)
=> -9
I'm doing an exercise where I'm to create a class representing functions (written as lambda expressions) and several methods involving them.
The ones I've written so far are:
class Func():
def __init__(self, func, domain):
self.func = func
self.domain = domain
def __call__(self, x):
if self.domain(x):
return self.func(x)
return None
def compose(self, other):
comp_func= lambda x: self.func(other(x))
comp_dom= lambda x: other.domain(x) and self.domain(other(x))
return Func(comp_func, comp_dom)
def exp(self, k):
exp_func= self
for i in range(k-1):
exp_func = Func.compose(exp_func, self)
return exp_func
As you can see above, the function exp composes a function with itself k-1 times. Now I'm to write a recursive version of said function, taking the same arguments "self" and "k".
However I'm having difficulty figuring out how it would work. In the original exp I wrote I had access to the original function "self" throughout all iterations, however when making a recursive function I lose access to the original function and with each iteration only have access to the most recent composed function. So for example, if I try composing self with self a certain number of times I will get:
f= x+3
f^2= x+6
(f^2)^2= x+12
So we skipped the function x+9.
How do I get around this? Is there a way to still retain access to the original function?
Update:
def exp_rec(self, k):
if k==1:
return self
return Func.compose(Func.exp_rec(self, k-1), self)
This is an exercise, so I won't provide the answer.
In recursion, you want to do two things:
Determine and check a "guard condition" that tells you when to stop; and
Determine and compute the "recurrence relation" that tells you the next value.
Consider a simple factorial function:
def fact(n):
if n == 1:
return 1
return n * fact(n - 1)
In this example, the guard condition is fairly obvious- it's the only conditional statement! And the recurrence relation is in the return statement.
For your exercise, things are slightly less obvious, since the intent is to define a function composition, rather than a straight integer computation. But consider:
f = Func(lambda x: x + 3)
(This is your example.) You want f.exp(1) to be the same as f, and f.exp(2) to be f(f(x)). That right there tells you the guard condition and the recurrence relation:
The guard condition is that exp() only works for positive numbers. This is because exp(0) might have to return different things for different input types (what does exp(0) return when f = Func(lambda s: s + '!') ?).
So test for exp(1), and let that condition be the original lambda.
Then, when recursively defining exp(n+1), let that be the composition of your original lambda with exp(n).
You have several things to consider: First, your class instance has data associated with it. That data will "travel along" with you in your recursion, so you don't have to pass so many parameters recursively. Second, you need to decide whether Func.exp() should create a new Func(), or whether it should modify the existing Func object. Finally, consider how you would write a hard-coded function, Func.exp2() that just constructed what we would call Func.exp(2). That should give you an idea of your recurrence relation.
Update
Based on some comments, I feel like I should show this code. If you are going to have your recursive function modify the self object, instead of returning a new object, then you will need to "cache" the values from self before they get modified, like so:
func = self.func
domain = self.domain
... recursive function modifies self.func and self.domain
So Ive been giving the following code in a kind of sort of python class. Its really a discrete math class but he uses python to demonstrate everything. This code is supposed to demonstate a multiplexer and building a xor gate with it.
def mux41(i0,i1,i2,i3):
return lambda s1,s0:{(0,0):i0,(0,1):i1,(1,0):i2,(1,1):i3}[(s1,s0)]
def xor2(a,b):
return mux41(0,1,1,0)(a,b)
In the xor2 function I dont understand the syntax behind return mux41(0,1,1,0)(a,b) the 1's and 0's are the input to the mux function, but what is the (a,b) doing?
The (a, b) is actually the input to the lambda function that you return in the mux41 function.
Your mux41 function returns a lambda function which looks like it returns a value in a dictionary based on the input to the mux41 function. You need the second input to say which value you want to return.
It is directly equivalent to:
def xor2(a,b):
f = mux41(0,1,1,0)
return f(a,b)
That is fairly advanced code to throw at Python beginners, so don't feel bad it wasn't obvious to you. I also think it is rather trickier than it needs to be.
def mux41(i0,i1,i2,i3):
return lambda s1,s0:{(0,0):i0,(0,1):i1,(1,0):i2,(1,1):i3}[(s1,s0)]
This defines a function object that returns a value based on two inputs. The two inputs are s1 and s0. The function object builds a dictionary that is pre-populated with the four values passed int to mux41(), and it uses s0 and s1 to select one of those four values.
Dictionaries use keys to look up values. In this case, the keys are Python tuples: (0, 0), (0, 1), (1, 0), and (1,1). The expression (s1,s0) is building a tuple from the arguments s0 and s1. This tuple is used as the key to lookup a value from the dictionary.
def xor2(a,b):
return mux41(0,1,1,0)(a,b)
So, mux41() returns a function object that does the stuff I just discussed. xor2() calls mux41() and gets a function object; then it immediately calls that returned function object, passing in a and b as arguments. Finally it returns the answer.
The function object created by mux41() is not saved anywhere. So, every single time you call xor2(), you are creating a function object, which is then garbage collected. When the function object runs, it builds a dictionary object, and this too is garbage collected after each single use. This is possibly the most complicated XOR function I have ever seen.
Here is a rewrite that might make this a bit clearer. Instead of using lambda to create an un-named function object, I'll just use def to create a named function.
def mux41(i0,i1,i2,i3):
def mux_fn(s1, s0):
d = {
(0,0):i0,
(0,1):i1,
(1,0):i2,
(1,1):i3
}
tup = (s1, s0)
return d[tup]
return mux_fn
def xor2(a,b):
mux_fn = mux41(0,1,1,0)
return mux_fn(a,b)
EDIT: Here is what I would have written if I wanted to make a table-lookup XOR in Python.
_d_xor2 = {
(0,0) : 0,
(0,1) : 1,
(1,0) : 1,
(1,1) : 0
}
def xor2(a,b):
tup = (a, b)
return _d_xor2[tup]
We build the lookup dictionary once, then use it directly from xor2(). It's not really necessary to make an explicit temp variable in xor2() but it might be a bit clearer. You could just do this:
def xor2(a,b):
return _d_xor2[(a, b)]
Which do you prefer?
And of course, since Python has an XOR operator built-in, you could write it like this:
def xor2(a,b):
return a ^ b
If I were writing this for real I would probably add error handling and/or make it operate on bool values.
def xor2(a,b):
return bool(a) ^ bool(b)
EDIT: One more thing just occurred to me. In Python, the rule is "the comma makes the tuple". The parentheses around a tuple are sometimes optional. I just checked, and it works just fine to leave off the parentheses in a dictionary lookup. So you can do this:
def xor2(a,b):
return _d_xor2[a, b]
And it works fine. This is perhaps a bit too tricky? If I saw this in someone else's code, it would surprise me.