While doing programming exercises on codewars.com, I encountered an exercise on currying and partial functions.
Being a novice in programming and new to the topic, I searched on the internet for information on the topic and got quite far into solving the exercise. However I have now stumbled upon an obstacle I can't seem to overcome and am here looking for a nudge in the right direction.
The exercise is rather simple: write a function that can curry and/or partial any input function and evaluates the input function once enough input parameters are supplied. The input function can accept any number of input parameters. Also the curry/partial function should be very flexible in how it is called, being able to handle many, many different ways of calling the function. Also, the curry/partial function is allowed to be called with more inputs than required by the input function, in that case all the excess inputs need to be ignored.
Following the exercise link, all the test cases can be found that the function needs to be able to handle.
The code I came up with is the following:
from functools import partial
from inspect import signature
def curry_partial(func, *initial_args):
""" Generates a 'curried' version of a function. """
# Process any initial arguments that where given. If the number of arguments that are given exceeds
# minArgs (the number of input arguments that func needs), func is evaluated
minArgs = len(signature(func).parameters)
if initial_args:
if len(initial_args) >= minArgs:
return func(*initial_args[:minArgs])
func = partial(func, *initial_args)
minArgs = len(signature(func).parameters)
# Do the currying
def g(*myArgs):
nonlocal minArgs
# Evaluate function if we have the necessary amount of input arguments
if minArgs is not None and minArgs <= len(myArgs):
return func(*myArgs[:minArgs])
def f(*args):
nonlocal minArgs
newArgs = myArgs + args if args else myArgs
if minArgs is not None and minArgs <= len(newArgs):
return func(*newArgs[:minArgs])
else:
return g(*newArgs)
return f
return g
Now this code fails when the following test is executed:
test.assert_equals(curry_partial(curry_partial(curry_partial(add, a), b), c), sum)
where add = a + b + c (properly defined function), a = 1, b = 2, c = 3, and sum = 6.
The reason this fails is because curry_partial(add, a) returns a function handle to the function g. In the second call, curry_partial(<function_handle to g>, b), the calculation minArgs = len(signature(func).parameters) doesn't work like I want it to, because it will now calculate how many input arguments function g requires (which is 1: i.e. *myArgs), and not how many the original func still requires. So the question is, how can I write my code such that I can keep track of how many input arguments my original func still needs (reducing that number each time I am partialling the function with any given initial arguments).
I still have much to learn about programming and currying/partial, so most likely I have not chosen the most convenient approach. But I'd like to learn. The difficulty in this exercise for me is the combination of partial and curry, i.e. doing a curry loop while partialling any initial arguments that are encountered.
Try this out.
from inspect import signature
# Here `is_set` acts like a flip-flop
is_set = False
params = 0
def curry_partial(func, *partial_args):
"""
Required argument: func
Optional argument: partial_args
Return:
1) Result of the `func` if
`partial_args` contains
required number of items.
2) Function `wrapper` if `partial_args`
contains less than the required
number of items.
"""
global is_set, params
if not is_set:
is_set = True
# if func is already a value
# we should return it
try: params = len(signature(func).parameters)
except: return func
try:
is_set = False
return func(*partial_args[:params])
except:
is_set = True
def wrapper(*extra_args):
"""
Optional argument: extra_args
Return:
1) Result of the `func` if `args`
contains required number of
items.
2) Result of `curry_partial` if
`args` contains less than the
required number of items.
"""
args = (partial_args + extra_args)
try:
is_set = False
return func(*args[:params])
except:
is_set = True
return curry_partial(func, *args)
return wrapper
This indeed isn't very good by design. Instead you should use class, to do all the internal works like, for example, the flip-flop (don't worry we don't need any flip-flop there ;-)).
Whenever there's a function that takes arbitrary arguments, you can always instantiate that class passing the function. But this time however, I leave that on you.
I am not sure about currying, but if you need a simple partial function generator, you could try something like this:
from functools import partial
from inspect import signature
def execute_or_partial(f, *args):
max = len(signature(f).parameters)
if len(args) >= max:
return f(*args[:max])
else:
return partial(f, *args)
s = lambda x, y, z: x + y + z
t = execute_or_partial(s, 1)
u = execute_or_partial(t, 2)
v = execute_or_partial(u, 3)
print(v)
or
print(execute_or_partial(execute_or_partial(execute_or_partial(s, 1), 2), 3))
Even if it doesn't solve your original problem, see if you can use the above code to reduce code repetition (I am not sure, but I think there is some code repetition in the inner function?); that will make the subsequent problems easier to solve.
There could be functions in the standard library that already solve this problem. Many pure functional languages like Haskell have this feature built into the language.
Related
def cons(a, b):
def pair(f):
return f(a, b)
return pair
def car(f):
def left(a, b):
return a
return f(left)
def cdr(f):
def right(a, b):
return b
return f(right)
Found this python code on git.
Just want to know what is f(a,b) in cons definition is, and how does it work?
(Not a function I guess)
cons is a function, that takes two arguments, and returns a function that takes another function, which will consume these two arguments.
For example, consider the following function:
def add(a, b):
return a + b
This is just a function that adds the two inputs, so, for instance, add(2, 5) == 7
As this function takes two arguments, we can use cons to call this function:
func_caller = cons(2, 5) # cons receives two arguments and returns a function, which we call func_caller
result = func_caller(add) # func_caller receives a function, that will process these two arguments
print(result) # result is the actual result of doing add(2, 5), i.e. 7
This technique is useful for wrapping functions and executing stuff, before and after calling the appropriate functions.
For example, we can modify our cons function to actually print the values before and after calling add:
def add(a, b):
print('Adding {} and {}'.format(a, b))
return a + b
def cons(a, b):
print('Received arguments {} and {}'.format(a, b))
def pair(f):
print('Calling {} with {} and {}'.format(f, a, b))
result = f(a, b)
print('Got {}'.format(result))
return result
return pair
With this update, we get the following outputs:
func_caller = cons(2, 5)
# prints "Received arguments 2 and 5" from inside cons
result = func_caller(add)
# prints "Calling add with 2 and 5" from inside pair
# prints "Adding 2 and 5" from inside add
# prints "Got 7" from inside pair
This isn't going to make any sense to you until you know what cons, car, and cdr mean.
In Lisp, lists are stored as a very simple form of linked list. A list is either nil (like None) for an empty list, or it's a pair of a value and another list. The cons function takes a value and a list and returns you another list just by making a pair:
def cons(head, rest):
return (head, rest)
And the car and cdr functions (they stand for "Contents of Address|Data Register", because those are the assembly language instructions used to implement them on a particular 1950s computer, but that isn't very helpful) return the first or second value from a pair:
def car(lst):
return lst[0]
def cdr(lst):
return lst[1]
So, you can make a list:
lst = cons(1, cons(2, cons(3, None)))
… and you can get the second value from it:
print(car(cdr(lst))
… and you can even write functions to get the nth value:
def nth(lst, n):
if n == 0:
return car(lst)
return nth(cdr(lst), n-1)
… or print out the whole list:
def printlist(lst):
if lst:
print(car(lst), end=' ')
printlist(cdr(lst))
If you understand how these work, the next step is to try them on those weird definitions you found.
They still do the same thing. So, the question is: How? And the bigger question is: What's the point?
Well, there's no practical point to using these weird functions; the real point is to show you that everything in computer science can be written with just functions, no built-in data structures like tuples (or even integers; that just takes a different trick).
The key is higher-order functions: functions that take functions as values and/or return other functions. You actually use these all the time: map, sort with a key, decorators, partial… they’re only confusing when they’re really simple:
def car(f):
def left(a, b):
return a
return f(left)
This takes a function, and calls it on a function that returns the first of its two arguments.
And cdr is similar.
It's hard to see how you'd use either of these, until you see cons:
def cons(a, b):
def pair(f):
return f(a, b)
return pair
This takes two things and returns a function that takes another function and applies it to those two things.
So, what do we get from cons(3, None)? We get a function that takes a function, and applies it to the arguments 3 and None:
def pair3(f):
return f(3, None)
And if we call cons(2, cons(3, None))?
def pair23(f):
return f(2, pair3)
And what happens if you call car on that function? Trace through it:
def left(a, b):
return a
return pair23(left)
That pair23(left) does this:
return left(2, pair3)
And left is dead simple:
return 2
So, we got the first element of (2, cons(3, None)).
What if you call cdr?
def right(a, b):
return a
return pair23(right)
That pair23(right) does this:
return right(2, pair3)
… and right is dead simple, so it just returns pair3.
You can work out that if we call car(cdr(pair23)), we're going to get the 3 out of it.
And now you can write lst = cons(1, cons(2, cons(3, None))), write the recursive nth and printlist functions above, and trace through how they work on lst.
I mentioned above that you can even get rid of integers. How do you do that? Read about Church numerals. You define zero and successor functions. Then you can define one as successor(zero) and two as successor(one). You can even recursively define add so that add(x, zero) is x but add(x, successor(y)) is successor(add(x, y)), and go on to define mul, etc.
You also need a special function you can use as a value for nil.
Anyway, once you've done that, using all of the other definitions above, you can do lst = cons(zero(cons(one, cons(two, cons(three, nil)))), and nth(lst, two) will give you back one. (Of course writing printlist will be a bit trickier…)
Obviously, this is all going to be a lot slower than just using tuples and integers and so on. But theoretically, it’s interesting.
Consider this: we could write a tiny dialect of Python that has only three kinds of statements—def, return, and expression statements—and only three kinds of expressions—literals, identifiers, and function calls—and it could do everything normal Python does. (In fact, you could get rid of statements altogether just by having a function-defining expression, which Python already has.) That tiny language would be a pain to use, but it would a lot easier to write a program to reason about programs in that tiny language. And we even know how to translate code using tuples, loops, etc. into code in this tiny subset language, which means we can write a program that reasons about that real Python code.
In fact, with a couple more tricks (curried functions and/or static function types, and lazy evaluation), the compiler/interpreter could do that kind of reasoning on the fly and optimize our code for us. It’s easy to tell programmatically that car(cdr(cons(2, cons(3, None)) is going to return 3 without having to actually evaluate most of those function calls, so we can just skip evaluating them and substitute 3 for the whole expression.
Of course this breaks down if any function can have side effects. You obviously can’t just substitute None for print(3) and get the same results. So instead, you need some clever trick where IO is handled by some magic object that evaluates functions to figure out what it should read and write, and then the whole rest of the program, the part that users write, becomes pure and can be optimized however you want. With a couple more abstractions, we can even make IO something that doesn’t have to be magical to do that.
And then you can build a standard library that gives you back all those things we gave up, written in terms of defining and calling functions, so it’s actually usable—but under the covers it’s all just reducing pure function calls, which is simple enough for a computer to optimize. And then you’ve basically written Haskell.
Imagine I have a two functions
def areaSquare(a,b):
print( a * b)
def areaCircle(radius):
print(3.14159 * radius ** 2)
And I want to create a third function that is called area.
area(areaCircle,radius = 3, repeat = 5)
# prints 3.14159 * 9 five times
area(areaSquare, a = 2, b = 3, repeat = 6)
# prints 2 * 6 six times
So the function takes a function as a parameter. Depending on the function which is passed to it as a parameter, it should require additional parameters. Is there a way to achieve this? I know function overloading would be an option. But I do not want to define multiple functions for this purpose.
Yes, kwargs are your friend here. We can define the function such that the remaining parameters are captured in a dictionary named kwargs, and then passed to the function we provide. Like:
def area(func, repeat, **kwargs):
for _ in range(repeat):
func(**kwargs)
So all parameters except func and repeat are stored in the kwargs dictionary, and later we call the func function (the first argument) with the named parameters.
Note that this will not work (correctly) for functions that require a func and/or repeat parameter, since we capture these at the area function level.
(disclaimed: not a Python kid, so please be gentle)
I am trying to compose functions using the following:
def compose(*functions):
return functools.reduce(lambda acc, f: lambda x: acc(f(x)), functions, lambda x: x)
which works as expected for scalar functions. I'd like to work with functions returning tuples and others taking multiple arguments, eg.
def dummy(name):
return (name, len(name), name.upper())
def transform(name, size, upper):
return (upper, -size, name)
# What I want to achieve using composition,
# ie. f = compose(transform, dummy)
transform(*dummy('Australia'))
=> ('AUSTRALIA', -9, 'Australia')
Since dummy returns a tuple and transform takes three arguments, I need to unpack the value.
How can I achieve this using my compose function above? If I try like this, I get:
f = compose(transform, dummy)
f('Australia')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in <lambda>
File "<stdin>", line 2, in <lambda>
TypeError: transform() takes exactly 3 arguments (1 given)
Is there a way to change compose such that it will unpack where needed?
This one works for your example but it wont handle just any arbitrary function - it will only works with positional arguments and (of course) the signature of any function must match the return value of the previous (wrt/ application order) one.
def compose(*functions):
return functools.reduce(
lambda f, g: lambda *args: f(*g(*args)),
functions,
lambda *args: args
)
Note that using reduce here, while certainly idiomatic in functional programming, is rather unpythonic. The "obvious" pythonic implementation would use iteration instead:
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
args = func(*args)
return args
return composed
Edit:
You ask "Is there a way to make have a compose function which will work in both cases" - "both cases" here meaning wether the functions returns an iterable or not (what you call "scalar functions", a concept that has no meaning in Python).
Using the iteration-based implementation, you could just test if the return value is iterable and wrap it in a tuple ie:
import collections
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
if not isinstance(args, collections.Iterable):
args = (args,)
args = func(*args)
return args
return composed
but this is not garanteed to work as expected - actually this is even garanteed to NOT work as expected for most use cases. There are a lot of builtin iterable types in Python (and even more user-defined ones) and just knowing an object is iterable doesn't say much about it's semantic.
For example a dict or str are iterable but in this case should obviously be considered a "scalar". A list is iterable too, and how it should be interpreted in this case is actually just undecidable without knowing exactly what it contains and what the "next" function in composition order expects - in some cases you will want to treat it as a single argument, in other cases ase a list of args.
IOW only the caller of the compose() function can really tell how each function result should be considered - actually you might even have cases where you want a tuple to be considered as a "scalar" value by the next function. So to make a long story short: no, there's no one-size-fits-all generic solution in Python. The best I could think of requires a combination of result inspection and manual wrapping of composed functions so the result is properly interpreted by the "composed" function but at this point manually composing the functions will be both way simpler and much more robust.
FWIW remember that Python is first and mostly a dynamically typed object oriented language so while it does have a decent support for functional programming idioms it's obviously not the best tool for real functional programming.
You might consider inserting a "function" (really, a class constructor) in your compose chain to signal the unpacking of the prior/inner function's results. You would then adjust your composer function to check for that class to determine if the prior result should be unpacked. (You actually end up doing the reverse: tuple-wrap all function results except those signaled to be unpacked -- and then have the composer unpack everything.) It adds overhead, it's not at all Pythonic, it's written in a terse lambda style, but it does accomplish the goal of being able to properly signal in a function chain when the composer should unpack a result. Consider the following generic code, which you can then adapt to your specific composition chain:
from functools import reduce
from operator import add
class upk: #class constructor signals composer to unpack prior result
def __init__(s,r): s.r = r #hold function's return for wrapper function
idt = lambda x: x #identity
wrp = lambda x: x.r if isinstance(x, upk) else (x,) #wrap all but unpackables
com = lambda *fs: ( #unpackable compose, unpacking whenever upk is encountered
reduce(lambda a,f: lambda *x: a(*wrp(f(*x))), fs, idt) )
foo = com(add, upk, divmod) #upk signals divmod's results should be unpacked
print(foo(6,4))
This circumvents the problem, as called out by prior answers/comments, of requiring your composer to guess which types of iterables should be unpacked. Of course, the cost is that you must explicitly insert upk into the callable chain whenever unpacking is required. In that sense, it is by no means "automatic", but it is still a fairly simple/terse way of achieving the intended result while avoiding unintended wraps/unwraps in many corner cases.
The compose function in the answer contributed by Bruno did do the job for functions with multiple arguments but didn't work any more for scalar ones unfortunately.
Using the fact that Python `unpacks' tuples into positional arguments, this is how I solved it:
import functools
def compose(*functions):
def pack(x): return x if type(x) is tuple else (x,)
return functools.reduce(
lambda acc, f: lambda *y: f(*pack(acc(*pack(y)))), reversed(functions), lambda *x: x)
which now works just as expected, eg.
#########################
# scalar-valued functions
#########################
def a(x): return x + 1
def b(x): return -x
# explicit
> a(b(b(a(15))))
# => 17
# compose
> compose(a, b, b, a)(15)
=> 17
########################
# tuple-valued functions
########################
def dummy(x):
return (x.upper(), len(x), x)
def trans(a, b, c):
return (b, c, a)
# explicit
> trans(*dummy('Australia'))
# => ('AUSTRALIA', 9, 'Australia')
# compose
> compose(trans, dummy)('Australia')
# => ('AUSTRALIA', 9, 'Australia')
And this also works with multiple arguments:
def add(x, y): return x + y
# explicit
> b(a(add(5, 3)))
=> -9
# compose
> compose(b, a, add)(5, 3)
=> -9
I'm doing an exercise where I'm to create a class representing functions (written as lambda expressions) and several methods involving them.
The ones I've written so far are:
class Func():
def __init__(self, func, domain):
self.func = func
self.domain = domain
def __call__(self, x):
if self.domain(x):
return self.func(x)
return None
def compose(self, other):
comp_func= lambda x: self.func(other(x))
comp_dom= lambda x: other.domain(x) and self.domain(other(x))
return Func(comp_func, comp_dom)
def exp(self, k):
exp_func= self
for i in range(k-1):
exp_func = Func.compose(exp_func, self)
return exp_func
As you can see above, the function exp composes a function with itself k-1 times. Now I'm to write a recursive version of said function, taking the same arguments "self" and "k".
However I'm having difficulty figuring out how it would work. In the original exp I wrote I had access to the original function "self" throughout all iterations, however when making a recursive function I lose access to the original function and with each iteration only have access to the most recent composed function. So for example, if I try composing self with self a certain number of times I will get:
f= x+3
f^2= x+6
(f^2)^2= x+12
So we skipped the function x+9.
How do I get around this? Is there a way to still retain access to the original function?
Update:
def exp_rec(self, k):
if k==1:
return self
return Func.compose(Func.exp_rec(self, k-1), self)
This is an exercise, so I won't provide the answer.
In recursion, you want to do two things:
Determine and check a "guard condition" that tells you when to stop; and
Determine and compute the "recurrence relation" that tells you the next value.
Consider a simple factorial function:
def fact(n):
if n == 1:
return 1
return n * fact(n - 1)
In this example, the guard condition is fairly obvious- it's the only conditional statement! And the recurrence relation is in the return statement.
For your exercise, things are slightly less obvious, since the intent is to define a function composition, rather than a straight integer computation. But consider:
f = Func(lambda x: x + 3)
(This is your example.) You want f.exp(1) to be the same as f, and f.exp(2) to be f(f(x)). That right there tells you the guard condition and the recurrence relation:
The guard condition is that exp() only works for positive numbers. This is because exp(0) might have to return different things for different input types (what does exp(0) return when f = Func(lambda s: s + '!') ?).
So test for exp(1), and let that condition be the original lambda.
Then, when recursively defining exp(n+1), let that be the composition of your original lambda with exp(n).
You have several things to consider: First, your class instance has data associated with it. That data will "travel along" with you in your recursion, so you don't have to pass so many parameters recursively. Second, you need to decide whether Func.exp() should create a new Func(), or whether it should modify the existing Func object. Finally, consider how you would write a hard-coded function, Func.exp2() that just constructed what we would call Func.exp(2). That should give you an idea of your recurrence relation.
Update
Based on some comments, I feel like I should show this code. If you are going to have your recursive function modify the self object, instead of returning a new object, then you will need to "cache" the values from self before they get modified, like so:
func = self.func
domain = self.domain
... recursive function modifies self.func and self.domain
I often have to randomly generate stuff with certain constraints. In many cases, it's quicker to ignore the constraints in generation, check if they are met afterwards and redo the process otherwise. Lacking a do keyword, I usually write
r = random_stuff()
while not meets_condition(r):
r = random_stuff()
That's a bit ugly, as I have the same line of code twice. What I'd really like to have is a construct like
r = random_stuff() until meets_condition(r)
similar to the ternary operator introduced in 2.5:
a = b if condition else c
Just that here condition is evaluated before the left-hand side of the statement is executed. Does anybody have a suggestion for a design pattern (should work in Python 2.7) that remedies the while-constructs intrinsic unpythonic ugliness?
while True:
r = random_stuff()
if meets_condition(r):
break
or
condition = True
while condition:
r = random_stuff()
condition = not meets_condition(r)
Your idea is not bad - but not with the new keyword until, but rather like
a = (<expression> while <condition>)
extending the idea of generator expressions.
As they don't exist, it won't help you.
But what you can use, is the iter function with the sentinel.
dummy_sentinel = object()
for r in iter(random_stuff, dummy_sentinel):
if meets_condition(r): break
If you can be sure about your random_stuff() returning only a certain kind of values, such as numbers, strings, etc., you can take some other value as sentinel. Especially, when None never can occur, take this, in order to have a never-ending generator.
for r in iter(random_stuff, None):
if meets_condition(r): break
Then random_stuff() gets called until it meets the condition.
Even better might be
r = next(r for r in iter(random_stuff, None) if meets_condition(r))
which gives you the first matching one.
Maybe syntactic sugar is what the Dr. ordered? You could do something like this.
I was too lazy to handle kw args in find_condition. You get what you pay for :D.
def find_condition(cond_mk, cond_ck, *args):
"""
.. function:: find_condition(cond_mk, cond_ck) -> cond
Create conditions by calling cond_mk until one is found that passes
the condition check, cond_ck. Once cond_ck returns True, iteration
over cond_mk stops and the last value processed is returned.
** WARNING **
This function could loop infinitely.
``cond_mk`` - callable that creates a condition. It's return value is
passed to cond_ck for verification.
``cond_ck`` - callable that checks the return value of cond_mk.
``args`` - any arguments to pass to cond_mk should be supplied here.
"""
v = cond_mk(*args)
while not cond_ck(v):
v = cond_mk(*args)
return v
# Test it out..
import random
random.seed()
print find_condition(random.randint, lambda x: x > 95, 1, 100)
while not meets_condition(random_stuff()): pass
If you actually need the random_stuff() then it can be stored elsewhere as a side effect (e.g. make random_stuff the __call__ method of a class).
Okay, inspired by #jaime I wrote the following decorator:
def retry(condition):
def deco_retry(f):
def f_retry(*args, **kwargs):
success = False
while not success:
result = f(*args, **kwargs)
success = condition(result)
return result
return f_retry
return deco_retry
Now, the following works:
def condition(calue):
return value < .5
#retry(condition)
def random_stuff():
return random.random()
print random_stuff()
Also, inline:
#retry(lambda x: x < .5)
def random_stuff():
return random.random()
print random_stuff()
However, the retry is now bound to the random_stuff() method, which can now only be used with the condition with which it was decorated. Also, it doesn't work for instance methods (as in #retry(self.condition)). Any ideas to circumvent that?