I have a Python script which creates a dictionary of its own functions and I'd like it to execute them by reading in a function name and arguments using YARP (knowledge of YARP is irrelevant to this question though).
I create a list of strings called "inc" which is populated by values coming into the program. The first item is a function name, and any other strings in the list are arguments. I create a dictionary called "methods" where the key is the function name and the value is a reference to the function object (using the inspect module). I store the return value of the function in a variable "result".
The snippet below shows a simplified version of what I'm using so far, which works fine, but can't handle functions with more than one argument. To circumvent this I use a list if a function needs more parameters:
if len(inc) == 1:
result = methods[inc[0]]() # call method with 0 arguments
elif len(inc) == 2:
result = methods[inc[0]](inc[1]) # call method passing a string
else:
args = []
result = methods(inc[0])(inc[1:]) # call method passing a list
Ideally, I'd like to change this so that my functions can have any number of arguments, but I can't figure out how I can do this. I'm new to Python and I have looked at the documentation and various websites - I just can't find a solution. I've tried things like creating a tuple of the arguments, but that doesn't work either as it ends up passing the whole tuple in as one parameter.
Is there a better solution to this problem, like creating some kind of object which represents a set of parameters and passing that into the function? Any suggestions would be greatly appreciated!
You should check out https://stackoverflow.com/a/3394898/1395668.
The magic you are looking for is the *. Apply this to your list and it unpacks the items into the argument fields of your function:
a = [ 1, 2, 3]
def myfunc(a, b, c):
return a + b + c
print myfunc(*a)
Check out ** for the same approach for dict
It's a bit strange to have this kind of mixed structure:
inc = [func_name, arg1, arg2, ...]
Wouldn't it be much more natural to have two separate bits of information?
func_name = ...
args = [arg1, arg2, ...]
The you could do
methods[func_name](*args)
(Usually, I wouldn't bind the functions name to a variable, but preferably the function itself.)
Related
I have a question about arguments in functions, in particular initialising an array or other data structure within the function call, like the following:
def helper(root, result = []):
...
My question is, what is the difference between the above vs. doing:
def helper(root):
result = []
I can see why this would be necessary if we were to run recursions, i.e. we would need to use the first case in some instances.
But are there any other instances, and am I right in saying it is necessary in some cases for recursion, or can we always use the latter instead?
Thanks
Python uses pointers for lists, so initializing a list or any other mutable objects in function definition is a bad idea.
The best way of doing it is like this:
def helper(root, result=None):
if isinstance(result, type(None)):
result = []
Now if you only pass one argument to the function, the "result" will be an empty list.
If you initiate the list within the function definition, by calling the function multiple times, "result" won't reset and it will keep the values from previous calls.
I have seen lots of information on storing functions in variables for execution, but the functions shown are always of the:
def foo():
print("Hello")
x = foo
x()
variety. Is it possible to store a function and its variables for later execution? I've currently worked around this by creating a tuple with the function as the first item and the variables as a nested tuple for the second item and then calling:
menu_input[KEY_D] = action_system.drop_item, (character,))
...LATER IN CODE...
for key in current_input:
if key in menu.menu_input.keys():
func, args = menu.menu_input[keys]
func(*args)
but I would really prefer to be able to store the function and its variables all together.
I would prefer to be able to store:
menu_input[KEY_D] = action_system.drop_item(character)
...LATER IN CODE...
for key in current_input:
if key in menu.menu_input.keys():
menu.menu_input[keys]()
because the function that handles menu input does not understand or care about the menu function itself, so it really doesn't need to see the input or care about the arguments passed.
Is this possible? If so, what am I missing? Sorry if this is obvious or obviously a bad idea--relatively new to python. If it is a terrible idea, I'd love to know why. I'm avoiding making this a class function of the menu item because I'm trying to work within an entity-component-system model where components store all data and the systems operate on them.
The functools.partial function is made for this purppose:
from functools import partial
menu_input[KEY_D] = partial(action_system.drop_item, character)
...
for key in current_input:
if key in menu.menu_input:
menu.menu_input[key]()
You could choose to use a lambda expression with no arguments.
def foo(value):
print(value)
x = lambda : foo("Hello")
x()
I am looking for a high order function in python that takes in a function as parameter and a list of corresponding parameters, and call the function on the parameter list. Something like:
def exec(func,paramlist):
return CALL(func,paramlist)
The paramlist length is undetermined, since it goes with each func passed in. The CALL should be able to extract the elements in the list and put each parameter in the right slot and make the function call.
For reference, in Q language there is this "apply" function that handles generic function calls:
f1: {x}
f2: {x+y}
execFunction: {[fun;param] .[fun;param] }
execFunction[f1;enlist 1] // result is 1
execFunction[f2;(1 2)] // result is 3
If func expects multiple parameters and paramlist is a list of parameters, is as simple as this:
func(*paramlist)
Python used to have an apply function but it was removed in Python 3 because it is so easy to unpack a list into individual arguments using the * operator. In fact an apply function can be written like so:
def apply(func, paramlist)
return func(*paramlist)
This is so trivial, of course, that you wouldn't bother to write a function for it; you'd just write func(*paramlist) wherever you needed it.
I recently attempted Googles foo.bar challenge. After my time was up I decided to try find a solution to the problem I couldn't do and found a solution here (includes the problem statement if you're interested). I'd previously been making a dictionary for every function I wanted to cache but it looks like in this solution any function/input can be cached using the same syntax.
Firstly I'm confused on how the code is even working, the *args variable isn't inputted as an argument (and prints to nothing). Heres an modified minimal example to illustrate my confusion:
mem = {}
def memoize(key, func, *args):
"""
Helper to memoize the output of a function
"""
print(args)
if key not in mem:
# store the output of the function in memory
mem[key] = func(*args)
return mem[key]
def example(n):
return memoize(
n,
lambda: longrun(n),
)
def example2(n):
return memoize(
n,
longrun(n),
)
def longrun(n):
for i in range(10000):
for j in range(100000):
2**10
return n
Here I use the same memoize function but with a print. The function example returns memoize(n, a lambda function,). The function longrun is just an identity function with lots of useless computation so it's easy to see if the cache is working (example(2) will take ~5 seconds the first time and be almost instant after).
Here are my confusions:
Why is the third argument of memoize empty? When args is printed in memoize it prints (). Yet somehow mem[key] stores func(*args) as func(key)?
Why does this behavior only work when using the lambda function (example will cache but example2 won't)? I thought lambda: longrun(n) is just a short way of giving as input a function which returns longrun(n).
As a bonus, does anyone know how you could memoize functions using a decorator?
Also I couldn't think of a more descriptive title, edits welcome. Thanks.
The notation *args stands for a variable number of positional arguments. For example, print can be used as print(1), print(1, 2), print(1, 2, 3) and so on. Similarly, **kwargs stands for a variable number of keyword arguments.
Note that the names args and kwargs are just a convention - it's the * and ** symbols that make them variadic.
Anyways, memoize uses this to accept basically any input to func. If the result of func isn't cached, it's called with the arguments. In a function call, *args is basically the reverse of *args in a function definition. For example, the following are equivalent:
# provide *args explicitly
print(1, 2, 3)
# unpack iterable to *args
arguments = 1, 2, 3
print(*arguments)
If args is empty, then calling print(*args) is the same as calling print() - no arguments are passed to it.
Functions and lambda functions are the same in python. It's simply a different notation for creating a function object.
The problem is that in example2, you are not passing a function. You call a function, then pass on its result. Instead, you have to pass on the function and its argument separately.
def example2(n):
return memoize(
n,
longrun, # no () means no call, just the function object
# all following parameters are put into *args
n
)
Now, some implementation details: why is args empty and why is there a separate key?
The empty args comes from your definition of the lambda. Let's write that as a function for clarity:
def example3(n):
def nonlambda():
return longrun(n)
return memoize(n, nonlambda)
Note how nonlambda takes no arguments. The parameter n is bound from the containing scope as a closure, bound from the containing scope. As such, you don't have to pass it to memoize - it is already bound inside the nonlambda. Thus, args is empty in memoize, even though longrun does receive a parameter, because the two don't interact directly.
Now, why is it mem[key] = f(*args), not mem[key] = f(key)? That's actually slightly the wrong question; the right question is "why isn't it mem[f, args] = f(*args)?".
Memoization works because the same input to the same function leads to the same output. That is, f, args identifies your output. Ideally, your key would be f, args as that's the only relevant information.
The problem is you need a way to look up f and args inside mem. If you ever tried putting a list inside a dict, you know there are some types which don't work in mappings (or any other suitable lookup structure, for that matter). So if you define key = f, args, you cannot memoize functions taking mutable/unhashable types. Python's functools.lru_cache actually has this limitation.
Defining an explicit key is one way of solving this problem. It has the advantage that the caller can select an appropriate key, for example taking n without any modifications. This offers the best optimization potential. However, it breaks easily - using just n misses out the actual function called. Memoizing a second function with the same input would break your cache.
There are alternative approaches, each with pros and cons. Common is the explicit conversion of types: list to tuple, set to frozenset, and so on. This is slow, but the most precise. Another approach is to just call str or repr as in key = repr((f, args, sorted(kwargs.items()))), but it relies on every value having a proper repr.
Okay this one is confusing. My old piece of code has something like
map(lambda x:x.func1(arg1), other_args_to_be_mapped)
now I would like to make arg1 -> *args
while other_args_to_be_mapped stays unchanged.
in func1, the length of arguments will be checked different operations. My questions are
1) which length will be checked? arg1 or other_args_to_be_mapped
2) in func1, how should I set up the default? It was like
def func1(arg1=something)
but now with potential multiple arguments, I don't know what to do with the initialization. I want to be able to do something like
def func1(args*=something, something_else)
Is that even possible?
If I understand your question correctly, you're looking for variable arguments. These can be mixed with fixed arguments, provided you obey a logical ordering (fixed arguments first, then keyword arguments or variable arguments).
For example, the following shows how map to a function that takes in one constant argument and one variable argument. If you would like different behaviour, please provide a concrete example of what you are trying to accomplish
import random
class Foo:
def get_variable_parameters(self):
return [1] if random.random() > .5 else [1,2]
def foo( self, arg, *args ):
print("Doing stuff with constant arg", arg)
if len(args) == 1:
print("Good",args)
else:
print("Bad",args)
list(map( lambda x : x.foo( 'Static Argument', *x.get_variable_parameters()), [Foo(),Foo(),Foo()] ))
We don't know how many arguments are going to be passed to foo (in this trivial case, it's one or two), but the "*" notation accepts any number of objects to be passed
Note I've encapsulated map in list so that it gets evaluated, as in python3 it is a generator. List comprehension may be more idiomatic in python. Also don't forget you can always use a simple for loop - an obfuscated or complex map call is far less pythonic than a clear (but several line) for-loop, imo.
If, rather, you're trying to combine multiple arguments in a map call, I would recommend using the same variable argument strategy with the zip function, e.g.,
def foo(a,*b): ...
map(lambda x : foo(x[0],*x[1]), zip(['a','b'],[ [1], [1,2] ]))
In this case, foo will get called first as foo('a',1), and then as foo('b',2,3)