We Keep Coding

Decorating a function to create a class that splits keyword and positional arguments into two other methods

I have an abstract class that #abstractmethods its __init__ and __call__. In principle, the class definition is like this:
class Base(abc.ABC):
#abc.abstractmethod
def __init__(self, **params):
pass
#abc.abstractmethod
def __call__(self, *input):
pass
I want to create a decorator that converts a function's keyword arguments into arguments into __init__ and positional arguments into __call__: so if
def func(a, b, *args, k=1, g=2, **kwargs):
pass # it does something
then I would like to wrap my func in a decorator that will output like along the lines of
class Func(Base):
def __init__(self, k=1, g=2, **kwargs):
inspected_kwargs = ... # arguments of init in dictionary form
for argname, val in inspected_kwargs.items():
setattr(self, argname, val)
def __call__(self, a, b, *args):
inspected_args = ... # arguments of call in tuple form
return func(*inspected_args, **self.__dict__)
I had a few attempts using inspect but I'm not sure how to inject the parameters from inspect.Signature object back to create a function. Also, I had some trouble because when local methods are added to a class that is inner to a function with simple attribute setting, the local functions are no longer in scope when the class is returned.
The question is: how to create a class inside a function whose methods are defined based on the signature of the function's input callable?
#edit
To provide more details on the idea:
def decorator(function):
class Inner(Base):
pass
# non-existent functions ahead, just to convey the idea
# the exact way to do this is exactly the matter of this question
sig = inspect.get_signature(function)
init = create_from_signature(signature=sig.keyword_arguments(),
body="for name, val in sig.keyword_arguments():\nsetattr(self, name, val)")
call = create_from_signature(signature=sig.positional(),
body="return partial(func, **self.__dict__)")
Inner.__init__ = init
Inner.__call__ = call
# it would be nice if Inner class name would also depend on
# func name:
setattr(Inner, __name__, "_".join(function.__name__, "decorated"))
return Inner
def users_function(a, b, k=5):
# user's code; does whatever, for me it's a black box
return a * b / k
Expected behaviour:
decorator(users_function)(k=10)(1, 2) == users_function(1, 2, k=10)
Why do I want to do this? Because then one can call the methods of Base on the outputted object, for which knowledge of only keyword arguments would be required:
my_obj = decorator(users_function)(k=10)
my_obj.basemethod() # basemethod is implementd in Base class

I am trying to understand exactly what your problem is so let's start with this example:
def decorator(function):
def wrapper(*args, **kwargs):
print(' [decorator] Printing before function [' + function.__name__ + '] execution')
result = function(*args, **kwargs)
print(' [decorator] Found args: ' + str(args))
print(' [decorator] Found kwargs: ' + str(kwargs))
print(' [decorator] Printing after function [' + function.__name__ + '] execution')
return result
return wrapper
#decorator
def MyFunction(*args, **kwargs):
print('[MyFunction] Received args:' + str(args))
print('[MyFunction] Received kwargs:' + str(kwargs))
MyFunction(1, 2, kwarg1=3, kwarg2=2)
It will result in:
[decorator] Printing before function [MyFunction] execution
[MyFunction] Received args:(1, 2)
[MyFunction] Received kwargs:{'kwarg1': 3, 'kwarg2': 2}
[decorator] Found args: (1, 2)
[decorator] Found kwargs: {'kwarg1': 3, 'kwarg2': 2}
[decorator] Printing after function [MyFunction] execution
Where do you need to take your problem from here?

What is the benefit of using kwargs (or args) over a simple dict?

I'd like to get idea why should I use kwargs or args over passing in a simple dict (or tuple in case of args)?
I wrote a very simple code snippet to check what exactly happens and I can't find any pros to use kwargs over a dict. If anyone could tell me why should I use those I'd be happy.
Now as I can see it just more pythonic but don't makes any difference. Also if you use a simple dict then it's more readable because all the languages can do that but not the kwargs way.
def test_normal(input: dict):
for element in input.items():
print('Type: {}, raw: {}'.format(type(input), input))
print('key: {}, value: {}'.format(element[0], element[1]))
def test_kwargs(**kwargs):
for element in kwargs.items():
print('Type: {}, raw: {}'.format(type(kwargs), kwargs))
print('key: {}, value: {}'.format(element[0], element[1]))
test_normal(dict(name='Joseph'))
test_kwargs(name='Joseph')
Type: <class 'dict'>, raw: {'name': 'Joseph'}
key: name, value: Joseph
Type: <class 'dict'>, raw: {'name': 'Joseph'}
key: name, value: Joseph

These are different things and both have their use-cases. Just a rule of thumb: If it looks like a function parameter, it should be a function parameter.
There are several neat use-cases for *args and **kwargs. One of which is passing through parameters that you don't care about at this point:
Say you have class Base and class A inherited from Base:
class Base:
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
class A(Base):
def __init__(self, n, *args, **kwargs):
super().__init__(*args, **kwargs)
self.n = n
As you see class A does not care about Base's __init__ parameters, so it just passes everything (except n that it needs) forward. So if you were to change Base's __init__ you would not need to change A.
But when you create A object you would pass parameters normally:
a = A(5, 3, y=6, z=42)
A similar idea is when you implement a decorator which you'd like to use on a function with any kind and number of arguments:
def say_hello_first(fn):
def wrapper(*args, *kwargs):
print('Hello')
return fn(*args, **kwargs)
return wrapper
#say_hello_first
def foo(x):
print(x)
#say_hello_first
def bar(a, b, c=3.14):
print((a + b) * c)
then:
>>> foo(42)
Hello
42
>>> bar(1, 2, c=3)
Hello
9

Master function that executes subfunctions

I have two functions that accept different parametes:
def foo(name, age):
pass
def bar(color, shape):
pass
Now, I have a master function that I want to be able to call with the function I want to execute and it's parameters. Since it's a master function that might call either foo or bar, it's called with only two params (the function to be executed and the parameters for this function.
function is a string telling what function to execute
params will be a dictionary of parameters (like **kwargs)
I can do this to make it work:
def master(function, params):
if function == 'foo':
foo(params['name'], params['age'])
elif function == 'bar':
foo(params['color'], params['shape'])
And then I call master like:
master('foo',{'name': 'John', 'age': 99})
However if master has a lot of subfuntions to call, there's too much conditions and picking the right parameters for each function.
So I basically have two questions:
1) Instead of calling master with the name of the function and then checking this name in a condition, can I directly pass the function to be executed? If so, how do I execute the function then?
Something like calling master like this:
master(foo(), {'name': 'John', 'age': 99})
2) functions foo and bar don't have **kwargs, however it would be very convinient if I can call them passing just a dictionary and then they assign to each variable their corresponding value from the dict.
So basically, could I do:
params = {'name':'John', 'age':99, 'color':red, 'shape':circle}
foo(params) # I would like to to this even if foo doesn't have **kwargs
bar(params) # same for bar
So at the end my ideal call of master would be:
params = {'name':'John', 'age':99, 'color':red, 'shape':circle}
master(foo(), params) # to execute foo
master(bar(), params) # to execute bar

You can pass functions as arguments:
def master(func, arguments: dict):
if func is foo:
args = arguments["name"], arguments["age"]
elif func is bar:
args = arguments["color"], arguments["shape"]
return func(*args)
This can be done even simpler if you don't know the names of the functions' arguments:
def master(func, arguments: list):
return func(*arguments)
A much more generic version is the following:
def master(function, *positional_arguments, **keyword_arguments):
function(*positional_arguments, **keyword_arguments)
master(foo, 'John', 56)
master(foo, **{'name': 'John', 'age': 56})
master(foo, name='John', age=56)
master(foo, *['John', 56])

Function in python are objects, so yes, you can pass them as a parameter (but do not use parenthesis).
def master(function, **kwargs):
function(params, kwargs)
Then you call the master:
master(foo, name='John', age=99...)

You can execute a function through master function not by passing a function as a parameter inside master, but by passing the definition of the function. Please follow the example below:
def foo(name, age):
pass
def bar(color, shape):
pass
Now consider a master function:
Func param will contain the defination of the function by adding parentheses after function name, it will make it to execute.
def masterFunc(func, params):
return func(params) # Func param will contain the defination of the function by adding parentheses after function name, it will make it to execute.
You will use this master function to execute the passed function defination as below:
masterFunc(foo,{a:1}) #Sample execution

function is a first-class object in Python and therefore can be assigned to an identifier, passed as an argument or returned by a function.
The unified way where you don't want to bother of keyword args for a particular function - with inspect.getfullargspec feature:
import inspect
def foo(name, age):
print(name, age)
def bar(color, shape):
print(color, shape)
def master(func, params):
arg_names = inspect.getfullargspec(func).args
func(**{k:v for k,v in params.items() if k in arg_names})
params = {'name':'John', 'age':99, 'color':'red', 'shape':'circle'}
master(foo, params)
master(bar, params)
Sample output:
John 99
red circle

In Python, functions are objects, like everything else. You can pass functions as parameters and execute them very simply.
In your case,
def say_hello(name):
print "hi ", name
def say_bye(name, time):
print name, ", have a good ", time
say_bye("john", "night")
john , have a good night
def master(fun, **kwargs):
fun(**kwargs)
master(say_bye, name='john', time='night')
john , have a good night
params1 = {'name': 'jay', 'time': 'morning'}
params2 = {'name': 'cole'}
master(say_hello, **params2)
hi cole
master(say_bye, **params1)
jay , have a good morning
this should work.

You can try with the below approach.
Code
class A:
def a(self, *args, **kwargs):
print('A')
def b(self, *args, **kwargs):
print('B', args, kwargs)
def main(func, *args, **kwargs):
x = A()
if hasattr(x, func):
func = getattr(x, func)
func(*args, **kwargs)
else:
print('No function name {} defined under class'.format(func))
if __name__ == '__main__':
func = raw_input('Func_name: a/b')
main(func, (1), {1:1})
Output
~/del$ python test.py
Func_name: a/ba
True
A
~/del$ python test.py
Func_name: a/bb
True
('B', (1, {1: 1}), {})
~/del$
Here, we're using getattr function in __builtin__.
* First it checks whether there's a function/method named a or b under the instance of class A.
* If yes, then only it will getattr from it and execute passing the args and kwargs.
* If somebody passes function name not defined, it will not crash and return saying the needful statement there.
Hope this helps !

Python __getattribute__ (or __getattr__) to emulate php __call

I would like to create a class that effectively does this (mixing a little PHP with Python)
class Middle(object) :
# self.apply is a function that applies a function to a list
# e.g self.apply = [] ... self.apply.append(foobar)
def __call(self, name, *args) :
self.apply(name, *args)
Thus allowing for code to say:
m = Middle()
m.process_foo(a, b, c)
In this case __call() is the PHP __call() method which is invoked when a method is not found on an object.

You need to define __getattr__, it is called if an attribute is not otherwise found on your object.
Notice that getattr is called for any failed lookup, and that you don't get it like a function all, so you have to return the method that will be called.
def __getattr__(self, attr):
def default_method(*args):
self.apply(attr, *args)
return default_method

Consider passing arguments to your methods as arguments, not encoded into the method name which will then be magically used as an argument.
Where are you writing code that doesn't know what methods it will be calling?
Why call c.do_Something(x) and then unpack the method name instead of just calling c.do('Something', x) ?
In any case it's easy enough to handle unfound attributes:
class Dispatcher(object):
def __getattr__(self, key):
try:
return object.__getattr__(self, key)
except AttributeError:
return self.dispatch(key)
def default(self, *args, **kw):
print "Assuming default method"
print args, kw
def dispatch(self, key):
print 'Looking for method: %s'%(key,)
return self.default
A test:
>>> d = Dispatcher()
>>> d.hello()
Looking for method: hello
Assuming default method
() {}
This seems to be fraught with "gotchas" - the thing returned by getattr is going to be presumed to be not just a function, but a bound method on that instance. So be sure to return that.

I actually did this recently. Here's an example of how I solved it:
class Example:
def FUNC_1(self, arg):
return arg - 1
def FUNC_2(self, arg):
return arg - 2
def decode(self, func, arg):
try:
exec( "result = self.FUNC_%s(arg)" % (func) )
except AttributeError:
# Call your default method here
result = self.default(arg)
return result
def default(self, arg):
return arg
and the output:
>>> dude = Example()
>>> print dude.decode(1, 0)
-1
>>> print dude.decode(2, 10)
8
>>> print dude.decode(3, 5)
5

Getting the keyword arguments actually passed to a Python method

I'm dreaming of a Python method with explicit keyword args:
def func(a=None, b=None, c=None):
for arg, val in magic_arg_dict.items(): # Where do I get the magic?
print '%s: %s' % (arg, val)
I want to get a dictionary of only those arguments the caller actually passed into the method, just like **kwargs, but I don't want the caller to be able to pass any old random args, unlike **kwargs.
>>> func(b=2)
b: 2
>>> func(a=3, c=5)
a: 3
c: 5
So: is there such an incantation? In my case, I happen to be able to compare each argument against its default to find the ones that are different, but this is kind of inelegant and gets tedious when you have nine arguments. For bonus points, provide an incantation that can tell me even when the caller passes in a keyword argument assigned its default value:
>>> func(a=None)
a: None
Tricksy!
Edit: The (lexical) function signature has to remain intact. It's part of a public API, and the primary worth of the explicit keyword args lies in their documentary value. Just to make things interesting. :)

I was inspired by lost-theory's decorator goodness, and after playing about with it for a bit came up with this:
def actual_kwargs():
"""
Decorator that provides the wrapped function with an attribute 'actual_kwargs'
containing just those keyword arguments actually passed in to the function.
"""
def decorator(function):
def inner(*args, **kwargs):
inner.actual_kwargs = kwargs
return function(*args, **kwargs)
return inner
return decorator
if __name__ == "__main__":
#actual_kwargs()
def func(msg, a=None, b=False, c='', d=0):
print msg
for arg, val in sorted(func.actual_kwargs.iteritems()):
print ' %s: %s' % (arg, val)
func("I'm only passing a", a='a')
func("Here's b and c", b=True, c='c')
func("All defaults", a=None, b=False, c='', d=0)
func("Nothin'")
try:
func("Invalid kwarg", e="bogon")
except TypeError, err:
print 'Invalid kwarg\n %s' % err
Which prints this:
I'm only passing a
a: a
Here's b and c
b: True
c: c
All defaults
a: None
b: False
c:
d: 0
Nothin'
Invalid kwarg
func() got an unexpected keyword argument 'e'
I'm happy with this. A more flexible approach is to pass the name of the attribute you want to use to the decorator, instead of hard-coding it to 'actual_kwargs', but this is the simplest approach that illustrates the solution.
Mmm, Python is tasty.

Here is the easiest and simplest way:
def func(a=None, b=None, c=None):
args = locals().copy()
print args
func(2, "egg")
This give the output: {'a': 2, 'c': None, 'b': 'egg'}.
The reason args should be a copy of the locals dictionary is that dictionaries are mutable, so if you created any local variables in this function args would contain all of the local variables and their values, not just the arguments.
More documentation on the built-in locals function here.

One possibility:
def f(**kw):
acceptable_names = set('a', 'b', 'c')
if not (set(kw) <= acceptable_names):
raise WhateverYouWantException(whatever)
...proceed...
IOW, it's very easy to check that the passed-in names are within the acceptable set and otherwise raise whatever you'd want Python to raise (TypeError, I guess;-). Pretty easy to turn into a decorator, btw.
Another possibility:
_sentinel = object():
def f(a=_sentinel, b=_sentinel, c=_sentinel):
...proceed with checks `is _sentinel`...
by making a unique object _sentinel you remove the risk that the caller might be accidentally passing None (or other non-unique default values the caller could possibly pass). This is all object() is good for, btw: an extremely-lightweight, unique sentinel that cannot possibly be accidentally confused with any other object (when you check with the is operator).
Either solution is preferable for slightly different problems.

How about using a decorator to validate the incoming kwargs?
def validate_kwargs(*keys):
def entangle(f):
def inner(*args, **kwargs):
for key in kwargs:
if not key in keys:
raise ValueError("Received bad kwarg: '%s', expected: %s" % (key, keys))
return f(*args, **kwargs)
return inner
return entangle
###
#validate_kwargs('a', 'b', 'c')
def func(**kwargs):
for arg,val in kwargs.items():
print arg, "->", val
func(b=2)
print '----'
func(a=3, c=5)
print '----'
func(d='not gonna work')
Gives this output:
b -> 2
----
a -> 3
c -> 5
----
Traceback (most recent call last):
File "kwargs.py", line 20, in <module>
func(d='not gonna work')
File "kwargs.py", line 6, in inner
raise ValueError("Received bad kwarg: '%s', expected: %s" % (key, keys))
ValueError: Received bad kwarg: 'd', expected: ('a', 'b', 'c')

This is easiest accomplished with a single instance of a sentry object:
# Top of module, does not need to be exposed in __all__
missing = {}
# Function prototype
def myFunc(a = missing, b = missing, c = missing):
if a is not missing:
# User specified argument a
if b is missing:
# User did not specify argument b
The nice thing about this approach is that, since we're using the "is" operator, the caller can pass an empty dict as the argument value, and we'll still pick up that they did not mean to pass it. We also avoid nasty decorators this way, and keep our code a little cleaner.

There's probably better ways to do this, but here's my take:
def CompareArgs(argdict, **kwargs):
if not set(argdict.keys()) <= set(kwargs.keys()):
# not <= may seem weird, but comparing sets sometimes gives weird results.
# set1 <= set2 means that all items in set 1 are present in set 2
raise ValueError("invalid args")
def foo(**kwargs):
# we declare foo's "standard" args to be a, b, c
CompareArgs(kwargs, a=None, b=None, c=None)
print "Inside foo"
if __name__ == "__main__":
foo(a=1)
foo(a=1, b=3)
foo(a=1, b=3, c=5)
foo(c=10)
foo(bar=6)
and its output:
Inside foo
Inside foo
Inside foo
Inside foo
Traceback (most recent call last):
File "a.py", line 18, in
foo(bar=6)
File "a.py", line 9, in foo
CompareArgs(kwargs, a=None, b=None, c=None)
File "a.py", line 5, in CompareArgs
raise ValueError("invalid args")
ValueError: invalid args
This could probably be converted to a decorator, but my decorators need work. Left as an exercise to the reader :P

Perhaps raise an error if they pass any *args?
def func(*args, **kwargs):
if args:
raise TypeError("no positional args allowed")
arg1 = kwargs.pop("arg1", "default")
if kwargs:
raise TypeError("unknown args " + str(kwargs.keys()))
It'd be simple to factor it into taking a list of varnames or a generic parsing function to use. It wouldn't be too hard to make this into a decorator (python 3.1), too:
def OnlyKwargs(func):
allowed = func.__code__.co_varnames
def wrap(*args, **kwargs):
assert not args
# or whatever logic you need wrt required args
assert sorted(allowed) == sorted(kwargs)
return func(**kwargs)
Note: i'm not sure how well this would work around already wrapped functions or functions that have *args or **kwargs already.

Magic is not the answer:
def funky(a=None, b=None, c=None):
for name, value in [('a', a), ('b', b), ('c', c)]:
print name, value