passing functions and its arguments to another function - python

I have tree types of sub-functions:
one without any parameters (arguments),
second with one parameter
third with multiple parameters (tuple)
I am trying to pass that functions and its arguments to another function which sum results of all sub-functions and return the sum value.
Parameters in that function should be: names of each sub-function as position arguments (*args) and arguments of each subfunction as key-value arguments (*kvargs).
Example:
def no_arg()
def one_arg(a)
def multiple_args(a, b, c, e, f)
# execution of function_results_sum:
function_results_sum(
no_arg, one_arg, multiple_args,
one_arg=23,
multiple_args=(1, 2, 3, 4, 5))
What i have done so far:
def no_arg():
return 5
def ident(x):
return x
def mult(x, y):
return x * y
def function_results_sum(*args, **kwargs):
return no_arg() + ident(kwargs[ident.__name__]) + mult(*kwargs[mult.__name__])
The code above is passing arguments to each sub-function, but sub-function names are hardcoded. I would like to modify the current code to be able to get function names from *args. Below I wrote a pseudocode expressing more less what i am trying to achieve:
def function_results_sum(*args, **kwargs):
for functionName in args:
result = sum(funcionName(kwargs))
return result
I have already spent all day struggling with that problem, so please don't write me that "using google doesn't hurt" ;)

Something like this would work:
def no_arg():
return 5
def one_arg(x):
return x
def multiple_args(x, y):
return x * y
def function_results_sum(*args, **kwargs):
result = 0
for func in args:
result += func(*kwargs[func.__name__])
return result
Output:
function_results_sum(
no_arg, one_arg, multiple_args,
no_arg=(),
one_arg=(23, ),
multiple_args=(1,5))
33
The only difference between what you are asking is that you have to put args in a tuple to then unpack as args to pass in later.
If you dont want to have to supply anything for no argument functions, you can double check if the func name is in kwargs:
def function_results_sum(*args, **kwargs):
result = 0
for func in args:
if func.__name__ i kwargs:
result += func(*kwargs[func.__name__])
else:
result += func()
return result

Post of R Nar is exactly what I tried to achieve. I added additional if statement to verify if kwarg is an integer or a tuple. Thanks that it is not neccessary to put all **kwargs in a tuple. Thank you guys for help!
def function_results_sum(*args, **kwargs):
result = 0
for func in args:
if func.__name__ in kwargs:
if type(kwargs[func.__name__]) == int:
result += func(kwargs[func.__name__])
elif type(kwargs[func.__name__]) == tuple:
result += func(*kwargs[func.__name__])
else:
result += func()
return result
result = function_results_sum(no_arg, ident, mult, ident=2, mult=(3, 4))
print(result)

by goolging 'python determine number of args for passed function' I found How can I find the number of arguments of a Python function?
I'm pretty sure you don't want the **kwars key, value syntax so I use a func_list regular arg and *args
from inspect import signature
def function_results_sum(func_list, *args):
arg_gen = (e for e in args)
return sum([func(*(next(arg_gen)
for _ in range(len(signature(func).parameters))))
for func in func_list])
function_results_sum([no_arg, ident, mult], 7,8,9)
84
the input can be made flatter by parsing *args for Functions and (presumed) arguments (anything not Type Function)
from inspect import signature
import types
def function_results_sum(*args):
func_gen = (e for e in args if isinstance(e, types.FunctionType))
arg_gen = (e for e in args if not isinstance(e, types.FunctionType))
return sum(func(*(next(arg_gen)
for _ in range(len(signature(func).parameters))))
for func in func_gen)
function_results_sum(no_arg, ident, mult, 10,6,90)
555
order of functions and order of args are important, but separately, can be interleaved:
function_results_sum(no_arg, 10, ident, 6, 90, mult)
Out[399]: 555

Related

How to wrap a function with accurate __code__.co_argcount?

Here is my problem, i'm working with an API,
precisely with a high-order function that only accepts functions with N arguments. (I cannot monkey-patch this API).
#this is an example of a high order function i may encounter
#there are many more of such functions in the API that require N ammount of arguments
#this example fct required 3 arg, but a valid solution should adapt to any required args count
def high_order_function(f):
"""high order function expecting a function with 3 arguments!"""
print(f"\nprocessing function {f.__name__}")
if f.__code__.co_argcount!=3:
raise Exception(f"Error Expecting a function with 3 arguments, the passed function got {f.__code__.co_argcount}")
print("Function is Ok")
#...
return None
And my problem is that I simply cannot use any wrapper because of this check.
what am I supposed to do ?
def my_wrapper(func):
import functools
#functools.wraps(func)
def inner(*args, **kwargs):
print("wrapped1!")
r = func(*args,**kwargs)
print("wrapped2!")
return r
return inner
def original(a, b, c):
return None
wrapped = my_wrapper(original)
high_order_function(original)
#ok!
high_order_function(wrapped)
#will cause error
#because wrapped.__code__.co_argcount == 0 and is readonly!
After a lot of tinkering, I found a pretty procedural way that might work for you.
The trick was to use __code__.replace(). There are some caveats, probably more than I know.
def high_order_function(f):
"""high order function expecting a function with 3 arguments!"""
print(f"\nprocessing function {f.__name__}")
if f.__code__.co_argcount!=3:
raise Exception(f"Error Expecting a function with 3 arguments, the passed function got {f.__code__.co_argcount}")
print("Function is Ok")
#...
return None
def my_wrapper(func):
import functools
#functools.wraps(func)
def inner(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z):
kwargs = locals().copy()
del kwargs["func"]
print("wrapped1!")
r = func(**kwargs) # func(*kwargs.values()) would work too
print("wrapped2!")
return r
func_args = func.__code__.co_varnames
inner.__code__ = inner.__code__.replace(co_varnames=func_args, co_argcount=len(func_args))
return inner
def original(a, b, c):
return None
wrapped = my_wrapper(original)
high_order_function(original)
high_order_function(wrapped)
Result
processing function original
Function is Ok
processing function original
Function is Ok
functools.wraps changes the name of inner to original
Caveats
__code__.replace() raised ValueError: code: varnames is too small when inner's parameters were *args or **kwargs
If inner instead had no parameters then locals() inside it would not get the supplied values, therefore you got the whole alphabet instead
Inside inner you can access the parameters by the letter like normal if you're sure it´s supplied otherwise you'll get IndexError: tuple index out of range
I recommend to use e.g. kwargs.get("d") instead
__code__.replace may only be for 3.8+, it has sys.version_info >= (3, 8) in the source code
Why not define 2 version of the inner function in your wrapper based on whether you need to pass the wrapped function to higher_order_function or not.
Something like this:
def high_order_function(f):
"""high order function expecting a function with 3 arguments!"""
print(f"\nprocessing function {f.__name__}")
if f.__code__.co_argcount != 3:
raise Exception(
f"Error Expecting a function with 3 arguments, the passed function got {f.__code__.co_argcount}")
print("Function is Ok")
# ...
return None
def my_wrapper(func, higher_order_compatible=True): # switch to control whether should be compatible with the higher_order_function or not
import functools
if higher_order_compatible:
#functools.wraps(func)
def inner(a, b, c, *args, **kwargs): # extra args to satisfy the condition
print("wrapped1!")
r = func(a, b, c, *args, **kwargs)
print("wrapped2!")
return r
else:
#functools.wraps(func)
def inner(*args, **kwargs): # normal wrapper
print("wrapped1!")
r = func(*args, **kwargs)
print("wrapped2!")
return r
return inner
def original(a, b, c):
return None
wrapped_compat = my_wrapper(original, higher_order_compatible=True)
wrapped_nocompat = my_wrapper(original, higher_order_compatible=False)
print("Original")
high_order_function(original)
print("Compatible")
high_order_function(wrapped_compat)
print("Not compatible")
try:
high_order_function(wrapped_nocompat)
except:
print("no not working")
Results in:
Original
processing function original
Function is Ok
Compatible
processing function original
Function is Ok
Not compatible
processing function original
no not working
co_argcount: number of arguments (not including keyword only arguments, * or ** args)
Hence the goal is to bypass such definition. Make a fake signature with 3 fake parameters, these are taken into consideration by the code attribute co_argcount. Then the parameters of the original function must by passed as keys.
def wrapper(f):
def extended_signature(fake1=None, fake2=None, fake3=None, **kwargs):
return f(**kwargs)
return extended_signature
def a(q, w): print(q, w)
a_wrapped = wrapper(a)
high_order_function(a_wrapped)(q=1, w=2)
#processing function true_signature
#Function is Ok
This is an attempt to solve the problem
i feel like it's almost a potential solution
However it is not working as expected, strange. f()for a function object class is not as f.__call__() hmm
def my_wrapper(func):
import copy
def inner(*args, **kwargs):
print("wrapped front")
r = func(*args, **kwargs)
print("wrapped end")
return r
newfunc = copy.deepcopy(func)
newfunc.__name__ = func.__name__ + "_wrapped"
newfunc.__call__ = inner
return newfunc
def original(a, b, c=6):
print("original",a,b,c)
return None
###testing if the original function work
high_order_function(original)
#will pass requirement
###testing if the wrap works?
high_order_function(my_wrapper(original))
#will pass requirement, however the wrap did not work
well, here's a solution, couldn't find a procedural way to generate the functions... it scales up to 5 forced arguments
def my_wrapper(func):
"""see https://stackoverflow.com/questions/73601340/how-to-wrap-a-function-with-accurate-code-argcount?noredirect=1#comment129973896_73601340
yes this is a shit show, did not found a procedural way to generate functions. tried exec() code generation & was also a mess"""
#find back the expected arguments so func.__code__.co_argcount will be accurate
if (func.__defaults__ is not None):
force_arg = func.__code__.co_argcount - len(func.__defaults__)
else: force_arg = func.__code__.co_argcount
import functools
if (force_arg==0):
#functools.wraps(func)
def inner(**kwargs):
print("wrapped1!")
r = func(**kwargs)
print("wrapped2!")
return r
elif (force_arg==1):
#functools.wraps(func)
def inner(a,**kwargs):
print("wrapped1!")
r = func(a,**kwargs)
print("wrapped2!")
return r
elif (force_arg==2):
#functools.wraps(func)
def inner(a,b,**kwargs):
print("wrapped1!")
r = func(a,b,**kwargs)
print("wrapped2!")
return r
elif (force_arg==3):
#functools.wraps(func)
def inner(a,b,c,**kwargs):
print("wrapped1!")
r = func(a,b,c,**kwargs)
print("wrapped2!")
return r
elif (force_arg==4):
#functools.wraps(func)
def inner(a,b,c,d,**kwargs):
print("wrapped1!")
r = func(a,b,c,d,**kwargs)
print("wrapped2!")
return r
elif (force_arg==5):
#functools.wraps(func)
def inner(a,b,c,d,e,**kwargs):
print("wrapped1!")
r = func(a,b,c,d,e,**kwargs)
print("wrapped2!")
return r
else: raise Exception("my_wrapper() do not support more than 5 forced argument")
return inner

How can I write a simple callback function?

I have this example code, trying to demonstrate using a callback function:
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(callback=None):
print('Add any two digits.')
if callback != None:
callback
main(callback(1, 2))
I get this result:
Sum = 3
Add any two digits.
It seems that the callback function executes before the logic in main. Why? How can I make it so that the callback is not called until it is used within main?
See also: Python Argument Binders
In this code
if callback != None:
callback
callback on its own doesn't do anything; it accepts parameters - def callback(a, b):
The fact that you did callback(1, 2) first will call that function, thereby printing Sum = 3, and then main() gets called with the result of the callback function, which is printing the second line
Since callback returns no explicit value, it is returned as None.
Thus, your code is equivalent to
callback(1, 2)
main()
Solution
You could try not calling the function at first and just passing its handle.
def callback(n):
print("Sum = {}".format(n))
def main(a, b, _callback = None):
print("adding {} + {}".format(a, b))
if _callback:
_callback(a+b)
main(1, 2, callback)
Here's what you wanted to do :
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(a,b,f=None):
print('Add any two digits.')
if f is not None:
f(a,b)
main(1, 2, callback)
The problem is that you're evaluating the callback before you pass it as a callable. One flexible way to solve the problem would be this:
def callback1(a, b):
print('Sum = {0}'.format(a+b))
def callback2(a):
print('Square = {0}'.format(a**2))
def callback3():
print('Hello, world!')
def main(callback=None, cargs=()):
print('Calling callback.')
if callback is not None:
callback(*cargs)
main(callback1, cargs=(1, 2))
main(callback2, cargs=(2,))
main(callback3)
Optionally you may want to include a way to support keyword arguments.
As mentioned in the comments, your callback is called whenever it's suffixed with open and close parens; thus it's called when you pass it.
You might want to use a lambda and pass in the values.
#!/usr/bin/env python3
def main(callback=None, x=None, y=None):
print('Add any two digits.')
if callback != None and x != None and y != None:
print("Result of callback is {0}".format(callback(x,y)))
else:
print("Missing values...")
if __name__ == "__main__":
main(lambda x, y: x+y, 1, 2)
Your code is executed as follows:
main(callback(1, 2))
callback function is called with (1, 2) and it returns None (Without return statement, your function prints Sum = 3 and returns None)
main function is called with None as argument (So callback != None will always be False)
This is an old post, but perhaps the following may be additional clarification on writing and using a callback function, especially if you wonder where it gets its arguments from and whether you can access its return values (if there is no way to get it from the function that takes the callback function).
The following code defines a class CallBack that has two callback methods (functions) my_callback_sum and my_callback_multiply. The callback methods are fed into the method foo.
# understanding callback
class CallBack:
#classmethod
def my_callback_sum(cls, c_value1, c_value2):
value = c_value1 + c_value2
print(f'in my_callback_sum --> {c_value1} + {c_value2} = {value}')
cls.operator = '+'
return cls.operator, value
#classmethod
def my_callback_multiply(cls, c_value1, c_value2):
value = c_value1 * c_value2
print(f'in my_callback_multiply --> {c_value1} * {c_value2} = {value}')
cls.operator = '*'
return cls.operator, value
#staticmethod
def foo(foo_value, callback):
_, value = callback(10, foo_value)
# note foo only returns the value not the operator from callback!
return value
if __name__ == '__main__':
cb = CallBack()
value = cb.foo(20, cb.my_callback_sum)
print(f'in main --> {value} and the operator is {cb.operator}')
value = cb.foo(20, cb.my_callback_multiply)
print(f'in main --> {value} and the operator is {cb.operator}')
result:
in my_callback_sum --> 10 + 20 = 30
in main --> 30 and the operator is +
in my_callback_multiply --> 10 * 20 = 200
in main --> 200 and the operator is *
As you can see one value for the callback function c_value2 it gets from argument foo_value in foo and given in main the value 20, while c_value1 it gets internally from foo in this case the value 10 (and may be not clearly visible if foo is some method of a third party imported module, like pyaudio).
The return value of the callback function functions can be retrieved by adding it to the namespace of the class CallBack, in this case cls.operator
You can use anonymous functions
def callback(a, b):
print('Sum = {0}'.format(a+b))
def main(callback=None):
print('Add any two digits.')
if callback is not None:
callback()
tmp_func = lambda: main(lambda: callback(2,3))
tmp_func()
#OR
tmp_func = lambda x,y: main(lambda: callback(x,y))
tmp_func(2,4)

Convert all Input parameters of function with generic function

I am writing a function which gets users from database and returns a list of user objects.
Function signature is as given below:
def select_users(self,userid,firstname,lastname,emailid,tenants,groups):
result = self.authservice.select_users(userid,firstname,lastname,emailid,tenants,groups)
In this function, I call select_users method of authservice object which will return a list of custom user objects. But if any of input parameters has '' value then it must be converted to None because self.authservice.select_users cannot handle empty strings. I can check each element value and convert it to None if it is empty, but I want it to be generic and reusable. If I could write a different function which can give me updated list of input parameters it would be very helpful. Please let me know how do I do that?
I would write a generic decorator, like this
def convert_empty_to_none(func):
def inner_function(*args, **kwargs):
args = (None if item == "" else item for item in args)
kwargs = {k:(None if v == "" else v) for k, v in kwargs.items()}
return func(*args, **kwargs)
return inner_function
#convert_empty_to_none
def test_function(a, b, c):
print a, b, c
test_function("", "", "")
Output
None None None
Evil way:
def select_users(self, *args):
new_args = [(None if arg == '' else arg) for arg in args]
result = self.authservice.select_users(*new_args)
Decorator solution is also pretty evil: changing function arguments to spare writing a couple of function calls doesn't seem such a great idea.
In the real life I would go with the explicit:
def never_blank(s):
return None if s == '' else s
def select_users(self, userid,firstname,lastname,emailid,tenants,groups):
result = self.authservice.select_users(userid,never_blank(firstname),never_blank(lastname),emailid,
never_blank(tenants),groups)
Tedious? Sure. Clean? Yep. Will bite you in the ass in the future? Nope.
Create a function and use it like a function type decorator
def sanitize(func):
def handler(*args, **kwargs):
args = (e if e != '' else None for e in args)
kwargs = {k:(v if v != '' else None) for k, v in kwargs.items()}
return func(*args, **kwargs)
return handler
#sanitize
def select_users(self,userid,firstname,lastname,emailid,tenants,groups):
result = self.authservice.select_users(userid,firstname,lastname,emailid,tenants,groups)
Benifits
You do not need to modify the signature
The caller would still have clear idea, what parameters the function expects
Generic and can be used for any function call
Is a decorator, so can easily be used in a non-intrusive fashion
You could use a decorator to create a generic wrapper that will replace every empty string with None.
def none_for_empty_string(func):
def wrapper(*args, **kwargs):
args = tuple(arg if arg != '' else None for arg in args)
kwargs = {k : v if v != '' else None for k, v in kwargs.iteritems()}
return func(*args, **kwargs)
return wrapper
#none_for_empty_string
def select_users(self,userid,firstname,lastname,emailid,tenants,groups):
...

Python/Django: Are there any decorators to tell that input is not none?

I am new to Django and come from Java/Spring background.
I am wondering if there are decorators something like following that can be done in Django or Python?
Want
def addToList(#not_none a, #not_none b):
# so that I do not check for nullity explicitly
do_things_with_a(a)
do_things_with_b(b)
Since this is something which is pretty easy to get in Java, just looking if Python/Django has it
One doesn't typically constraint data-types in Python. Also, decorators can only be applied to classes and to methods/functions.
Although, you shouldn't really be doing this, this is how you would.
(You could amend this to accept argument names to enforce constraints on with a little work).
def not_none(f):
def func(*args, **kwargs):
if any(arg is None for arg in args):
raise ValueError('function {}: does not take arguments of None'.format(f.__name__))
return f(*args, **kwargs)
return func
#not_none
def test(a, b):
print a, b
You can write a decorator rejectNone as follows:
def rejectNone(f):
def myF(*args, **kwargs):
if None in args or None in kwargs.values():
raise Exception('One of the arguments passed to {0} is None.'.format(f.__name__)
return f(*args, **kwargs)
return myF
#rejectNone
def f(a, b, k=3):
print a * b
You will now get an Exception if you try to call f with a None argument. Note that decorators can be applied to functions or class methods but you can't put them in front of function parameters.
I know this is late, but to those who it may be helpful.
I have a simple repo based off of Jon's answer that accepts arguments for nullable fields here.
def not_none(nullable_parameters=None):
def the_actual_test(f, args, filter_array):
has_none = False
bad_parameters = []
if type(filter_array) is str:
filter_array = [filter_array]
if not filter_array:
if any(arg[1] is None for arg in args):
raise ValueError('function {}: Parameters cannot be None. '.format(f.__name__))
elif type(filter_array) is list:
for a in args:
for ff in filter_array:
if a[0] != ff:
if a[1] is None:
has_none = True
bad_parameters.append(a[0])
break
if has_none:
raise ValueError('function {}: Parameters {} cannot be None. '.format(f.__name__, bad_parameters))
def real_decorator(f):
v_names = f.__code__.co_varnames
def wrapper(*args, **kwargs):
n_args = []
for a in range(0, len(args)):
n_args.append((v_names[a], args[a]))
the_actual_test(f, n_args, nullable_parameters)
result = f(*args, **kwargs)
return result
return wrapper
return real_decorator
Usage
from not_none import not_none
#not_none()
def no_none(a,b):
return (a,b)
#not_none(nullable_parameters=["b"])
def allow_b_as_none(a,b):
return (a,b)
#passes
no_none(1,1)
#fails
no_none(None,1)
#passes
allow_b_as_none(1,None)
#fails
allow_b_as_none(None,1)
After my first answer got deleted. Here is an updated version:
I tried to use the very nice answer from Bigbob556677, but for me it didn't work with **kwargs, so I edited it and put it in a Gist, here: https://gist.github.com/devTechi/6e633ded72cc83637f34b1a3f4a96984 (code also below)
I didn't test it with just *args, but with (I posted more or less just the gist-link) **kwargs it works nicely.
def not_none(nullable_parameters=None):
# values given by real_decorator (see below)
def the_actual_test(f, expected_args_with_given, allowed_nullable_args):
has_none = False
bad_parameters = []
for key, value in expected_args_with_given.items():
if (value is None and nullable_parameters is None) or \
(value is None and key not in nullable_parameters):
bad_parameters.append(key)
has_none = True
if has_none:
raise ValueError("[Function '{}' of '{}'] - IMPORTANT: Parameters '{}' cannot be None. ".format(f.__name__, f.__module__, bad_parameters))
# here the code REALLY begins
def not_null_decorator(original_func):
import inspect
has_self = False
# f.__code__.co_varnames --> local variables (not only parameters), see: https://python-reference.readthedocs.io/en/latest/docs/code/varnames.html
# get declared arguments from ogirinal function
argspec = inspect.getargspec(original_func)
if 'self' in argspec.args:
argnames = argspec.args[1:] # no self
has_self = True
else:
argnames = argspec.args
args_dict = dict.fromkeys(argnames)
def get_args(*args, **kwargs):
for arg in args:
if arg in args_dict.keys():
args_dict[arg] = arg
for key, value in kwargs.items():
if key in args_dict.keys():
args_dict[key] = value
return args_dict
def wrapper_with_self(self, *args, **kwargs):
the_actual_test(original_func, get_args(*args, **kwargs), nullable_parameters)
return original_func(self, *args, **kwargs)
def wrapper(*args, **kwargs):
the_actual_test(original_func, get_args(*args, **kwargs), nullable_parameters)
return original_func(*args, **kwargs)
if has_self:
return wrapper_with_self
else:
return wrapper
return not_null_decorator
Usage:
from .nullable_decorator import not_none
#not_none(nullable_parameters=["nullable_arg1", "nullable_arg2"])
def some_function(self, nullable_arg1=None, nullable_arg2=None, non_nullable_arg1=None):
pass
#not_none()
def some_other_function(self, non_nullable_arg1=None, non_nullable_arg2=None):
pass

How can I get the values of the locals of a function after it has been executed?

Suppose I have a function like f(a, b, c=None). The aim is to call the function like f(*args, **kwargs), and then construct a new set of args and kwargs such that:
If the function had default values, I should be able to acquire their values. For example, if I call it like f(1, 2), I should be able to get the tuple (1, 2, None) and/or the dictionary {'c': None}.
If the value of any of the arguments was modified inside the function, get the new value. For example, if I call it like f(1, 100000, 3) and the function does if b > 500: b = 5 modifying the local variable, I should be able to get the the tuple (1, 5, 3).
The aim here is to create a a decorator that finishes the job of a function. The original function acts as a preamble setting up the data for the actual execution, and the decorator finishes the job.
Edit: I'm adding an example of what I'm trying to do. It's a module for making proxies for other classes.
class Spam(object):
"""A fictional class that we'll make a proxy for"""
def eggs(self, start, stop, step):
"""A fictional method"""
return range(start, stop, step)
class ProxyForSpam(clsproxy.Proxy):
proxy_for = Spam
#clsproxy.signature_preamble
def eggs(self, start, stop, step=1):
start = max(0, start)
stop = min(100, stop)
And then, we'll have that:
ProxyForSpam().eggs(-10, 200) -> Spam().eggs(0, 100, 1)
ProxyForSpam().eggs(3, 4) -> Spam().eggs(3, 4, 1)
There are two recipes available here, one which requires an external library and another that uses only the standard library. They don't quite do what you want, in that they actually modify the function being executed to obtain its locals() rather than obtain the locals() after function execution, which is impossible, since the local stack no longer exists after the function finishes execution.
Another option is to see what debuggers, such as WinPDB or even the pdb module do. I suspect they use the inspect module (possibly along with others), to get the frame inside which a function is executing and retrieve locals() that way.
EDIT: After reading some code in the standard library, the file you want to look at is probably bdb.py, which should be wherever the rest of your Python standard library is. Specifically, look at set_trace() and related functions. This will give you an idea of how the Python debugger breaks into the class. You might even be able to use it directly. To get the frame to pass to set_trace() look at the inspect module.
I've stumbled upon this very need today and wanted to share my solution.
import sys
def call_function_get_frame(func, *args, **kwargs):
"""
Calls the function *func* with the specified arguments and keyword
arguments and snatches its local frame before it actually executes.
"""
frame = None
trace = sys.gettrace()
def snatch_locals(_frame, name, arg):
nonlocal frame
if frame is None and name == 'call':
frame = _frame
sys.settrace(trace)
return trace
sys.settrace(snatch_locals)
try:
result = func(*args, **kwargs)
finally:
sys.settrace(trace)
return frame, result
The idea is to use sys.trace() to catch the frame of the next 'call'. Tested on CPython 3.6.
Example usage
import types
def namespace_decorator(func):
frame, result = call_function_get_frame(func)
try:
module = types.ModuleType(func.__name__)
module.__dict__.update(frame.f_locals)
return module
finally:
del frame
#namespace_decorator
def mynamespace():
eggs = 'spam'
class Bar:
def hello(self):
print("Hello, World!")
assert mynamespace.eggs == 'spam'
mynamespace.Bar().hello()
I don't see how you could do this non-intrusively -- after the function is done executing, it doesn't exist any more -- there's no way you can reach inside something that doesn't exist.
If you can control the functions that are being used, you can do an intrusive approach like
def fn(x, y, z, vars):
'''
vars is an empty dict that we use to pass things back to the caller
'''
x += 1
y -= 1
z *= 2
vars.update(locals())
>>> updated = {}
>>> fn(1, 2, 3, updated)
>>> print updated
{'y': 1, 'x': 2, 'z': 6, 'vars': {...}}
>>>
...or you can just require that those functions return locals() -- as #Thomas K asks above, what are you really trying to do here?
Witchcraft below read on your OWN danger(!)
I have no clue what you want to do with this, it's possible but it's an awful hack...
Anyways, I HAVE WARNED YOU(!), be lucky if such things don't work in your favorite language...
from inspect import getargspec, ismethod
import inspect
def main():
#get_modified_values
def foo(a, f, b):
print a, f, b
a = 10
if a == 2:
return a
f = 'Hello World'
b = 1223
e = 1
c = 2
foo(e, 1000, b = c)
# intercept a function and retrieve the modifed values
def get_modified_values(target):
def wrapper(*args, **kwargs):
# get the applied args
kargs = getcallargs(target, *args, **kwargs)
# get the source code
src = inspect.getsource(target)
lines = src.split('\n')
# oh noes string patching of the function
unindent = len(lines[0]) - len(lines[0].lstrip())
indent = lines[0][:len(lines[0]) - len(lines[0].lstrip())]
lines[0] = ''
lines[1] = indent + 'def _temp(_args, ' + lines[1].split('(')[1]
setter = []
for k in kargs.keys():
setter.append('_args["%s"] = %s' % (k, k))
i = 0
while i < len(lines):
indent = lines[i][:len(lines[i]) - len(lines[i].lstrip())]
if lines[i].find('return ') != -1 or lines[i].find('return\n') != -1:
for e in setter:
lines.insert(i, indent + e)
i += len(setter)
elif i == len(lines) - 2:
for e in setter:
lines.insert(i + 1, indent + e)
break
i += 1
for i in range(0, len(lines)):
lines[i] = lines[i][unindent:]
data = '\n'.join(lines) + "\n"
# setup variables
frame = inspect.currentframe()
loc = inspect.getouterframes(frame)[1][0].f_locals
glob = inspect.getouterframes(frame)[1][0].f_globals
loc['_temp'] = None
# compile patched function and call it
func = compile(data, '<witchstuff>', 'exec')
eval(func, glob, loc)
loc['_temp'](kargs, *args, **kwargs)
# there you go....
print kargs
# >> {'a': 10, 'b': 1223, 'f': 'Hello World'}
return wrapper
# from python 2.7 inspect module
def getcallargs(func, *positional, **named):
"""Get the mapping of arguments to values.
A dict is returned, with keys the function argument names (including the
names of the * and ** arguments, if any), and values the respective bound
values from 'positional' and 'named'."""
args, varargs, varkw, defaults = getargspec(func)
f_name = func.__name__
arg2value = {}
# The following closures are basically because of tuple parameter unpacking.
assigned_tuple_params = []
def assign(arg, value):
if isinstance(arg, str):
arg2value[arg] = value
else:
assigned_tuple_params.append(arg)
value = iter(value)
for i, subarg in enumerate(arg):
try:
subvalue = next(value)
except StopIteration:
raise ValueError('need more than %d %s to unpack' %
(i, 'values' if i > 1 else 'value'))
assign(subarg,subvalue)
try:
next(value)
except StopIteration:
pass
else:
raise ValueError('too many values to unpack')
def is_assigned(arg):
if isinstance(arg,str):
return arg in arg2value
return arg in assigned_tuple_params
if ismethod(func) and func.im_self is not None:
# implicit 'self' (or 'cls' for classmethods) argument
positional = (func.im_self,) + positional
num_pos = len(positional)
num_total = num_pos + len(named)
num_args = len(args)
num_defaults = len(defaults) if defaults else 0
for arg, value in zip(args, positional):
assign(arg, value)
if varargs:
if num_pos > num_args:
assign(varargs, positional[-(num_pos-num_args):])
else:
assign(varargs, ())
elif 0 < num_args < num_pos:
raise TypeError('%s() takes %s %d %s (%d given)' % (
f_name, 'at most' if defaults else 'exactly', num_args,
'arguments' if num_args > 1 else 'argument', num_total))
elif num_args == 0 and num_total:
raise TypeError('%s() takes no arguments (%d given)' %
(f_name, num_total))
for arg in args:
if isinstance(arg, str) and arg in named:
if is_assigned(arg):
raise TypeError("%s() got multiple values for keyword "
"argument '%s'" % (f_name, arg))
else:
assign(arg, named.pop(arg))
if defaults: # fill in any missing values with the defaults
for arg, value in zip(args[-num_defaults:], defaults):
if not is_assigned(arg):
assign(arg, value)
if varkw:
assign(varkw, named)
elif named:
unexpected = next(iter(named))
if isinstance(unexpected, unicode):
unexpected = unexpected.encode(sys.getdefaultencoding(), 'replace')
raise TypeError("%s() got an unexpected keyword argument '%s'" %
(f_name, unexpected))
unassigned = num_args - len([arg for arg in args if is_assigned(arg)])
if unassigned:
num_required = num_args - num_defaults
raise TypeError('%s() takes %s %d %s (%d given)' % (
f_name, 'at least' if defaults else 'exactly', num_required,
'arguments' if num_required > 1 else 'argument', num_total))
return arg2value
main()
Output:
1 1000 2
{'a': 10, 'b': 1223, 'f': 'Hello World'}
There you go... I'm not responsible for any small children that get eaten by demons or something the like (or if it breaks on complicated functions).
PS: The inspect module is the pure EVIL.
Since you are trying to manipulate variables in one function, and do some job based on those variables on another function, the cleanest way to do it is having these variables to be an object's attributes.
It could be a dictionary - that could be defined inside the decorator - therefore access to it inside the decorated function would be as a "nonlocal" variable. That cleans up the default parameter tuple of this dictionary, that #bgporter proposed.:
def eggs(self, a, b, c=None):
# nonlocal parms ## uncomment in Python 3
parms["a"] = a
...
To be even more clean, you probably should have all these parameters as attributes of the instance (self) - so that no "magical" variable has to be used inside the decorated function.
As for doing it "magically" without having the parameters set as attributes of certain object explicitly, nor having the decorated function to return the parameters themselves (which is also an option) - that is, to have it to work transparently with any decorated function - I can't think of a way that does not involve manipulating the bytecode of the function itself.
If you can think of a way to make the wrapped function raise an exception at return time, you could trap the exception and check the execution trace.
If it is so important to do it automatically that you consider altering the function bytecode an option, feel free to ask me further.

Categories