We Keep Coding

Python decorators: howto list wrapped functions by decorator

Is that possible to use python decorators to mark a method, and get it for the later use, if I don't know the name of the wrapped function?
Here is the example, and I don't know the name of method_with_custom_name:
#run_this_method
def method_with_custom_name(my_arg):
return "The args is: " + my_arg
def _init_and_run():
# Here, I want to get and call method_with_custom_name
# but I don't know it's name,
# so the next line isn't valid.
return run_this_method()(my_arg_value)
def run_this_method(m):
def w(my_arg):
_do_some_magic(my_arg, m)
return w
def _do_some_magic(callback_arg, callback):
if some_checks():
callback(callback_arg)
So how can I get a list of methods wrapped with #run_this_method

If you need to track all functions and methods decorated with your decorator you need to create global variable to register all such functions and methods. I've modified your code:
funcs_registry = [] #List of all functions decorated with #run_this_method
def run_this_method(m):
global functions_registry
funcs_registry.append(m) #Add function/method to the registry
def w(my_arg):
_do_some_magic(my_arg, m)
return w
def _do_some_magic(callback_arg, callback):
if some_checks():
callback(callback_arg)
#run_this_method
def method_with_custom_name(my_arg):
return "The args is: " + my_arg
def _init_and_run():
global functions_registry
# Here you can iterate over "functions_registry"
# and do something with each function/method in it
for m in functions_registry:
print(m.__name__)
Instead of using global variable functions_registry you can create class to be used as decorator and register functions in entity field. Something like this:
class FunDecorator:
def __init__(self):
self.registry = []
def __call__(self, m):
"This method is called when some method is decorated"
self.registry.append(m) #Add function/method to the registry
def w(my_arg):
_do_some_magic(my_arg, m)
return w
run_this_method = FunDecorator() #Create class instance to be used as decorator
#run_this_method
def method_with_custom_name(my_arg):
return "The args is: " + my_arg
#do some magic with each decorated method:
for m in run_this_method.registry:
print(m.__name__)

If I understand your question correctly (how to decorate a method with an unknown name?) then it is totally possible.
#decorator
def foo(bar):
pass
is syntastic sugar for
def foo(bar):
pass
foo = decorator(foo)
So in your case you should just do:
method_with_custom_name = run_this_method(method_with_custom_name)
The example you provided is confusing, though. Why don't you know the name of method_with_custom_name? It is right there. It is called method_with_custom_name. To use the decorated version later, you just call method_with_custom_name.

How to retrieve arguments from a "fluid" decorated function

Suppose I have this decorator:
def decorator(f):
def f_wrap(*args):
for item in args:
print(args)
return f(*args)
return f_wrap
When used as "permanent" decorators with the # syntax, args retrieves the arguments of the wrapped function. For example, when used with the class below, I receive the instance of MyObject.
Class MyObject(object):
def __init__(self):
pass
#decorator
def function(self):
return
How can I achieve the same result using a "fluid" decorator. Or a decorator that is not permanently bound to the function it is decorating? For example:
def decorator(f):
def f_wrap(*args):
if (not args):
print("Nothing in args")
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = decorator(myobj.function)
myobj.function()
In this case, the args tuple always returns empty (I always get "Nothing in args"), even though I anticipated that it would return the instance variable myobj.
EDIT:
In case it was not clear from #AChampion's post the solution is to simply call the fluid-decoratored method as an "unbound" method. E.g.,
from types import MethodType
def decorator(f):
def f_wrap(*args):
# I replaced this with an iteration through
# args. It's a bit more demonstrative.
for item in args:
print(item)
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = MethodType(decorator(MyClass.function), myobj)
myobj.function()

The reason for the difference is that you are wrapping different things, a unbound method vs a bound method:
class MyObject(object):
#decorator
def function(self):
pass
Is equivalent to:
import types
class MyClass(object):
def function(self):
pass
m = MyClass(object)
m.function = types.MethodType(decorator(MyClass.function), m)
Not:
m.function = decorator(m.function)
The first being an unbound method, the second being a bound method.

You aren't using all properly. all returns a bool on whether all conditions are met inside what you are checking for in all. In your case, you aren't really doing anything. You will always evaluate to True with how you are using all.
I believe what you are looking for is simply this:
if not args:
Now, ultimately what this checks is if the method you are executing has *args. For the case of the function you have, you aren't passing any arguments, therefore, with the if not args check, you will actually get:
"Nothing in args"
However, if you add an argument to your method as such:
def function(self, x):
return
Then call: myobj.function(1)
You will not get "Nothing in args".
To answer your last question about not getting your instance. If you print out f using this method of calling your decorator:
myobj.function = decorator(myobj.function)
myobj.function()
You will get a bound method:
<bound method MyClass.function of <__main__.MyClass object at 0x102002390>>
Now, set up your decorator as such:
#decorator
def function(self):
return
You will see you get a function attached to your class object:
<function MyClass.function at 0x102001620>
Hence showing that they aren't doing the exact same thing you would expect. Hope this helps clarify a bit.

How to print graph call as tree?

For instanse, I have the following code snippet:
def func1(num):
print(num)
def func2(num):
func1(num)
def func3(num):
func2(num)
func1(num)
def begin():
pass
def print_graph():
pass
def main():
begin()
func3(3)
print_graph()
Is there any simple way to print something like that:
func3(1)
func2(1)
func1(1)
func1(1)
I believe, that I have to use globals(), but I don't know, what I do next. It is some sort of study task, therefore I cant use any libraries.

I can go one better than #jme. Here's a version of his decorator that indents and dedents according to your location in the call stack:
import functools
# a factory for decorators
def create_tracer(tab_width):
indentation_level = 0
def decorator(f): # a decorator is a function which takes a function and returns a function
#functools.wraps(f)
def wrapper(*args): # we wish to extend the function that was passed to the decorator, so we define a wrapper function to return
nonlocal indentation_level # python 3 only, sorry
msg = " " * indentation_level + "{}({})".format(f.__name__, ", ".join([str(a) for a in args]))
print(msg)
indentation_level += tab_width # mutate the closure so the next function that is called gets a deeper indentation level
result = f(*args)
indentation_level -= tab_width
return result
return wrapper
return decorator
tracer = create_tracer(4) # create the decorator itself
#tracer
def f1():
x = f2(5)
return f3(x)
#tracer
def f2(x):
return f3(2)*x
#tracer
def f3(x):
return 4*x
f1()
Output:
f1()
f2(5)
f3(2)
f3(40)
The nonlocal statement allows us to mutate the indentation_level in the outer scope. Upon entering a function, we increase the indentation level so that the next print gets indented further. Then upon exiting we decrease it again.
This is called decorator syntax. It's purely 'syntactic sugar'; the transformation into equivalent code without # is very simple.
#d
def f():
pass
is just the same as:
def f():
pass
f = d(f)
As you can see, # simply uses the decorator to process the decorated function in some way, and replaces the original function with the result, just like in #jme's answer. It's like Invasion of the Body Snatchers; we are replacing f with something that looks similar to f but behaves differently.
If you're stuck on Python 2, you can simulate the nonlocal statement by using a class with an instance variable. This might make a bit more sense to you, if you've never used decorators before.
# a class which acts like a decorator
class Tracer(object):
def __init__(self, tab_width):
self.tab_width = tab_width
self.indentation_level = 0
# make the class act like a function (which takes a function and returns a function)
def __call__(self, f):
#functools.wraps(f)
def wrapper(*args):
msg = " " * self.indentation_level + "{}({})".format(f.__name__, ", ".join([str(a) for a in args]))
print msg
self.indentation_level += self.tab_width
result = f(*args)
self.indentation_level -= self.tab_width
return result
return wrapper
tracer = Tracer(4)
#tracer
def f1():
# etc, as above
You mentioned that you're not allowed to change the existing functions. You can retro-fit the decorator by messing around with globals() (though this generally isn't a good idea unless you really need to do it):
for name, val in globals().items(): # use iteritems() in Python 2
if name.contains('f'): # look for the functions we wish to trace
wrapped_func = tracer(val)
globals()[name] = wrapped_func # overwrite the function with our wrapped version
If you don't have access to the source of the module in question, you can achieve something very similar by inspecting the imported module and mutating the items it exports.
The sky's the limit with this approach. You could build this into an industrial-strength code analysis tool by storing the calls in some sort of graph data structure, instead of simply indenting and printing. You could then query your data to answer questions like "which functions in this module are called the most?" or "which functions are the slowest?". In fact, that's a great idea for a library...

If you don't want to use modify code, you can always use sys.settrace. Here is a simple sample:
import sys
import inspect
class Tracer(object):
def __init__(self):
self._indentation_level = 0
#property
def indentation_level(self):
return self._indentation_level
#indentation_level.setter
def indentation_level(self, value):
self._indentation_level = max(0, value)
def __enter__(self):
sys.settrace(self)
def __exit__(self, exc_type, exc_value, traceback):
sys.settrace(None)
def __call__(self, frame, event, args):
frameinfo = inspect.getframeinfo(frame)
filename = frameinfo.filename
# Use `in` instead of comparing because you need to cover for `.pyc` files as well.
if filename in __file__:
return None
if event == 'return':
self.indentation_level -= 1
elif event == 'call':
print "{}{}{}".format(" " * self.indentation_level,
frameinfo.function,
inspect.formatargvalues(*inspect.getargvalues(frame)))
self.indentation_level += 1
else:
return None
return self
Usage:
from tracer import Tracer
def func1(num):
pass
def func2(num):
func1(num)
def func3(num):
func2(num)
func1(num)
def main():
with Tracer():
func3(1)
And results:
func3(num=1)
func2(num=1)
func1(num=1)
func1(num=1)

How about using decorators to print a function's name when it is called? Something like this:
from functools import wraps
def print_on_entry(fn):
#wraps(fn)
def wrapper(*args):
print "{}({})".format(fn.func_name, ", ".join(str(a) for a in args))
fn(*args)
return wrapper
Then you can wrap each of your functions up:
func1 = print_on_entry(func1)
func2 = print_on_entry(func2)
func3 = print_on_entry(func3)
So that:
>>> func3(1)
func3(1)
func2(1)
func1(1)
1
func1(1)
1
Of course there are a lot of assumptions in the above code -- the arguments can be converted to strings, for example -- but you get the picture.

Python - If a function is a first class object, can a function have a method?

I have a class which maintains a list of functions. These functions are just objects sitting in a queue and every so often the class pops one off and executes it. However, there are times when I would like to print out this list, and I'm imagining code as follows:
for function in self.control_queue:
print function.summarize()
if function.ready():
function()
In other words, I would like to call methods called summarize() and ready(), that I want to define somewhere, on these function objects. Also, I would like to be able to toss anonymous functions on this queue - i.e., generate everything dynamically.

you can make it a class and define __call__
class MyClass():
def summarize(self):
#summarize stuff
pass
def ready(self):
#ready stuff
pass
def _call__(self):
#put the code here, for when you call myClass()
pass
How you run it:
function = MyClass()
print function.summarize()
if function.ready():
function()

You have a couple possible approaches.
You could add the definitions to functions.
def foo():
pass
# later..
foo.summarize = lambda: "To pair with bar"
foo.ready = lambda: True
You could create class objects to wrap the function operation.
class Func():
def summarize(self):
return "Function!"
def ready(self):
return self.ready
def __call__(self):
# Act as a function
Or you can have a function which checks the function label for these capabilities.
def summarize_func(func):
return func.__name__ # Or branch here on specific names/attributes
def ready_func(func):
return True # Or branch on names/attributes
Finally to accommodate anonymous functions you can check for prescience of these attributes and return optimistically if the attributes are absent. Then you can combine above approaches with something that will work on any function.
def summarize_func(func):
if hasattr(func, summarize):
return func.summarize()
else:
# Note this will just be '<lambda>' for anonymous funcs
return func.__name__
def ready_func(func):
if hasattr(func, ready):
return func.ready()
else:
return True

One option is to implement function as a class instance:
class Function(object):
def summarize(self): pass # some relevant code here
def __call__(self): pass # and there
and use it later with
function = Function()
With __call__ magic method implemented, this function becomes a callable object.
For sure, you can assign attributes to functions, but it is rather obscure and conterintuitive:
>>> def summ(a): return sum(a)
...
>>> def function(a): return a
...
>>> function.sum=summ
>>> function.sum([1,2,3])
6

Find Out If a Function has been Called

I am programming in Python, and I am wondering if i can test if a function has been called in my code
def example():
pass
example()
#Pseudocode:
if example.has_been_called:
print("foo bar")
How would I do this?

If it's OK for the function to know its own name, you can use a function attribute:
def example():
example.has_been_called = True
pass
example.has_been_called = False
example()
#Actual Code!:
if example.has_been_called:
print("foo bar")
You could also use a decorator to set the attribute:
import functools
def trackcalls(func):
#functools.wraps(func)
def wrapper(*args, **kwargs):
wrapper.has_been_called = True
return func(*args, **kwargs)
wrapper.has_been_called = False
return wrapper
#trackcalls
def example():
pass
example()
#Actual Code!:
if example.has_been_called:
print("foo bar")

A minimal example using unittest.mock.Mock from the standard library:
from unittest.mock import Mock
def example():
pass
example_mock = Mock(side_effect=example)
example_mock()
#Pseudocode:
if example_mock.called:
print("foo bar")
Console output after running the script:
foo bar
This approach is nice because it doesn't require you to modify the example function itself, which is useful if you want to perform this check in some unit-testing code, without modifying the source code itself (EG to store a has_been_called attribute, or wrap the function in a decorator).
Explanation
As described in the documentation for the unittest.mock.Mock class, the side_effect argument to the Mock() constructor specifies "a function to be called whenever the Mock is called".
The Mock.called attribute specifies "a boolean representing whether or not the mock object has been called".
The Mock class has other attributes you may find useful, EG:
call_count: An integer telling you how many times the mock object has been called
call_args: This is either None (if the mock hasn’t been called), or the arguments that the mock was last called with
call_args_list: This is a list of all the calls made to the mock object in sequence (so the length of the list is the number of times it has been called). Before any calls have been made it is an empty list
The Mock class also has convenient methods for making assert statements based on how many times a Mock object was called, and what arguments it was called with, EG:
assert_called_once_with(*args, **kwargs): Assert that the mock was called exactly once and that that call was with the specified arguments

We can use mock.Mock
from unittest import mock
def check_called(func):
return mock.Mock(side_effect=func)
#check_called
def summator(a, b):
print(a + b)
summator(1, 3)
summator.assert_called()
assert summator.called == True
assert summator.call_count > 0
summator.assert_called_with(1, 3)
summator.assert_called_with(1, 5) # error
# AssertionError: Expected call: mock(1, 5)
# Actual call: mock(1, 3)

Memoization functions have been around since the 1960s. In python you can use them as decorators on your example() function.
The standard memoization function looks something like this:
def memoize(func):
memo = {}
def wrapper(*args):
if not args in memo:
memo[args] = func(*args)
return memo[args]
return wrapper
and you decorate your function like this:
#memoize
def example():
pass
In python3.2, you can use the functools.lru_cache instead of the memoziation function.
import functools
#functools.lru_cache(maxsize=None)
def example():
pass

Here's a decorator that will watch all your functiona, using colorama, and return a nice output.
try:
import colorama
except ImportError:
class StdClass: pass
def passer(*args, **kwargs): pass
colorama = StdClass()
colorama.init = passer
colorama.Fore = StdClass()
colorama.Fore.RED = colorama.Fore.GREEN = ''
def check_for_use(show=False):
if show:
try:
check_for_use.functions
except AttributeError:
return
no_error = True
for function in check_for_use.functions.keys():
if check_for_use.functions[function][0] is False:
print(colorama.Fore.RED + 'The function {!r} hasn\'t been called. Defined in "{}" '.format(function, check_for_use.functions[function][1].__code__.co_filename))
no_error = False
if no_error:
print(colorama.Fore.GREEN + 'Great! All your checked function are being called!')
return check_for_use.functions
try:
check_for_use.functions
except AttributeError:
check_for_use.functions = {}
if colorama:
colorama.init(autoreset=True)
def add(function):
check_for_use.functions[function.__name__] = [False, function]
def func(*args, **kwargs):
check_for_use.functions[function.__name__] = [True, function]
function(*args, **kwargs)
return func
return add
#check_for_use()
def hello():
print('Hello world!')
#check_for_use()
def bonjour(nb):
print('Bonjour tout le monde!')
# hello(); bonjour(0)
hello()
check_for_use(True) # outputs the following
Output:
Hello world!
The function 'bonjour' hasn't been called. Defined in "path_to_file.py"

You can also create a variable and increment it in the function. Later you can check if it's 1 or >= 0.