I tried lately to train myself a lot in unit-testing best practices. Most of it makes perfect sense, but there is something that is often overlooked and/or badly explained: how should one unit-test decorated functions ?
Let's assume I have this code:
def stringify(func):
#wraps(func)
def wrapper(*args):
return str(func(*args))
return wrapper
class A(object):
#stringify
def add_numbers(self, a, b):
"""
Returns the sum of `a` and `b` as a string.
"""
return a + b
I can obviously write the following tests:
def test_stringify():
#stringify
def func(x):
return x
assert func(42) == "42"
def test_A_add_numbers():
instance = MagicMock(spec=A)
result = A.add_numbers.__wrapped__(instance, 3, 7)
assert result == 10
This gives me 100% coverage: I know that any function that gets decorated with stringify() gets his result as a string, and I know that the undecorated A.add_numbers() function returns the sum of its arguments. So by transitivity, the decorated version of A.add_numbers() must return the sum of its argument, as a string. All seems good !
However I'm not entirely satisfied with this: my tests, as I wrote them could still pass if I were to use another decorator (that does something else, say multiply the result by 2 instead of casting to a str). My function A.add_numbers would not be correct anymore yet the tests would still pass. Not awesome.
I could test the decorated version of A.add_numbers() but then I would overtest things since my decorator is already unit-tested.
It feels like I'm missing something here. What is a good strategy to unit-test decorated functions ?
I ended up splitting my decorators in two. So instead of having:
def stringify(func):
#wraps(func)
def wrapper(*args):
return str(func(*args))
return wrapper
I have:
def to_string(value):
return str(value)
def stringify(func):
#wraps(func)
def wrapper(*args):
return to_string(func(*args))
return wrapper
Which allows me later to simply mock-out to_string when testing the decorated function.
Obviously in this simple example case it might seem overkill, but when used over a decorator that actually does something complex or expensive (like opening a connection to a DB, or whatever), being able to mock it out is a very nice thing.
Test the public interface of your code. If you only expect people to call the decorated functions, then that's what you should test. If the decorator is also public, then test that too (like you did with test_stringify()). Don't test the wrapped versions unless people are directly calling them.
One of the major benefits of unit testing is to allow refactoring with some degree of confidence that the refactored code continues to work the same as it did previously. Suppose you had started with
def add_numbers(a, b):
return str(a + b)
def mult_numbers(a, b):
return str(a * b)
You would have some tests like
def test_add_numbers():
assert add_numbers(3, 5) == "8"
def test_mult_numbers():
assert mult_numbers(3, 5) == "15"
Now, you decide to refactor the common parts of each function (wrapping the output in a string), using your stringify decorator.
def stringify(func):
#wraps(func)
def wrapper(*args):
return str(func(*args))
return wrapper
#stringify
def add_numbers(a, b):
return a + b
#stringify
def mult_numbers(a, b):
return a * b
You'll notice that your original tests continue to work after this refactoring. It doesn't matter how you implemented add_numbers and mult_numbers; what matters is they continue to work as defined: returing a stringified result of the desired operation.
The only remaining test you need to write is one to verify that stringify does what it is intended to do: return the result of the decorated function as a string, which your test_stringify does.
Your issue seems to be that you want to treat the unwrapped function, the decorator, and the wrapped function as units. But if that's the case, then you are missing one unit test: the one that actually runs add_wrapper and tests its output, rather than just add_wrapper.__wrapped__. It doesn't really matter if you consider testing the wrapped function as a unit test or an integration test, but whatever you call it, you need to write it, because as you pointed out, it's not sufficient to test just the unwrapped function and the decorator separately.
Related
I have a python dataclass in which I want to conditionally assign certain decorators depending on some global variable.
The condition is checked at the top of the script, but for my example below, I've simply supplied the result of that checking. If the check is True, I want to give those methods the #functools.cached_property decorator. If it is False, I just want them to receive the standard #property decorator.
The issue I keep running into is that I can't quite figure out how (or if it's even possible) to make this work as a simple decorator. I mostly get errors about method objects when calling or manipulating test.x_times_y, and I'm not sure if it is possible to write the function in such a way that calling test.x_times_y in the example below actually yields the result that I want.
import functools
import dataclasses
_value_checked = False
def myDecorator(func):
def decorator(self):
if not _value_checked:
return property(func)(self)
else:
return functools.cached_property(func)(self)
return decorator
#dataclasses.dataclass
class MyClass():
x: int
y: int
z: int = 0
#myDecorator
def x_times_y(self):
return self.x*self.y
test = MyClass(5,6,7)
I'd also like to avoid getter and setter methods, so I'm hopeful that that is possible. I've looked at many answers on here (such as this one) but haven't been able to find an answers that actually works, as most don't apply to decorating methods. I'm using Python 3.8 for this.
The behavior you want can be implemented with a simple conditional assignment:
my_decorator = functools.cached_property if _value_checked else property
or
if _value_checked:
my_decorator = functools.cached_property
else:
my_decorator = property
If you need to do more complex logic at each use of the decorator, you can use a function that returns the decorator you want:
def my_decorator():
if not _value_checked:
return property
else
return functools.cached_property
No complex argument forwarding required. Just delegate to the decorators you already have.
The way you've written myDecorator it can only be applied to functions that take a single argument:
def myDecorator(func):
def decorator(self):
if not _value_checked:
return property(func)(self)
else:
return functools.cached_property(func)(self)
return decorator
The simplest thing is to just return the function and not call it inside a wrapper:
def myDecorator(func):
if not _value_checked:
return property(func)
else
return functools.cached_property(func)
If you did need to build a wrapper, the generally correct way is to have the wrapper function take arbitrary *args, **kwargs arguments so you can invoke the wrapped function with them:
def myDecorator(func):
def wrapper(*args, **kwargs):
if not _value_checked:
return property(func)(*args, **kwargs)
else:
return functools.cached_property(func)(*args, **kwargs)
return wrapper
Note that the function that myDecorator returns is not itself a decorator, it's a wrapper that replaces the decorated function -- that's why I've renamed it in the above implementation.
Note also that there is a practical difference between these implementations, which is that the second version (with the wrapper) evaluates _value_checked at the time the function is called, whereas the first version evaluates it at the time the function is defined. If that value is a constant it doesn't matter, but if you want to be able to toggle it at runtime and have the behavior change dynamically, you want the second version.
For example:
def get_val(n):
return n
def check_args(func):
# gets the arguments of a function at runtime
get_val(1)
get_val(2)
Note: This is probably bad practice, but I want to understand more about how python works.
Python decorators allow you to do this with minimal effort:
import functools
def check_args(func):
#functools.wraps(func) # Copies documentation and other stuff from wrapped func to wrapper, making it look as much like the wrapped func as possible
def wrapper(*args, **kwargs):
# do stuff to inspect arguments
return func(*args, **kwargs)
return wrapper
#check_args
def get_val(n):
return n
Using #check_args is equivalent to fully defining get_val, then doing:
get_val = check_args(get_val)
which means get_val gets replaced with the wrapper function, that now gets called first, can perform checks, then delegate to the wrapped function (in this case, get_args). Obviously, for just one function, it's kind of pointless (you could just put the checking in get_val), but if you want to check other functions, prefixing their definition with #check_args is a single line of code that doesn't get intermingled with the rest of the code, and keeps boilerplate down.
I have a code like this:
class A():
def __init__(self, a):
self.a = a
def outer_method(self):
def inner_method():
return self.a +1
return inner_method()
I want to write a test for inner_method. For that, I am using a code like this:
def find_nested_func(parent, child_name):
"""
Return the function named <child_name> that is defined inside
a <parent> function
Returns None if nonexistent
"""
consts = parent.__code__.co_consts
item = list(filter(lambda x:isinstance(x, CodeType) and x.co_name==child_name, consts ))[0]
return FunctionType(item, globals())
Calling it with find_nested_func(A().outer_method, 'inner_method') but it fails when calling to 'FunctionType' because the function cannot be created since 'self.a' stops existing in the moment the function stops being an inner function. I know the construction FunctionType can recive as an argument a closure that could fix this problem , but I don't know how to use it. How can I pass it?
The error it gives is the next one:
return FunctionType(item, globals())
TypeError: arg 5 (closure) must be tuple
Why are you trying to test inner_method? In most cases, you should only test parts of your public API. outer_method is part of A's public API, so test just that. inner_method is an implementation detail that can change: what if you decide to rename it? what if you refactor it slightly without modifying the externally visible behavior of outer_method? Users of the class A have no (easy) way of calling inner_method. Unit tests are usually only meant to test things that users of your class can call (I'm assuming these are for unit tests, because integration tests this granular would be strange--and the same principle would still mostly hold).
Practically, you'll have a problem extracting functions defined within another function's scope, for several reasons include variable capture. You have no way of knowing if inner_method only captures self or if outer_method performs some logic and computes some variables that inner_method uses. For example:
class A:
def outer_method():
b = 1
def inner_method():
return self.a + b
return inner_method()
Additionally, you could have control statements around the function definition, so there is no way to decide which definition is used without running outer_method. For example:
import random
class A:
def outer_method():
if random.random() < 0.5:
def inner_method():
return self.a + 1
else:
def inner_method():
return self.a + 2
return inner_method()
You can't extract inner_method here because there are two of them and you don't know which is actually used until you run outer_method.
So, just don't test inner_method.
If inner_method is truly complex enough that you want to test it in isolation (and if you do so, principled testing says you should mock out its uses, eg. its use in outer_method), then just make it a "private-ish" method on A:
class A:
def _inner_method(self):
return self.a + 1
def outer_method(self):
return self._inner_method()
Principled testing says you really shouldn't be testing underscore methods, but sometimes necessity requires it. Doing this things way allows you test _inner_method just as you would any other method. Then, when testing outer_method, you could mock it out by doing a._inner_method = Mock() (where a is the A object under test).
Also, use class A. The parens are unnecessary unless you have parent classes.
I was working through a decorator design pattern tutorial
(credit to Jungwoo Ryoo)
I'm curious as to why I can swap the lines: return decorator
and print(hello_world()) with return decorator() and print(hello_world)
from functools import wraps
def make_blink(function):
"""Defines the decorator"""
#wraps(function)
# Define the inner function
def decorator():
# Grab the return value of the function being decorated
ret = function()
# Add new functionality to the function being decorated
return "<blink>"+ ret + "<b/link>"
return decorator #return decorator()#<THIS LINE HERE SWAPPED
# Apply the decorator here!
#make_blink
def hello_world():
"""Original function! """
return "Hello, World!"
# Check the result of decorating
print(hello_world()) #print(hello_world) #<THIS LINE HERE SWAPPED
Wouldn't the interpreter be doing something different each time? I'm just looking for some insight to have a better understanding of what's going on
Decorators are just functions really, and functions are just objects.
The lines
#make_blink
def hello_world():
# ...
are essentially the same as
def hello_world():
# ...
hello_world = make_blink(hello_world)
except the function object is never assigned to hello_world first (it's on the stack for to be passed to the decorator).
So whatever you return from make_blink() is assigned back to hello_world. That can be a function object, but it can also be something entirely different.
So when you use return decorator, you tell Python to set hello_world to the nested function object. When you use return decorator(), you tell Python to use the result of the decorator() function. Here, that's a string value. It's as if you did this:
def hello_world():
"""Original function! """
return "Hello, World!"
hello_world = "<blink>" + hello_world() + "</blink>"
And that is fine for this specific example, because body of the hello_world() function only ever returns the same string each time.
But what if you changed the original hello_world() function body to return something different each time you called it? What if you had:
import random
#make_blink
def random_greeting():
return 'Hello ' + random.choice('DonAr', 'Martijn Pieters', 'Guido van Rossum') + '!'
Now it makes a big difference what you return from the make_blink() call! For the top-level of a module, decorators are executed only once, when importing. If you used return decorator() you'd run random.choice() just once, and you have fixed the value of random_greeting to a single, static string result.
Generally speaking, decorators are expected to return a callable object again. That can be the original function (where the decorator just updates some kind of registration), a wrapper function (which does extra things before or after calling the original), or even something different entirely. But that's not set in stone anywhere, and the interpreter doesn't care either way.
Decorators are just reusable things to use in your program, a tool. If you have a specific use for a decorator that returns the result of the original function, then you are free to do so.
Is it possible to inspect a function/method to see whether it can be used as a decorator? In that it follows the usual way decorators wrap other functions and return a callable? Specifically, I'm looking to validate 3rd party code.
By applying a suspected decorator, catching exceptions, and then testing whether the result contains a __call__ method, you could produce a guess as to whether a given callable is a decorator or not. But it will be only a guess, not a guarantee.
Beyond that, I do not believe what you want will be possible in general, due to the dynamically typed nature of the Python language and to the special treatment of built-in functions in the CPython interpreter. It is not possible to programmatically tell whether a callable will accept another callable as an argument, or what type its return value will have. Also, in CPython, for functions implemented in C, you cannot even inspect a callable to see how many arguments it accepts.
The word "decorator" can be taken to mean different things. One way to define it is, a decorator is any callable that accepts a single (callable) argument and returns a callable.
Note that I have not even used the word "function" in this definition; it would actually be incorrect to do so. Indeed, some commonly used decorators have strange properties:
The built-in classmethod and staticmethod decorators return descriptor objects, not functions.
Since language version 2.6 you can decorate classes, not just functions and methods.
Any class containing an __init__(self, somecallable) method and a __call__(self, *args, **kwargs) method can be used as a decorator.
Since there is no standardized decorator in Python, there's no real way of telling if a function is a decorator unless you know something about the decorator you're looking for.
If the decorator is under your control, you can add a mark to indicate it's a decorated function. Otherwise there is no real unified way of doing this. Take this example for instance:
def decorator(func):
return g
#decorator
def f()
pass
def g():
pass
In the above example, in run-time, f and g will be identical, and there is no way of telling the two apart.
Any callable with the right number of arguments can be used as a decorator. Remember that
#foo
def bar(...):
is exactly the same as
def bar(...):
...
bar = foo(bar)
Naturally, since foo could return anything, you have no way of checking whether a function has been decorated or not. Although foo could be nice and leave a mark, it has no obligation to do so.
If you are given some Python code and you want to find all the things that are decorators, you can do so by parsing the code into an abstract syntax tree then walking the tree looking for decorated functions. Here's an example, storing the .ids of the decorators. Obviously, you could store the astobjects if you wanted to.
>>> class DecoratorFinder(ast.NodeVisitor):
... def __init__(self, *args, **kwargs):
... super(DecoratorFinder, self).__init__(*args, **kwargs)
... self.decorators = set()
...
... def visit_FunctionDef(self, node):
... self.decorators.update(dec.id for dec in node.decorator_list)
... self.generic_visit(node)
...
>>> finder = DecoratorFinder()
>>> x = ast.parse("""
... #dec
... def foo():
... pass
... """)
>>> finder.visit(x)
>>> finder.decorators
set(['dec'])
No this is not possible. May be instead of checking if f is a decorator, you should think why you need to check that?
If you are expecting some specific decorator, you can directly check that, if you want some specific behavior/methods/attributes you can check that
If you want to check if some callable f can be used as decorator, you can test the decorator behavior by passing some dummy function, but in general it may not work or have different behavior for different inputs.
Here is a such naive check:
def decorator1(func):
def _wrapper(*args, **kwargs):
print "before"
func(*args, **kwargs)
print "after"
return _wrapper
def dummy_func(): pass
out_func = decorator1(dummy_func)
if callable(out_func) and dummy_func != out_func:
print "aha decorated!"
I've never done anything like this, but in general python relies on "duck-typing" in situations like this. So you could just try to decorate a dummy function and see if a callable is returned.