I am sorry if the question title is vague, I could not think of a better one.
I have a bunch of functions inside a module which I wish behaved differently when called locally versus when called from other modules.
Here is a toy example
moduleA.py
def func(arg1):
pass
do something
moduleB.py
import moduleA
func(arg1)
In moduleB the call for func() needs to do
initSomething
func(arg1)
doSomethingElse
And when func() is called from moduleA, I still need the original behavior.
While the problem screams at me for using decorators, I am not sure on writing a decorator for func() that will be triggered only when called from a module.
Sounds to like you want to give the function calls a certain context. That's what context managers are for. You could do sth like:
from contextlib import contextmanager
#contextmanager
def func_context():
# init_something
yield
# do_something_else
with func_context():
func(arg1)
Related
I'm currently setting up a test suite for a file called main.py. The test file is called test_main.py. Here's an example:
# main.py
def add(a,b):
return a+b
#test_main.py
import pytest
from main import *
def test_add():
assert add(1,2) == 3
For reasons which are outside the scope of this question, I would like to dynamically load the function add in test_main.py as opposed to calling it directly. I'm already aware this is possible using the following
globals or vars
use of importlib
use of eval
However, I'd like to see if there's another option. globals and vars are bad practice. eval is allright, but it doesn't return the function object and I have to do some string manipulation to get the function call, including its arguments right. importlib is by far the best option, but main.py happens to contain functions which I want to import the "normal" way. It feels wrong to import functions in test_main.py using both an import statement and the importlib module.
So, is there a better way? One which is more pythonic?
I have a function that has a decorator. The decorator accepts arguments and the value of the argument is derived from another function call.
example.py
from cachetools import cached
from cachetools import TTLCache
from other import get_value
#cached(cache=TTLCache(maxsize=1, ttl=get_value('cache_ttl')))
def my_func():
return 'result'
other.py
def get_value(key):
data = {
'cache_ttl': 10,
}
# Let's assume here we launch a shuttle to the space too.
return data[key]
I'd like to mock the call to get_value(). I'm using the following in my test:
example_test.py
import mock
import pytest
from example import my_func
#pytest.fixture
def mock_get_value():
with mock.patch(
"example.get_value",
autospec=True,
) as _mock:
yield _mock
def test_my_func(mock_get_value):
assert my_func() == 'result'
Here I'm injecting mock_get_value to test_my_func. However, since my decorator is called on the first import, get_value() gets called immediately. Any idea if there's a way to mock the call to get_value() before module is imported right away using pytest?
Move the from example import my_func inside your with in your test function. Also patch it where it's really coming from, other.get_value. That may be all it takes.
Python caches modules in sys.modules, so module-level code (like function definitions) only runs on the first import from anywhere. If this isn't the first time, you can force a re-import using either importlib.reload() or by deleting the appropriate key in sys.modules and importing again.
Beware that re-importing a module may have side effects, and you may also want to re-import the module again after running the test to avoid interfering with other tests. If another module was using objects defined in the re-imported module, these don't just disappear, and may not be updated the way it expects. For example, re-importing a module may create a second instance of what was supposed to be a singleton.
One more robust approach would be save the original imported module object somewhere else, delete from sys.modules, re-import with the patched version for the duration of the test, and then put back the original import into sys.modules after the test. You could do this with an import inside of a patch.dict() context on sys.modules.
import mock
import sys
import pytest
#pytest.fixture
def mock_get_value():
with mock.patch(
"other.get_value",
autospec=True,
) as _mock, mock.patch.dict("sys.modules"):
sys.modules.pop("example", None)
yield _mock
def test_my_func(mock_get_value):
from example import my_func
assert my_func() == 'result'
Another possibility is to call the decorator yourself in the test, on the original function. If the decorator used functools.wraps()/functools.update_wrapper(), then original function should be available as a __wrapped__ attribute. This may not be available depending on how the decorator was implemented.
During a test phase, I would like to modify a part of my source code. For instance, I don't need to plot, so the code responsible for plotting can be commented. Is there a proper tool in unittest module to do so?
Another question is regarding the decorator #patch. Is there a way I can put them into source code in runtime during testing?
Try working with mock,
As it sounds it mocks your code and can be manipulated thru the test.
You may mock an method returned val or an object instance etc etc.
https://www.toptal.com/python/an-introduction-to-mocking-in-python
As Ohad the Lad already said, mocks are at your help.
There are several ways how to mock a function, but in general, you will decorate your test-functions - not the function to mock!
In your case the code could look as follows:
# your_file.py
from ... import ... as plt #depends on the class you use
class MyPlotter(object):
def draw_calling_method(self):
....
plt.draw()
...
return something
# test.py
import mock
from unittest import TestCase
from your_file import MyPlotter
from ... import ... as plt # same import as in the your_file.py
class TestMyPlotter(TestCase):
#mock.patch.object(plt, 'draw')
def test_draw_calling_method_returns_something(self, draw):
plotter = MyPlotter()
plotter_return = plotter.draw_calling_method()
self.assertEqual(plotter_return, something)
This will replace all calls to the plt.draw() with MagicMocks and hinder the draw-execution. If all of your test-methods need to mock the draw-call, the mock decorator could also be applied to the class instead. You only have to make sure then, that all your test-methods accept the mock as second argument (as the test_draw_calling_mehtod_returns_something does). Further you could mock any return values the draw() function may have by setting draw.return_value = ... in the test-method. Attention: This has to happen prior to the call of the function which calls the mock, otherwise the return-values will not be applied.
For more information on mocking refer to the docs python-docs. They are quite comprehensive.
One last thing; As Jonathon Reinhart already mentioned, if you feel it difficult to write your tests it may be good to refactor your code before. This will not only make it more testable, but more readable as well!
I found the same question, but I can't comment an answer there.
Python: Calling a function from a file that has current file imported
I have one.py:
import two
def one_bis()
print('something')
def one():
two.two()
one()
... and two.py:
def two():
one_bis()
Ulrich Eckhardt throw some posibilities and there're two that I'm interested in (in bold):
Move the common function to a module imported by both other modules.
Merge both modules into one.
Pass the function from main to the code that needs to call it.
Monkey patch the function into the check module after importing it.
Refactor the whole thing so that you don't have circular dependencies.
How should I do those solution?
This is not an answer to your "how do I do it the complicated way?" question but one more alternative to do this.
# one.py
import two
def one_bis():
print('something')
def one():
two.two()
one()
.
# two.py
def two():
from one import one_bis
one_bis()
If you really want to patch module two then add the following code to module one before you call one() (which calls two.two()).
two.two = one_bis
But I recommend refactoring your application.
well I don't know if this helps, but I did a bit of tinkering and found that for me, the best method is to pass it in as an arguement, I was also looking for something similar, so here it is
# one.py
import two
def one_bis():
print('something')
def one():
a=one_bis
two.two(a)
one()
then
#two.py
def two(n=None):
if n!=None:
return n()
I have a module which contains a lot of functions (more than 25). I want to add a common decorator function to each of these functions. The normal way to do is to add a #decorator line above each function, but I was wondering if there is a better way to do it? Probably I can declare a global decorator at the top of the module or something else?
Note that since I am using someone else's code, I want to minimize the number of lines changed, so modifying the module is not ideal for me.
Thanks.
If your decorator is called my_decorator
### Decorate all the above functions
import types
for k,v in globals().items():
if isinstance(v, types.FunctionType):
globals()[k] = my_decorator(v)
You could also apply this to the module after importing it
import othermodule
import types
for k,v in vars(othermodule).items():
if isinstance(v, types.FunctionType):
vars(othermodule)[k] = my_decorator(v)
I think applying a decorator en-masse such that it's not obvious where you will go looking to find out about the function (at its definition) is generally a bad idea. Explicit is better than implicit, and all that.
If you want to apply the decorator to some third party module's functions, without modifying the third-party code, here is how I would do it:
# my_wrapper_module.py
import some_module
import functools
def some_decorator(func):
#functools.wraps(func):
def wrapper(*args, **kwargs):
...
return wrapper
FUNCTION_NAMES = [
'some_func_1',
'some_func_2',
'some_func_3',
...
]
for name in FUNCTION_NAMES:
globals()[name] = some_decorator(getattr(some_module, name))
And then use these functions elsewhere by doing from my_wrapper_module import some_func_2, etc.
For me, this has the following advantages:
No need to modify the third-party source file
It is clear from the call site that I should go look at my_wrapper_module to see what I'm calling, and that I'm not using the undecorated versions of the functions
It is clear from my_wrapper_module what functions are being exported, that they originally come from some_module, and that they all have the same decorator applied
Any code that imports some_module directly isn't silently and inexplicably affected; this could be particularly important if the third-party code is more than one module
But if what you're trying to do is hack a third-party library so that internal calls are affected, then this is not what you want.