I am trying to produce a Mock of matplotlib so that I can compile my docs using ReadTheDocs, but have run into a problem.
In my code, I import matplotlib using from matplotlib.pyplot import *.
I am using the following code for my Mocks (as suggested by the ReadTheDocs FAQ):
class Mock(object):
def __init__(self, *args, **kwargs):
pass
def __call__(self, *args, **kwargs):
return Mock()
#classmethod
def __getattr__(cls, name):
if name in ('__file__', '__path__'):
return '/dev/null'
elif name[0] == name[0].upper():
return type(name, (), {})
else:
return Mock()
MOCK_MODULES = ['numpy', 'scipy', 'matplotlib', 'matplotlib.pyplot']
for mod_name in MOCK_MODULES:
sys.modules[mod_name] = Mock()
However, when running from matplotlib.pyplot import * I get an error saying that TypeError: 'type' object does not support indexing.
Is there a way that I can change my Mock so that it allows me to import matplotlib using the from x import * style? I don't need any particular functions to be made available, I just need it to be able to be imported so that ReadTheDocs can import the code properly.
In case of importing via * you need to define the __all__ list in the module. The same goes with your class: just add the __all__ attribute to the class and it should work fine:
class Mock(object):
__all__ = []
Related
I'm working on a project that dynamically collects classes from another module and creates new classes based off the ones it found. For example, if the source module has a "AdaptiveFilter" class, a new class called "AdaptiveFilterNode" is created that contains an "AdaptiveFilter" instance. I've been adding these generated classes to a dynamically created module (module_from_spec()) and created a MetaPathFinder and Loader to extend the import system. This works perfectly fine when importing the dynamic module, e.g. import nodes.auto_gen or from nodes.auto_gen import AdaptiveFilterNode, but fails when trying to directly import a class, e.g. import nodes.auto_gen.AdaptiveFilterNode. I need to be able to directly import a class because the generated classes need to be pickleable, which requires a direct import. I believe the problem is in the Finder that I made:
class MyNodeFinder(MetaPathFinder):
def find_spec(self, fullname, path=None, target=None):
if fullname == 'nodes.auto_gen':
auto_gen_spec = = ModuleSpec('nodes.auto_gen',
MyLoader())
return auto_gen_spec
return None
From my limited understanding on the import system, there's nothing in the finder or the module spec that tells the import system there's classes contained in the nodes.auto_gen module until the module is already loaded. How can I create a finder/loader that replicates the standard import system for a dynamic module with dynamic classes?
EDIT: Full example, module "dynamic_creator":
dynamic_creator/__init__.py
from importlib.machinery import ModuleSpec
from importlib.util import module_from_spec
import sys
from importlib.abc import MetaPathFinder
from importlib.abc import Loader
__version__ = '0.0.0'
class _MyWrapperClass():
wrapped_func = None
def __init__(self):
super().__init__()
def __call__(self, *args, **kwargs):
return self.wrapped_func(*args, **kwargs)
class MyFinder(MetaPathFinder):
def find_spec(self, fullname, path=None, target=None):
if fullname == 'dynamic_creator.auto_gen':
auto_node_spec = ModuleSpec('dynamic_creator.auto_gen',
MyLoader())
return auto_node_spec
return None
class MyLoader(Loader):
def create_module(self, spec):
return module_from_spec(ModuleSpec('dynamic_creator.auto_gen', None))
def exec_module(self, module):
def create_type(func):
t = type(func.__name__, (_MyWrapperClass, ), {})
t.wrapped_func = func
# Wait, you can do this?
t.__module__ = 'dynamic_creator.auto_gen'
return t
funcs_to_wrap = [
sum,
abs,
pow
]
for func in funcs_to_wrap:
setattr(module, func.__name__, create_type(func))
my_finder = MyFinder()
sys.meta_path.append(my_finder)
Test script:
import dynamic_creator.auto_gen # Works
from dynamic_creator.auto_gen import abs # Works
import dynamic_creator.auto_gen.pow as my_pow # Fails
EDIT: The reported error is as follows:
No module named 'dynamic_creator.auto_gen.pow'; 'dynamic_creator.auto_gen' is not a package
Suppose I have a decorator that collects all the functions it decorates to be called at some point in the future.
mydecorator.py
class CallLater(object):
funcs = []
def __init__(self, func):
self.funcs.append(func)
#classmethod
def call_now(cls, *args, **kwargs):
for func in cls.funcs:
func(*args, **kwargs)
Then, I have a function in a module, one of which will be saved by my decorator.
mymodule.py
import logging
from mydecorator import CallLater
logging.basicConfig(level=logging.INFO)
#CallLater
def log_a():
logging.info("A")
#CallLater
def log_b():
logging.info("B")
def log_c():
logging.info("C")
Now, if I import mymodule and call CallLater.call_now(), log_a and log_b will be called. But let's say that during testing, I want log_b to be substituted with log_c. I'll try to do the replacement with a mock.
mock_test.py
import logging
import pytest
from mymodule import log_a, log_c
from mydecorator import CallLater
logging.basicConfig(level=logging.INFO)
pytest_plugins = ('pytest_mock',)
def test_mocking(mocker, caplog):
mocker.patch('mymodule.log_b', log_c)
CallLater.call_now()
logs = [rec.message for rec in caplog.records]
assert logs == ["A", "C"]
But when I run pytest, I see that my mock didn't work.
FAILED mock_test.py::test_mocking - AssertionError: assert ['A', 'B'] == ['A', 'C']
I imagine that 'mymodule.log_b' is the wrong mocking target since it's not being invoked as mymodule.log_b(), but I'm not sure what to use instead in this situation. Any advice is appreciated!
This is not possible as such, the problem being that the functions are already assigned to the list at load time. The only way I can see to patch this would be to patch CallLater.funcs directly, which is a bit awkward, because you have to replace log_b by log_c manually - but here it goes:
import logging
from unittest import mock
from mymodule import log_c
from mydecorator import CallLater
logging.basicConfig(level=logging.INFO)
def test_mocking(caplog):
funcs = [log_c if f.__name__ == 'log_b' else f for f in CallLater.funcs]
with mock.patch.object(CallLater, 'funcs', funcs):
CallLater.call_now()
logs = [rec.message for rec in caplog.records]
assert logs == ["A", "C"]
Note that you cannot compare directly to the function (e.g. f == log_b), because log_b is the decorated function, not the function as saved in CallLater.funcs.
I am using unittest module for writing tests.
I need to test initialization of the object inside a testcase using different inputs.
For this purpose I am importing the class inside setUp(). But when I try to use the class inside test_*() functions, I get this error - NameError: name 'Example' is not defined
Here is my code sample-
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
def test_sample_function(self):
e = Example(1,2)
I know that I can simply import the class at top of the script. But I do not want to do that. I need to import it only during setup of the testscript.
Looking for some help here.
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
self.Example = Example
def test_sample_function(self):
e = self.Example(1,2)
There's no reason to import the module in setUp. The module is still available globally in sys.modules, but you've only bound it to a local name that goes away after setUp returns. Just import it globally.
import unittest
import Example
class TestExample(unittest.TestCase):
def test_sample_function(self):
e = Example(1,2)
Assume I have a function defined in a module:
module_a.py
def foo():
return 10
And I want to create an API to patch the function:
patcher.py
import mock
class Patcher(object):
def __enter__(self):
self.patcher = mock.patch('module_a.foo',
mock.Mock(return_value=15))
self.patcher.start()
def __exit__(self, *args):
self.patcher.stop()
The thing is, I don't know what is the name of the module that will use my API. so a test looking like this:
test1.py
from patcher import Patcher
import module_a
with Patcher():
assert module_a.foo() == 15
will work. But a test written like this:
test2.py
from patcher import Patcher
from module_a import foo
with Patcher():
assert foo() == 15
will Fail.
Is there anyway not making the API user to write it's tests and modules(!) like the first option?
There is a way to "patch" over a function without knowing where the patch is occurring. That was the requirement for my question since the patcher is my library API, and I don't want to be given a path to each test module using my library.
The solution I found was to pass on all loaded modules and try to find foo in them, and then changing it - sorta implementing patch by myself. If the import will happen only after the Patcher is started, I loaded the module myself, and changed it too.
Now the code will look like this:
Patcher
import sys
import mock
from module_a import foo as _orig_foo
import module_a
class Patcher(object):
def __init__(self):
self.undo_set = set()
self.fake_foo = mock.Mock(return_value=15)
def __enter__(self):
modules = [
module for mod_name, module in sys.modules.items() if
mod_name is not None and module is not None and
hasattr(module, '__name__') and
module.__name__ not in ('module_a', 'patcher')
]
for module in modules:
for attr in dir(module):
try:
attribute_value = getattr(module, attr)
except (ValueError, AttributeError, ImportError):
# For some libraries, this happen.
continue
if id(attribute_value) == id(_orig_foo):
setattr(module, attr, self.fake_foo)
self.undo_set.add((module, attr, attribute_value))
# Solve for future imports
module_a.foo = self.fake_foo
def __exit__(self, *args):
module_a.foo = _orig_foo
for mod, attr, val in self.undo_set:
setattr(mod, attr, val)
self.undo_set = set()
I have class called 'my_class' placed in 'my_module'. And I need to import this class. I tried to do it like this:
import importlib
result = importlib.import_module('my_module.my_class')
but it says:
ImportError: No module named 'my_module.my_class'; 'my_module' is not a package
So. As I can see it works only for modules, but can't handle classes. How can I import a class from a module?
It is expecting my_module to be a package containing a module named 'my_class'. If you need to import a class, or an attribute in general, dynamically, just use getattr after you import the module:
cls = getattr(import_module('my_module'), 'my_class')
Also, yes, it does only work with modules. Remember importlib.import_module is a wrapper of the internal importlib.__import__ function. It doesn't offer the same amount of functionality as the full import statement which, coupled with from, performs an attribute look-up on the imported module.
import importlib
import logging
logger = logging.getLogger(__name__)
def factory(module_class_string, super_cls: type = None, **kwargs):
"""
:param module_class_string: full name of the class to create an object of
:param super_cls: expected super class for validity, None if bypass
:param kwargs: parameters to pass
:return:
"""
module_name, class_name = module_class_string.rsplit(".", 1)
module = importlib.import_module(module_name)
assert hasattr(module, class_name), "class {} is not in {}".format(class_name, module_name)
logger.debug('reading class {} from module {}'.format(class_name, module_name))
cls = getattr(module, class_name)
if super_cls is not None:
assert issubclass(cls, super_cls), "class {} should inherit from {}".format(class_name, super_cls.__name__)
logger.debug('initialising {} with params {}'.format(class_name, kwargs))
obj = cls(**kwargs)
return obj