We Keep Coding

Pytest collection warning __init__ constructor

I really couldn't find a solution for :
PytestCollectionWarning: cannot collect test class 'TestBlaBla' because it has a init constructor
Here is my testing module and it needs to take the arguments outside of this file because at the end of the day I'm gonna call all test modules in a single file and run them all over different names and there is a bunch of names. I have to init them but when I run pytest it always ignores these classes. Idk how to handle without initializing them. If there is any suggestions I would be glad to hear.
tests/test_bla_bla.py
class TestBlaBla():
def __init__(self, **kwargs):
self.name1 = kwargs.get("name1")
self.name2 = kwargs.get("name2")
#pytest.fixture(scope='session')
def load_data_here(self):
return load_data(self.name1) # a function comes from a utils file. it only stands for load data and it needs take a name for path
..
"continue with test_ functions that use output of load_data_here"
tests/main.py
class TestingAll:
def __init__(self, *args, **kwargs):
self.name1 = kwargs.get("name1")
self.name2 = kwargs.get("name2")
self._process()
def _process(self):
TestBlaBla(name1 = self.name1, name2= self.name2)
TestBlaBla2(name1 = self.name1, name2= self.name2)
if __name__ == "__main__":
Test = TestingAll(name1 = "name1", name2= "name2")

Python test modules cannot have init methods as python test instantiates the class itself and there is not any way (IMHO?) to extend the instantiation and add arguments.
Yes, it is a natural idea to want to make your tests flexible by passing in command-line arguments. But you can't :-P. So you need to find another way of doing this.
Note also the if name == 'main': can work if you call the test file with python and add some code to explicitly call a py test runner. Note you do not just call your test class. A python test runner needs to be instantiated itself and run tests in a particular way.
e.g. we can have this which will allow python to instantiate and run tests in a TestingAll class (as long as it doesn't have an init method).
This uses the unittest.TextTestRunner
Note there are all sorts of python test runners, also runners like nose2 or like py.test which use a different test library.
if __name__ == '__main__':
unittest.main()
suite = unittest.TestLoader().loadTestsFromTestCase(TestingAll)
unittest.TextTestRunner(verbosity=3).run(suite)
You could maybe have an ArgsProcess class to process command line args.
Then iterate and set a global var with each value to be used and call the test runner each time.
But it depends on how your tests will be used.
The answers on this question already mentioned in comments explain and link to documentation for this warning:
py.test skips test class if constructor is defined
The answer on this question shows how an init method is replaced by using fixture:
Pytest collection warning due to __init__ constructor
Maybe something like this would work for you.

unittest blacklist namespace and fail any attempt to reference it

in the case of unit testing a wrapper library, testing the wrapper without depending/exercising the upstream library is a goal; In a known case, all calls to the upstream library can be mocked and that's what I've done, but I've been frustrated by changes to the wrapper that introduce more calls to the upstream library being missed by the mock tools;
How can I best fail any test that tries to use a given namespace?
My idea currently is to change all the unittest methods to have a monkey patch like
#unittest.mock.patch('wrapper_namespace.upsteam_namespace')
and reply the upstream library with a mock that can be asserted untouched; I'm hoping for an option that works globally, so that I
don't have to add a monkeypatch to every test method, though this level of granularity is acceptable; but also don't have to perform the assertion that the mock was never used in the test methods (or make a decorator to do all that either)
prohibits access to the upstream library from any part of the software
(e.g, Wrapper calls B calls Upstream, B's call to upstream might not be caught)

You don't have to patch every test method. You can easily patch over the class if you're using unittest, or just assign the module to whatever you want to patch over it with. Here's a workable example:
A fake lib in some_lib.py:
def some_lib_func():
raise ValueError("I've been called.")
def some_other_lib_func():
raise ValueError("I've been called.")
class SomeClass:
def __init__(self):
raise ValueError("I've been constructed.")
wrapper.py:
import some_lib
def wrapper1():
some_lib.some_lib_func()
def wrapper2():
some_lib.some_other_lib_func()
def wrapper3():
x = some_lib.SomeClass()
test.py:
from unittest.mock import patch, MagicMock
import unittest
import wrapper
# Alternative:
# wrapper.some_lib = MagicMock()
# Can patch an entire class
#patch('wrapper.some_lib', MagicMock())
class TestWrapper(unittest.TestCase):
def test_wrapper1(self):
wrapper.wrapper1()
def test_wrapper2(self):
wrapper.wrapper2()
def test_wrapper3(self):
wrapper.wrapper3()
if __name__ == "__main__":
unittest.main()
We would explode if the functions/classes in some_lib were called, but they aren't:
Matthews-MacBook-Pro:stackoverflow matt$ python test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
Feel free to comment out the patch and comment in wrapper.some_lib = MagicMock(). You'll get the same result in this toy example, but there is a major difference between the two approaches:
When using #patch('wrapper.some_lib', MagicMock()) the patch is only live for that Test Case class.
When using wrapper.some_lib = MagicMock(), however, that patch will stay live for the entire length of your python program, unless you save off the original module and patch it back manually at some point. Everything that is using the wrapper module will get the mocked version.
So you could so something like:
original_lib = wrapper.some_lib
wrapper.some_lib = MagicMock()
...
# call some test suite, every call to the wrapper module will be mocked out
...
wrapper.some_lib = original_lib
...
# call some other test suite that actually needs the real thing
...
HTH.
EDIT: Misread your question slightly, but you can inspect MagicMock objects to see if they've been called, and if so, fail the test. Or just patch over with something that fails when called (instead of MagicMock). I can provide code to do this if requested (just leave a comment), but hopefully the above can get you started. I think the crux of the question was really about the global patching. Cheers!

Skipping code in python if in a unittest

My current script calls an external script to perform some task. I want to check the code up to that point in a unittest, but not actually run the external script. Is there some way I can tell the script to effectively skip the following block IF the code is being run as part of a unit test?

The unittest package has extensive support for "mocking" functions and methods. Encapsulate the call to an external program in a simple function that your unit tests can override ("mock out") without modifying the structure of your program. Example:
Here is part of your program, in the module realcode.py
def internal_function_calling_exec(arg1):
"""The real thing"""
print("I am executing an external program")
def bigger_function_being_tested(arg1, arg2):
"""
A complex function with one or more calls to `internal_function_calling_exec`
"""
print("I will now call `internal_function_calling_exec()`")
internal_function_calling_exec(42)
Your unit test can then look like this:
import unittest
from unittest.mock import patch
import realcode
class MyTest(unittest.TestCase):
#patch("realcode.internal_function_calling_exec")
def test_something(self, mocked_func):
realcode.bigger_function_being_tested(1, 2)
mocked_func.assert_called_with(42)
This will never call the original internal_function_calling_exec(). Instead, this will trigger a call to the mock object; your test can then query the object to confirm that it was called properly.
There are ways to mock class methods etc., so you could mock subprocess.call instead, for example. But I think the above is the better pattern.

One possible approach is to set an environment variable in the unit test, and check for that environment variable in the script being tested.
For example, in unittest.py:
os.environ["testing"] = "1"
And in script-to-be-tested.py:
testing = os.environ["testing"]
... do stuff based on the testing variable
Since script-to-be-tested.py will be called from unittest.py, it should inherit the environment variables.
Possibly not the cleanest solution, but it should work.

How do I write a nose2 plugin that separates different types of tests?

I'm writing a plugin that will separate treat my unit tests, functional tests & integration tests differently.
My tests folder will have the following structure exactly:
/tests
-- /unit
-- /functional
-- /integration
Each unit test will reside in the unit directory and each functional test will reside in the functional directory and so on.
I am familiar with the Layers plugin but I'd rather have my tests follow a convention.
Which hook exactly should I use to inject the appropriate Layer before tests are run?
Should it be the loadTestsFromModule hook? Can you show me an example?
I'd also like to separate the summary report for each type of test.
Which hook should I use?

I got this working with nose2 by using the nose2 attrib plugin for discovery and some code copied from the nose1 attrib plugin which allowed me to decorate my tests.
Using the nose2 attrib plugin
You will see the nose2 attrib plugin allows for custom attributes to be defined on test functions and classes.
For this to work, you have to specify the attributes of the tests after defining the test function.
class MyTestCase(unittest.TestCase):
def test_function(self):
self.assertEqual(1+1, 2)
test_function.custom_attr1 = True
test_function.custom_attr2 = ['foo', 'bar']
Then you can run a set of filtered tests by specifying -A or --attribute as a nose2 command-line argument to list the attribute(s) you to match against your test suite. You can even use the expression command-line argument of -E or --eval-attribute which allows more complex Python expressions for matching test attributes.
e.g. nose2 -v -A custom_attr1
will run all tests which have a custom_attr1 specified with a truthy value.
Using decorators to specify test attributes
This wasn't quite good enough for me though because I didn't like the idea of defining these attributes on tests after their definition. I wanted to use a decorator instead but nose2 didn't have a built-in decorator for doing this.
I went to the nose1 source code for its attrib plugin and copied the source for the attr function.
def attr(*args, **kwargs):
"""Decorator that adds attributes to classes or functions
for use with the Attribute (-a) plugin.
"""
def wrap_ob(ob):
for name in args:
setattr(ob, name, True)
for name, value in kwargs.iteritems():
setattr(ob, name, value)
return ob
return wrap_ob
I put this into a test/attrib_util.py file. Now I can specify attributes using the decorator instead. My original test class code from above can be converted to the (IMO) simpler:
from test.attrib_util import attr
class MyTestCase(unittest.TestCase):
#attr('custom_attr1', custom_attr2=['foo', 'bar'])
def test_function(self):
self.assertEqual(1+1, 2)
You will notice that the attributes can be specified as either args or kwargs; all args will get a default value of True.
You can also even use this attr decorator on a test class or base class and the attributes will be applied to all test functions defined within. This allows for very easy separation of unit and functional tests.
from test.attrib_util import attr
#attr('functional')
class FunctionalTestCase(unittest.TestCase):
pass
class MyFunctionalCase(FunctionalTestCase):
def test_function(self):
print 'this will be considered a "functional" test function'

You don't need to write a plug-in, the built-in attr module is designed for this purpose. It does not depend on your file hierarchy, however. Instead, you mark individual tests as unit, functional, or integration. This would look like:
from nose.plugins import attrib
#attrib.attr("functional")
class FunctionalTestCase(unittest.TestCase):
pass
To run only the functional tests, you would then do:
nosetests -a functional
If I were creating this test layout, I would probably have 3 unittest.TestCase subclasses, already marked with "unit", "functional", and "integration". New tests could easily inherit the proper test type.

If you already have the tests sorted into directories (as you mentioned), you could write a plugin that uses the wantDirectory method.
import os.path
from nose.plugins import Plugin
class TestCategory(Plugin):
"""
Run tests in a defined category (unittest, functional, integration. Always
runs uncategorized tests.
"""
def wantDirectory(self, dirname):
dirname = os.path.basename(dirname)
if (dirname in ('unit', 'functional', 'integration') and
dirname != self.category):
return False
return None
You will want to write options() and configure() methods for this plug-in to deal with enabling and disabling it and gleaning the user's choice of category. When running nosetests you would choose from the three categories:
nosetests --category functional
Since only one test category is run at a time, you would get a separate report for each test category. You could always, of course, run all tests by not enabling this plugin.
(adding as a different answer because it is a completely different approach).

Nose test single setup function called once

How do I create a single setup function all my nose test cases that is only called once during initialization? I have a global configuration that only needs to be set once and I feel that adding the following to each module (even calling a setup function for each module) is a bit superfluous:
def setUp(self):
Configuration.configure('some configuration settings')

I figured it out! Nose provides package-level setup and teardown as documented here. All I have to do is define the setup method in the package's __init__.py file.
Here, you can see an example of how to use the setup function. To make things simple:
lines = []
def setup():
global lines
lines.append('test') # here, we can trigger a build
# and read in a file, for example
def test_this():
assert lines[0] == 'test'