Non-test methods in a Python TestCase - python

Ok, as Google search isn't helping me in a while (even when using the correct keywords).
I have a class extending from TestCase in which I want to have some auxiliary methods that are not going to be executed as part of the test, they'll be used to generate some mocked objects, etc, auxiliary things for almost any test.
I know I could use the #skip decorator so unittest doesn't run a particular test method, but I think that's an ugly hack to use for my purpose, any tips?
Thanks in advance, community :D

I believe that you don't have to do anything. Your helper methods should just not start with test_.

The only methods that unittest will execute [1] are setUp, anything that starts with test, and tearDown [2], in that order. You can make helper methods and call them anything except for those three things, and they will not be executed by unittest.
You can think of setUp as __init__: if you're generating mock objects that are used by multiple tests, create them in setUp.
def setUp(self):
self.mock_obj = MockObj()
[1]: This is not entirely true, but these are the main 3 groups of methods that you can concentrate on when writing tests.
[2]: For legacy reasons, unittest will execute both test_foo and testFoo, but test_foo is the preferred style these days. setUp and tearDown should appear as such.

The test runner will only directly execute methods beginning with test, so just make sure your helper methods' names don't begin with test.

Related

Test python class with method calls in __init__

I have a class which calls a lot of its methods in __init__. Since a lot is going on in these methods, I want to test them. Testing classes and class methods requires to instantiate the class and then call its methods. But if I instantiate the class, the methods will already be called before I can test it.
I have some ideas for possible solutions, but I am unsure if they are possible or a good way to go:
I could introduce a kwarg into the class like init=True and testing for it within __init__. So I could have default option to do all the magic stuff on object creation, and could deactivate it to instantiate the class and call functions separately.
I could define the methods outside the class in other classes or functions, if that works, and test them separately. The test of the bigger class would become something like an integration test.
It depends on what you would like to test
If you want to check if all the calls are happening correctly you could mock underlying functionality inside the __init__ method.
And then do assert on the mocks. (Pytest has a spy mocks which does not modify original behavior but could be tested as mocks for call count, arguments etc... I'm sure you could replicate that with unittest mock as well)
So you could mock everything that is necessary by the beginning and then create an instance.
If you want to check how it was assembled you could do this after initialization.
Generally modifying your source code just for the purpose of the test case is not a good idea.
Run your code through a debugger, check what you are looking for as a tester and automate it.

Using Fixtures vs passing method as argument

I'm just learning Python and Pytest and came across Fixtures. Pardon the basic question but I'm a bit wondering what's the advantage of using fixtures in Python as you can already pass a method as argument, for example:
def method1():
return 'hello world'
def method2(methodToRun):
result = methodToRun()
return result
method2(method1)
What would be the advantage of passing a #pytest.fixture object as argument instead?
One difference is that fixtures pass the result of calling the function, not the function itself. That doesn't answer your question though why you'd want to use pytest.fixture instead of just manually calling it, so I'll just list a couple of things.
One reason is the global availability. After you write a fixture in conftest.py, you can use it in your whole test suite just by referencing its name and avoid duplicating it, which is nice.
In case your fixture returns a mutable object, pytest also handles the new call for you, so that you can be sure that other tests using the same fixture won't change the behavior between each other. If pytest didn't do that by default, you'd have to do it by hand.
A big one is that the plugin system of pytest uses fixtures to make its functionality available. So if you are a web dev and want to have a mock-server for your tests, you just install pytest-localserver and now adding httpserver, httpsserver, and smtpserver arguments to your test functions will inject the fixtures from the library you just installed. This is incredibly convenient and intuitive, in particular when compared to injection mechanisms in other languages.
The bottom line is that it is useful to have a single way to include dependencies in your test suits, and pytest chooses a fixture mechanism that magically binds itself to function signatures. So while it really is no different from manually inserting the argument, the quality of life things pytest adds through it make it worth it.
Fixture are a way of centralizing your test variables, avoid redundancy. If you are confortable with the concept of Dependency Injection, that's basically the same advantages, i.e. python will automatically bind your parameters with the available fixtures so you build tests more quickly by simply asking for what you need.
Also, fixtures enables you to easily parametrize all your tests at once. Which will avoid some cumbersome code if you want to do it by hand. (more info about it on the documentation: https://docs.pytest.org/en/latest/fixture.html#parametrizing-fixtures)
Some references:
Official documentation: https://docs.pytest.org/en/latest/fixture.html
Dependency injection: https://en.wikipedia.org/wiki/Dependency_injection

Python Testing - Reset all mocks?

When doing unit-testing with Python / PyTest, if you do you not have patch decorators or with patch blocks throughout your code, is there a way to reset all mocks at the end of every file / module to avoid inter-file test pollution?
It seems like something that is mocked in one Python test file remains mocked in other file with the same return value, which means my mocks are persisting between tests and files (when a patch decorator or with patch block is NOT used).
Is there any way around this other than patching? There wouldn't happen to be a mock.reset_all_mocks() or something like that, would there?
What I ended up doing was using the pytest-mock library. According to the Readme:
This plugin installs a mocker fixture which is a thin-wrapper around
the patching API provided by the excellent mock package, but with the
benefit of not having to worry about undoing patches at the end of a
test. (Emphasis added.)
So now I can do: mocker.patch.object(module, 'method', return_value='hi'), and the patch will be removed at the end of the test.
There is no need to use with any more so that this solution scales nicely if you have many mocks in one test or if you want to change mocks during the test.
After monkey-patching, I'm undoing it at the end of the test to avoid any leaking to other tests or limit the patching within the scope.
def test1(monkeypatch):
monkeypatch.setattr(...)
assert(...)
monkeypatch.undo()
why don't use monkeypatch ?
The monkeypatch function argument helps you to safely set/delete an attribute, dictionary item or environment variable or to modify sys.path for importing.
you can:
def test1(monkeypatch):
monkeypatch.setattr(.....

Is it OK to assert in unittest tearDown method?

I have a TestCase with multiple tests and need to assert a few conditions (the same for every test) at the end of each test. Is it OK to add these assertions to the tearDown() method, or is it a bad habit since they're not "cleaning" anything?
What would be the right way of doing this?
Asserting something in your tearDown means that you need to be careful that all the cleaning is done before the actual asserting otherwise the cleaning code may not be called if the assert statement fails and raises.
If the assert is just one line it may be OK to have it in every test methods, if it is more than that having a specific method would be a possibility- that method should not be a test of its own i.e. not recognized as a test by your test framework. Using a method decorator or class decorator may also be an alternative.
Overall the idea is that tearDown shouldn't do any testing and that explicit is better than implicit.
Mmh i have never seen this before. Personally i wouldn't do it because it doesn't belong there. I would do it via a decorator that does the asserts for you at the end. Then just decorate the test functions that you do want to have these asserts.
For an excellent introduction to python decorators see the answers to this question

Is having a unit test that is mostly mock verification a smell?

I have a class that connects three other services, specifically designed to make implementing the other services more modular, but the bulk of my unit test logic is in mock verification. Is there a way to redesign to avoid this?
Python example:
class Input(object): pass
class Output(object): pass
class Finder(object): pass
class Correlator(object): pass
def __init__(self, input, output, finder):
pass
def run():
finder.find(input.GetRows())
output.print(finder)
I then have to mock input, output and finder. Even if I do make another abstraction and return something from Correlator.run(), it will still have to be tested as a mock.
Just ask yourself: what exactly do you need to check in this particular test case? If this check does not rely on other classes not being dummy, then you are OK.
However, a lot of mocks is usually a smell in sense that you are probably trying to test integration without actually doing integration. So if you assume that if the class passes test with mocks, it will be fine working with real classes, than yes, you have to write some more tests.
Personally, I don't write many Unit tests at all. I'm web developer and I prefer functional tests, that test the whole application via HTTP requests, as users would. Your case may be different
There's no reason to only use unit test - Maybe integration tests would be more useful for this case. Initialize all the objects properly, use the main class a bit, and assert on the (possibly complex) results. That way you'll test interfaces, output predictability, and other things which are important further up the stack. I've used this before, and found that something which is difficult to integration test probably has too many attributes / parameters or too complicated/wrongly formatted output.
On a quick glance, this do look like the level of mocking becomes to large. If you're on a dynamic language (I'm assuming yes here since your example is in Python), I'd try to construct either subclasses of the production classes with the most problematic methods overridden and presenting mocked data, so you'd get a mix of production and mocked code. If your code path doesn't allow for instantiating the objects, I'd try monkey patching in replacement methods returning mock data.
Weather or not this is code smell also depends on the quality of mocked data. Dropping into a debugger and copy-pasting known correct data or sniffing it from the network is in my experience the preferred way of ensuring that.
Integration vs unit testing is also an economical question: how painful is it to replace unit tests with integration/functional tests? The larger the scale of your system, the more there is to gain with light-weight mocking, and hence, unit tests.

Categories