I have a class that connects three other services, specifically designed to make implementing the other services more modular, but the bulk of my unit test logic is in mock verification. Is there a way to redesign to avoid this?
Python example:
class Input(object): pass
class Output(object): pass
class Finder(object): pass
class Correlator(object): pass
def __init__(self, input, output, finder):
pass
def run():
finder.find(input.GetRows())
output.print(finder)
I then have to mock input, output and finder. Even if I do make another abstraction and return something from Correlator.run(), it will still have to be tested as a mock.
Just ask yourself: what exactly do you need to check in this particular test case? If this check does not rely on other classes not being dummy, then you are OK.
However, a lot of mocks is usually a smell in sense that you are probably trying to test integration without actually doing integration. So if you assume that if the class passes test with mocks, it will be fine working with real classes, than yes, you have to write some more tests.
Personally, I don't write many Unit tests at all. I'm web developer and I prefer functional tests, that test the whole application via HTTP requests, as users would. Your case may be different
There's no reason to only use unit test - Maybe integration tests would be more useful for this case. Initialize all the objects properly, use the main class a bit, and assert on the (possibly complex) results. That way you'll test interfaces, output predictability, and other things which are important further up the stack. I've used this before, and found that something which is difficult to integration test probably has too many attributes / parameters or too complicated/wrongly formatted output.
On a quick glance, this do look like the level of mocking becomes to large. If you're on a dynamic language (I'm assuming yes here since your example is in Python), I'd try to construct either subclasses of the production classes with the most problematic methods overridden and presenting mocked data, so you'd get a mix of production and mocked code. If your code path doesn't allow for instantiating the objects, I'd try monkey patching in replacement methods returning mock data.
Weather or not this is code smell also depends on the quality of mocked data. Dropping into a debugger and copy-pasting known correct data or sniffing it from the network is in my experience the preferred way of ensuring that.
Integration vs unit testing is also an economical question: how painful is it to replace unit tests with integration/functional tests? The larger the scale of your system, the more there is to gain with light-weight mocking, and hence, unit tests.
Related
I have a custom framework which runs different code for different clients. I have monkeypatched certain methods in order to customize functionality for a client.
Here is the pattern simplified:
#import monkeypatches here
if self.config['client'] == 'cool_dudes':
from app.monkeypatches import Stuff
if self.config['client'] == 'cool_dudettes':
from app.monkeypatches import OtherStuff
Here is an example patch:
from app.framework.stuff import Stuff
def function_override(self):
return pass
Stuff.function = function_override
This works fine when the program executes as it is executed in a batch manner, spinning up from scratch every time. However, when running across unit tests, I find that the monkey patches persist across tests, causing unexpected behavior.
I realize that it would be far better to use an object oriented inheritance approach to these overrides, but I inherited this codebase and am not currently empowered to rearchitect it to that degree.
Barring properly re-architecting the program, how can I prevent these monkey patches from persisting across unit tests?
The modules, including app.framework.<whatever>, are not reloaded for every test. So, any changes in them you make persist. The same happens if your module is stateful (that's one of the reasons why global state is not such a good idea, you should rather keep state in objects).
Your options are to:
undo the monkey-patches when needed, or
change them into something more generic that would change (semi-)automatically depending on the test running, or
(preferred) Do not reinvent the wheel and use an existing, manageable, time-proven solution for your task (or at least, base your work on one if it doesn't meet your requirements completely). E.g. if you use them for mocking, see How can one mock/stub python module like urllib . Among the suggestions there is #mock.patch that does the patching for a specific test and undoes it upon its completion.
Anyone coming here looking for information about monkeypatching, might want to have a look at pytest's monkeypatch fixture. It avoids the problem of the OP by automatically undoing all modifications after the test function has finished.
I am very new to Python and I saw many projects on Github using Mock to do their test but I don't understand why.
When we use mock, we construct a Mock object with a specific return_value, I don't truely understand why we do this. I know sometimes it is difficult to build our needed resources but what is the point if we construct a object/function with a certain return value ?
Mock can help to write unit tests.
In unit tests, you want to test a small portion of your implementation. For example, as small as one function or one class.
In a moderately large software, these small parts depend on each other. Or sometimes there are external dependencies. You open files, do syscalls, or get external data in some other way.
While writing a directed unit test for a small portion of your code, you do not want to spend time to set-up everything else around it. (The files, syscalls, external data). Mock comes to your help there. With mock, you can make the other dependencies of your code behave exactly as you like. This way you can focus on testing your intended implementation.
Coming to the mock with the return value: Say you want to test func_a. And func_a calls func_b. func_b does a lot of funky processing to calculate its return value. For example, talking to an external service, doing bunch of syscalls, or some other expensive operation. Since you are testing func_a, you only care about possible return values of func_b. (So that func_a can use them) In this scenario you would mock func_b and set the return values explicitly. This can really simplify your test complexity.
A lot of my mock usage is in unit tests, but I am not sure if I can use the mock library for production, consider the following trivial example to get data from external source.
class Receiver(object):
def get_data(self):
return _call_api(...)
Now, can I use mock library to change the get_data() function for re-run purpose on production?
with patch('Receiver.get_data') as mock_get_data:
mock_get_data.return_values = [1, 2]
...
Some might suggest to write another Rerun receiver as a better approach, while I don't disagree but I am still raising this question for the sake of curiosity.
My questions include:
If no, what's the reason?
If yes, any caveats?
I would agree that for production use, a Receiver subclass that has an overridden get_data method would be much better.
The reason is simple -- if each type of receiver only receives data from a single source then your code will be much easier to read and maintain. If the same Reciever will end up returning data from multiple sources, then the code will be confusing and you'll end up needing to hunt down whether you were fetching data from one place of whether it's data that you explicitly set via mock, etc.
No. If a function is supposed to behave a certain way in production, then code it to behave that way. If you need fallback or retry behavior, mock is not the right way to do that.
I'm rather new to unit-testing and am trying to feel out the best practices for the thing. I've seen several questions on here relating to unit-test inheriting a base class that itself contains several tests, for example:
class TestBase(unittest.TestCase):
# some standard tests
class AnotherTest(TestBase):
# run some more tests in addition to the standard tests
I think what I've gathered from the community is that it's a better idea to write separate tests for each implementation and use multiple inheritance. But what if that base class actually doesn't contain any tests - just helpers for all your other tests. For example, let's say I've got some base test class which I've used to store some common methods that most if not all of my other tests will use. Let's also assume that I've got a database model in models.py called ContentModel
test_base.py
import webtest
from google.appengine.ext import testbed
from models import ContentModel
class TestBase(unittest.TestCase):
def setUp(self):
self.ContentModel = ContentModel
self.testbed = testbed.Testbed()
self.testbed.activate()
# other useful stuff
def tearDown(self):
self.testbed.deactivate()
def createUser(self, admin=False):
# create a user that may or may not be an admin
# possibly other useful things
It seems this would save me tons of time on all other tests:
another_test.py
from test_base import TestBase
class AnotherTest(TestBase):
def test_something_authorized(self):
self.createUser(admin=True)
# run a test
def test_something_unauthorized(self):
self.createUser(admin=False)
# run a test
def test_some_interaction_with_the_content_model(self):
new_instance = self.ContentModel('foo' = 'bar').put()
# run a test
Note: this is based on some of my work in webapp2 on google app engine, but I expect that
an analogous situation arises for pretty much any python web application
My Question
Is it good practice to use a base/helper class that contains useful methods/variables which all your other tests inherit, or should each test class be "self contained"?
Thanks!
Superb question. I think that almost anything you do that automates testing is excellent. That said, the tests really serve as the only reliable source of documentation. So the tests should be very easy to read and comprehend. The tests are reliable, unlike comments, because they show what the software really does and how to use it.
I like this approach. But you might also try out nose. Nose is a bit "lighter weight" to set up, and is well supported if you go the continuous integration route with something like Jenkins for automated build/test/deployment. Nose does not format its messages quite as nicely as the xUnit style (IMO, of course). But for many things, you might be willing to give that up.
BTW. Python is not Java. So it is perfectly acceptable to reuse just a plain old python function for re-use.
A base class is a good option for some uses - as long as you don't test anything in the base class. I use base classes all the time.
Also, think of the value of seeing the code in your test class. A good example is a base class I use all the time (in c#.NET): I use a SDK - ArcObjects from Esri - that requires a license. In normal execution this is handled elsewhere, but in testing, I have to check out (or activate) a license before I can use the objects in the library. This has absolutely nothing to do with the functionality of the code I am testing in the test class, but is required to make the tests run. Thus, I decided to tuck this functionality away in a base class that check s out a license before a test and checks it back in after. Tests that requires a licence are simply inherriting from this base class.
Finally, be very careful about where you setup and tear down the prerequisites for the test. It can get messy if some is done in the base class and others are done in the child class.
How do you do unit testing when you have
some general unit tests
more sophisticated tests checking edge cases, depending on the general ones
To give an example, imagine testing a CSV-reader (I just made up a notation for demonstration),
def test_readCsv(): ...
#dependsOn(test_readCsv)
def test_readCsv_duplicateColumnName(): ...
#dependsOn(test_readCsv)
def test_readCsv_unicodeColumnName(): ...
I expect sub-tests to be run only if their parent test succeeds. The reason behind this is that running these tests takes time. Many failure reports that go back to a single reason wouldn't be informative, either. Of course, I could shoehorn all edge-cases into the main test, but I wonder if there is a more structured way to do this.
I've found these related but different questions,
How to structure unit tests that have dependencies?
Unit Testing - Is it bad form to have unit test calling other unit tests
UPDATE:
I've found TestNG which has great built-in support for test dependencies. You can write tests like this,
#Test{dependsOnMethods = ("test_readCsv"))
public void test_readCsv_duplicateColumnName() {
...
}
Personally, I wouldn't worry about creating dependencies between unit tests. This sounds like a bit of a code smell to me. A few points:
If a test fails, let the others fail to and get a good idea of the scale of the problem that the adverse code change made.
Test failures should be the exception rather than the norm, so why waste effort and create dependencies when the vast majority of the time (hopefully!) no benefit is derived? If failures happen often, your problem is not with unit test dependencies but with frequent test failures.
Unit tests should run really fast. If they are running slow, then focus your efforts on increasing the speed of these tests rather than preventing subsequent failures. Do this by decoupling your code more and using dependency injection or mocking.
Proboscis is a python version of TestNG (which is a Java library).
See packages.python.org/proboscis/
It supports dependencies, e.g.
#test(depends_on=[test_readCsv])
public void test_readCsv_duplicateColumnName() {
...
}
I'm not sure what language you're referring to (as you don't specifically mention it in your question) but for something like PHPUnit there is an #depends tag that will only run a test if the depended upon test has already passed.
Depending on what language or unit testing you use there may also be something similar available
I have implemented a plugin for Nose (Python) which adds support for test dependencies and test prioritization.
As mentioned in the other answers/comments this is often a bad idea, however there can be exceptions where you would want to do this (in my case it was performance for integration tests - with a huge overhead for getting into a testable state, minutes vs hours).
You can find it here: nosedep.
A minimal example is:
def test_a:
pass
#depends(before=test_a)
def test_b:
pass
To ensure that test_b is always run before test_a.
You may want use pytest-dependency. According to theirs documentation code looks elegant:
import pytest
#pytest.mark.dependency()
#pytest.mark.xfail(reason="deliberate fail")
def test_a():
assert False
#pytest.mark.dependency()
def test_b():
pass
#pytest.mark.dependency(depends=["test_a"])
def test_c():
pass
#pytest.mark.dependency(depends=["test_b"])
def test_d():
pass
#pytest.mark.dependency(depends=["test_b", "test_c"])
def test_e():
pass
Please note, it is plugin for pytest, not unittest which is part of python itself. So, you need 2 more dependencies (f.e. add into requirements.txt):
pytest==5.1.1
pytest-dependency==0.4.0
According to best practices and unit testing principles unit test should not depend on other ones.
Each test case should check concrete isolated behavior.
Then if some test case fail you will know exactly what became wrong with our code.