I have problem organizing my unittest based class test for family of tests. For example assume I implement a "dictionary" interface, and have 5 different implementations want to testing.
I do write one test class that tests a dictionary interface. But how can I nicely reuse it to test my all classes? So far I do ugly:
DictType = hashtable.HashDict
In top of file and then use DictType in test class. To test another class I manually change the DictType to something else.
How can do this otherwise? Can't pass arguments to unittest classes so is there a nicer way?
The way I tackle this with standard unittest is by subclassing -- overriding data is as easy as overriding methods, after all.
So, I have a base class for the tests:
class MappingTestBase(unittest.TestCase):
dictype = None
# write all the tests using self.dictype
and subclasses:
class HashtableTest(MappingTestBase):
dictype = hashtable.HashDict
class OtherMappingTest(MappingTestBase):
dictype = othermodule.mappingimpl
Here, the subclasses need override only dictype. Sometimes it's handier to also expose MappingTestBase use "hook methods". When the types being tested don't have exactly identical interfaces in all cases, this can be worked around by having the subclasses override the hook methods as needed -- this is the "Template Method" design pattern, see e.g. this page which has a commented and timelined collection of a couple of video lectures of mine on design patterns -- part II is about Template Method and variants thereof for about the first 30 minutes.
You don't have to have all of this in a single module, of course (though I often find it clearest to lay things out this way, you could also make one separate test module per type you're testing, each importing the module with the abstract base class).
You could look at testscenarios which allows you to set a list called scenarios. The code then generates a version of the test class for each value/scenario in the list
See the example
So in your case the scenarios would be a list like [ {dicttype:hashtable.HashDict}, {dicttype:otherimpl.class}, ] and use self.dicttype in your test code.
Related
I have a class which calls a lot of its methods in __init__. Since a lot is going on in these methods, I want to test them. Testing classes and class methods requires to instantiate the class and then call its methods. But if I instantiate the class, the methods will already be called before I can test it.
I have some ideas for possible solutions, but I am unsure if they are possible or a good way to go:
I could introduce a kwarg into the class like init=True and testing for it within __init__. So I could have default option to do all the magic stuff on object creation, and could deactivate it to instantiate the class and call functions separately.
I could define the methods outside the class in other classes or functions, if that works, and test them separately. The test of the bigger class would become something like an integration test.
It depends on what you would like to test
If you want to check if all the calls are happening correctly you could mock underlying functionality inside the __init__ method.
And then do assert on the mocks. (Pytest has a spy mocks which does not modify original behavior but could be tested as mocks for call count, arguments etc... I'm sure you could replicate that with unittest mock as well)
So you could mock everything that is necessary by the beginning and then create an instance.
If you want to check how it was assembled you could do this after initialization.
Generally modifying your source code just for the purpose of the test case is not a good idea.
Run your code through a debugger, check what you are looking for as a tester and automate it.
I have a bunch of python classes that are responsible for validating files like so:
java file validator
html file validator
css file validator
javascript file validator
All of these classes define a validate_file(path) and validate_string(string) function.
In the client code of my application, I store all of these validators in an array and I iterate through all of them and call the methods on them in exactly the same way.
Is there a purpose in my case of defining a base "validator" class which defines these methods as abstract? How can I benefit from making these individual classes adhere to the base class? Or maybe in other words, what features would be unlocked if I had a base class with all of these validators implementing it?
If you ever write code that needs to process mixed types, some of which might define the API method, but which shouldn't be considered "validators", having a common base class for "true" validators might be helpful for checking types ahead of time and only processing the "true" validators, not just "things that look like validators". That said, the more complex the method name, the less likely you are to end up with this sort of confusion; the odds of a different type implementing a really niche method name by coincidence are pretty small.
That said, that's a really niche use case. Often, as you can see, Python's duck-typing behavior is good enough on its own; if there is no common functionality between the different types, the only meaningful advantages to having the abstract base class are for the developer, not the program itself:
Having a common point of reference for the API consumer makes it clear what methods they can expect to have access too
For actual ABCs with #abstractmethod decorated methods, it provides a definition-time check for subclasses, making it impossible for maintainers to omit a method by accident
Base classes in Python serve to share implementation details and document likenesses. We don't need to use a common base class for things that function similarly though, as we use protocols and duck typing. For these validation functions, we might not have a use for a class at all; Java is designed to force people to put everything inside classes, but if all you have in your class is one static method, the class is just noise. Having a common superclass would enable you to check dynamically for that using isinstance, but then I'd really wonder why you're handling them in a context where you don't know what they are. For a programmer, the common word in the function name is probably enough.
A fairly small question: does anyone know about a pre-made suite of Python unit tests that just check if a class conforms to one of the standard Python data structure interfaces (e.g., lists, sets, dictionaries, queues, etc). It's not overly hard to write them, but I'd hate to bother doing so if someone has already done this. It seems like very basic functionality that someone has probably done already.
The use case is that I am using a factory pattern to create data structures due to different restrictions related to platforms. As such, I need to be able to test that the resulting created objects still conform to the standard interfaces on the surface. Also, I should note that by "conform" I mean that the tests should check not just that the interface functions exist, but also check that they work (e.g., can set and retrieve a value in a map, for instance). Python 2.7 tests would be preferred.
First, "the standard Python data structure interfaces" are not lists, sets, dictionaries, queues, etc. Those are specific implementations of the interfaces. (And queue isn't even a data structure in the sense you're thinking of—its salient features are that its operations are atomic, and put and get optionally synchronize on a Condition, and so on.)
Anyway, the interfaces are defined in five different not-quite-compatible ways.
The Built-in Types section of the documentation describes what it means to be an iterator type, a sequence type, etc. However, these are not nearly as rigorous as you'd expect for reference documentation (at least if you're used to, say, C++ or Java).
I'm not aware of any tests for such a thing, so I think you'd have to build them from scratch.
The collections module contains Collections Abstract Base Classes that define the interfaces, and provide a way to register "virtual subclasses" via the abc module. So, you can declare "I am a mapping" by inheriting from collections.Mapping, or calling collections.Mapping.register. But that doesn't actually prove that you are a mapping, just that you're claiming to be. (If you inherit from Mapping, it also acts as a mixin that helps you complete the interface by implementing, e.g., __contains__ on top of __getitem__.)
If you want to test the ABC meaning, defuz's answer is very close, and with a little more work I think he or someone else can complete it.
The CPython C API defines an Abstract Objects Layer. While this is not actually authoritative for the language, it's obviously intended that the C-API protocols and the language-level interfaces are supposed to match. And, unlike the latter, the former are rigorously defined. And of course the source code from CPython 2.7, and maybe other implementations like PyPy, may help.
There are tests for this that come with CPython, but really, they're for testing that calling PyMapping_GetItem from C properly calls your mymapping.__getitem__ in Python, which is really at a tangent to what you want to test, so I don't think it will help much.
The actual concrete classes have additional interface on top of the protocols, that you may want to test, but that's harder to describe. In particular, the way the __new__ and __init__ methods work is often important. Implementing the Mapping protocol means someone can construct an empty Foo instance and add items to it with foo[key] = value, but it doesn't mean someone can construct Foo(key=value), or Foo({key: value}) or Foo([(key, value)]).
And for this case, there are existing tests that come with all of the standard Python implementations. CPython comes with a very extensive test suite that includes things like test_dict.py. PyPy runs all the (Python-level) CPython tests, and some extra ones besides.
You will obviously have to modify these tests to run on an arbitrary class instead of one hardcoded into the tests, and you may also have to modify them to handle whichever definition you pick. Plus, they probably test more than you asked for. You just want to know if a class conforms to the protocol, not whether its methods do the right thing, right? But still, I think they're a good starting point.
Finally, the C API defines a Concrete Objects Layer that, although it's not authoritative, matches the previous definition and is more rigorously defined.
Unfortunately, the tests for this one are definitely not going to be very useful to you, because they're checking things like whether PyDict_Check and PyDict_GetItem work on your class, which they will not for any mapping defined in pure Python.
If you do build something complete for any of these definitions, I would strongly suggest putting it on PyPI, and posting about it to python-list, so you get feedback (and bug reports).
There are abstract base classes in standart module collections based on ABC module.
You have to inherit your classes from these classes to be sure that your classes correspond to the standard behavior:
import collections
class MyDict(collections.Mapping):
...
Also, your can test already existed class that does not obviously inherit the abstract class:
class MyPerfectDict(object):
... realization ...
def is_inherit(cls, abstract):
try:
class Test(abstract, cls): pass
test = Test()
except TypeError:
return False
else:
return True
is_inherit(MyPerfectDict, Mapping) # False
is_inherit(dict, Mapping) # True
In his book Java Design, Peter Coad says that one of the five criteria a subclass should meet is that the subclass "does not subclass what is merely a utility class (useful functionality you'd like to reuse)." For an example in Java, he says that making one of your domain classes a subclass of the Observable class is a violation of his rule: "Observable is a utility class, a collection of useful methods--nothing more."
In that context, here are some example test classes patterned after actual tests I've written:
class BaseDataGeneratorTestCase (unittest.TestCase):
def _test_generate_data(self, generator, expected_value):
# Imagine there's a lot more code here, making it
# worthwhile to factor this method out.
assert generator.generate_data() == expected_value
class DataGeneratorTests (BaseDataGeneratorTestCase):
def test_generate_data(self):
self._test_generate_data(DataGenerator(), "data")
class VariantDataGeneratorTests (BaseDataGeneratorTestCase):
def test_generator_data(self):
self._test_generate_data(VariantDataGenerator(),
"different data")
Though this example is trivial, consider that the real tests and their surrounding system are, of course, much more complex. I think this example is usable as a vehicle to try and clear up some of my confusion about the proper use of inheritance.
Is subclassing BaseDataGeneratorTestCase a bad idea? Does it qualify as just "useful functionality [I'd] like to reuse"? Should _test_generate_data just be a function, not in any class?
Basically you can solve the problem any way you like. OOP is just a paradigm that's supposed to help you design things in a clearer and maintainable way.
So, the question you should ask yourself any time you want to use inheritance they are have they same role, and you just want to add functionality ? or do you want to just use the nice methods in the "base" class ?
It's just a matter of concept, and in your little example it could go either way because they have the same concept, but basically, you're just using the base class as a utility.
If you want the base class to actually be a base class you should design it so every inherited class will behave the similarly.
I do agree that conceptually it would make sense to have a unit test baseclass for all the tests that work similarly.
I have one python module that defines a set of base classes which another python module then implements.
The core python module also has test cases that test if the base classes are correctly implemented.
In the second module's test suite, I would like to automatically generate the TestCase subclasses from the first module, and adding a mixin that does the necessary setUp() for the specific implementation.
I see lots of answers for parametrizing test cases, but in my case I just want to re-use the common classes and subclass all of them.
It's ok if it needs some code in the parent test module to instantiate them, as long as it names the tests differently and runs them all automatically.
If it matters, this code is using twisted and trial over standard unittest.
Something that may do almost exactly what you want is twisted.internet.test.reactormixins.ReactorBuilder. You can see how it's used in test_tcp, for example.