I want to be able to get the result of a particular test method and output it inside the teardown method, while using the nose test runner.
There is a very good example here.
But unfortunately, running nosetests example.py does not work, since nose doesn't seem to like the fact that the run method in the superclass is being overridden:
AttributeError: 'ResultProxy' object has no attribute 'wasSuccessful'
Caveat: the following doesn't actually access the test during the tearDown, but it does access each result.
You might want to write a nose plugin (see the API documentation here). The method that you are probably interested in is afterTest(), which is run... after the test. :) Though, depending on your exact application, handleError()/handleFailure() or finalize() might actually be more useful.
Here is an example plugin that accesses the result of a test immediately after it is executed.
from nose.plugins import Plugin
import logging
log = logging.getLogger('nose.plugins.testnamer')
class ReportResults(Plugin):
def __init__(self, *args, **kwargs):
super(ReportResults, self).__init__(*args, **kwargs)
self.passes = 0
self.failures = 0
def afterTest(self, test):
if test.passed:
self.passes += 1
else:
self.failures += 1
def finalize(self, result):
print "%d successes, %d failures" % (self.passes, self.failures)
This trivial example merely reports the number of passes and failures (like the link you included, but I'm sure you can extend it to do something more interesting (here's another fun idea). To use this, make sure that it is installed in Nose (or load it into a custom runner), and then activate it with --with-reportresults.
If you are OK with adding some boilerplate code to the tests, something like the following might work.
In MyTest1, tearDown is called at the end of each test, and the value of self.result has been set to a tuple containing the method name and a dictionary (but you could set that to whatever you like). The inspect module is used to get the method name, so tearDown knows which test just ran.
In MyTest2, all the results are saved in a dictionary (results), which you can do with what you like in the tearDownClass method.
import inspect
import unittest
class MyTest1(unittest.TestCase):
result = None
def tearDown(self):
print "tearDown:", self.result
def test_aaa(self):
frame = inspect.currentframe()
name = inspect.getframeinfo(frame).function
del frame
self.result = (name, None)
x = 1 + 1
self.assertEqual(x, 2)
self.result = (name, dict(x=x))
def test_bbb(self):
frame = inspect.currentframe()
name = inspect.getframeinfo(frame).function
del frame
self.result = (name, None)
# Intentional fail.
x = -1
self.assertEqual(x, 0)
self.result = (name, dict(x=x))
class MyTest2(unittest.TestCase):
results = {}
#classmethod
def tearDownClass(cls):
print "tearDownClass:", cls.results
def test_aaa(self):
frame = inspect.currentframe()
name = inspect.getframeinfo(frame).function
del frame
self.results[name] = None
x = 1 + 1
self.assertEqual(x, 2)
self.results[name] = dict(x=x)
def test_bbb(self):
frame = inspect.currentframe()
name = inspect.getframeinfo(frame).function
del frame
self.results[name] = None
x = -1
self.assertEqual(x, 0)
self.results[name] = dict(x=x)
if __name__ == '__main__':
unittest.main()
Related
I had created a simple example to illustrate my issue. First is the setup say mydummy.py:
class TstObj:
def __init__(self, name):
self.name = name
def search(self):
return self.name
MyData = {}
MyData["object1"] = TstObj("object1")
MyData["object2"] = TstObj("object2")
MyData["object3"] = TstObj("object3")
def getObject1Data():
return MyData["object1"].search()
def getObject2Data():
return MyData["object2"].search()
def getObject3Data():
return MyData["object3"].search()
def getExample():
res = f"{getObject1Data()}{getObject2Data()}{getObject3Data()}"
return res
Here is the test that failed.
def test_get_dummy1():
dummy.MyData = MagicMock()
mydummy.MyData["object1"].search.side_effect = ["obj1"]
mydummy.MyData["object2"].search.side_effect = ["obj2"]
mydummy.MyData["object3"].search.side_effect = ["obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
The above failed with run time error:
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py:1078: StopIteration
Here is the test that passed:
def test_get_dummy2():
dummy.MyData = MagicMock()
mydummy.MyData["object1"].search.side_effect = ["obj1", "obj2", "obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
Am I missing something? I would have expected test_get_dummy1() to work and test_get_dummy2() to fail and not vice versa. Where and how can I find/learn more information about mocking to explain what is going on...
MyData["object1"] is converted to this function call: MyData.__getitem__("object1"). When you call your getExample method, the __getitem__ method is called 3 times with 3 parameters ("object1", "object2", "object3").
To mock the behavior you could have written your test like so:
def test_get_dummy_alternative():
mydummy.MyData = MagicMock()
mydummy.MyData.__getitem__.return_value.search.side_effect = ["obj1", "obj2", "obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
Note the small change from your version: mydummy.MyData["object1"]... became: mydummy.MyData.__getitem__.return_value.... This is the regular MagicMock syntax - we want to to change the return value of the __getitem__ method.
BONUS:
I often struggle with mock syntax and understanding what's happening under the hood. This is why I wrote a helper library: the pytest-mock-generator. It can show you the actual calls made to the mock object.
To use it in your case you could have added this "exploration test":
def test_get_dummy_explore(mg):
mydummy.MyData = MagicMock()
mydummy.getExample()
mg.generate_asserts(mydummy.MyData, name='mydummy.MyData')
When you execute this test, the following output is printed to the console, which contains all the asserts to the actual calls to the mock:
from mock import call
mydummy.MyData.__getitem__.assert_has_calls(calls=[call('object1'),call('object2'),call('object3'),])
mydummy.MyData.__getitem__.return_value.search.assert_has_calls(calls=[call(),call(),call(),])
mydummy.MyData.__getitem__.return_value.search.return_value.__str__.assert_has_calls(calls=[call(),call(),call(),])
You can easily derive from here what has to be mocked.
I'm having some issue while creating unittest for internal parameter.
My structure is:
[1] my_animal.py contains Myclass and method: do_bite()
my_animal.py
class Myclass():
def do_bite(self):
return 1
[2] my_module.py contains jobMain("") which is using the method from my_animal.py
my_module.py
import sys
from someclass import Myclass
def jobMain(directoryPath):
flag = -1
result = Myclass()
if result.do_bite() is None:
flag = 0
if result.do_bite() is 1:
flag = 1
if result.do_bite() is 2:
flag = 2
[3] my_test.py contains the unittest to test jobMain in my_module.py
my_test.py
# Mock Myclass.dobite to None
#pytest.fixture
def mock_dobite0():
with mock.patch('my_module.Myclass') as mocked_animal:
mocked_animal.return_value.do_bite.return_value = None
yield
# Mock Myclass.dobite to 1
#pytest.fixture
def mock_dobite1():
with mock.patch('my_module.Myclass') as mocked_animal:
mocked_animal.return_value.do_bite.return_value = 1
yield
# Mock Myclass.dobite to 2
#pytest.fixture
def mock_dobite2():
with mock.patch('my_module.Myclass') as mocked_animal:
mocked_animal.return_value.do_bite.return_value = 2
yield
# My unittest to test dobite() method
def test_dobite0(mock_Myclass, mock_dobite0):
jobMain("")
def test_dobite1(mock_Myclass, mock_dobite1):
jobMain("")
def test_dobite2(mock_Myclass, mock_dobite2):
jobMain("")
My question is: How to test 'flag' parameter inside JobMain?
'flag' para must be assigned the correct value.( eg: dobite = 1 => flag = 1)
The variable para only exists in the scope of jobMain. If you want to use the variable outside jobMain the most common ways are
1) return the value
This is quite obvious. Since jobMain is a function, it returns a value. Without an explicit return statement you return None. You could just
def jobmain(pth):
# do stuff and assign flag
return flag
# and inside tests
assert jobmain("") == 1
2) Use a class instead
If you want the jobMain to remember some state, then it is common practice to use objects. Then flag would be attribute of the object and could be accessed from outside, after you call any method (function) of JobMain. For example
class JobMain:
def __init__(self):
self.flag = -1
def run(self, pth):
result = Myclass()
if result.do_bite() is None:
self.flag = 0
if result.do_bite() is 1:
self.flag = 1
if result.do_bite() is 2:
self.flag = 2
# and inside test
job = JobMain()
job.run()
assert job.flag == 1
Note
I just copy-pasted your code for setting the flag. Note that you call do_bite() many times, if the resulting value is None or 1. Also, when testing against a number, one should use == instead of is.
How to test 'flag' parameter inside JobMain?
You don't. It's an internal variable. Testing it would be glass-box testing; the test will break if the implementation changes.
Instead, test the effect of flag. This is black-box testing. Only the interface is tested. If the implementation changes the test still works allowing the code to be aggressively refactored.
Note: If you don't hard code result = Myclass() you don't need to mock. Pass it in as an argument with the default being Myclass().
def jobMain(directoryPath, result=Myclass()):
Then you don't need to patch Myclass(). Instead, pass in a mock object.
# I don't know unittest.mock very well, but something like this.
mock = Mock(Myclass)
mock.do_bite.return_value = 2
jobMain('', result=mock)
This also makes the code more flexible outside of testing.
After:
import unittest
loader = unittest.TestLoader()
tests = loader.discover('.')
testRunner = unittest.runner.TextTestRunner()
testResult = testRunner.run(tests)
I can get a list failures names and messages with:
for t in testResult.failures:
print t[0].id()
print t[1]
How to do the same for the successes?
I expect to be able to do it by overriding something in TextTestRunner. What is the easiest way?
Tested on Python 2.7.
elethan put me on the right direction: https://stackoverflow.com/a/40613695/895245
This is what I got after using TextTestRunner(resultclass= and adding a nice constructor:
import unittest
from unittest.runner import TextTestResult
class TextTestResultWithSuccesses(TextTestResult):
def __init__(self, *args, **kwargs):
super(TextTestResultWithSuccesses, self).__init__(*args, **kwargs)
self.successes = []
def addSuccess(self, test):
super(TextTestResultWithSuccesses, self).addSuccess(test)
self.successes.append(test)
loader = unittest.TestLoader()
tests = loader.discover('.')
testRunner = unittest.runner.TextTestRunner(resultclass=TextTestResultWithSuccesses)
testResult = testRunner.run(tests)
for t in testResult.failures:
print t[0].id()
print t[1]
print
for t in testResult.errors:
print t[0].id()
print t[1]
print
for t in testResult.successes:
print t.id()
I have done something similar in the past. The way I would do this is by first creating a test result class that in inherits from TestTextResult with a custom addSuccess method which will be called on each test success (addFailure is the method where the failing tests get appended to failures):
from unittest.runner import TextTestResult, TextTestRunner
class MyTestResult(TextTestResult):
successes = [] # As OP points out, this makes more sense as instance attribute
def addSuccess(self, test):
super(TextTestResult, self).addSuccess(test)
self.successes.append(test) # add other info if you want
and then make your own custom test runner, adding your custom test result as resultclass
class MyTestRunner(TextTestRunner):
resultclass = MyTestResult
# If this is your only customization, you can just pass
# the test runner class to `TextTestRunner`; see OP's answer
I am trying to introduce python 3 support for the package mime and the code is doing something I have never seen before.
There is a class Types() that is used in the package as a static class.
class Types(with_metaclass(ItemMeta, object)): # I changed this for 2-3 compatibility
type_variants = defaultdict(list)
extension_index = defaultdict(list)
# __metaclass__ = ItemMeta # unnessecary now
def __init__(self, data_version=None):
self.data_version = data_version
The type_variants defaultdict is what is getting filled in python 2 but not in 3.
It very much seems to be getting filled by this class when is in a different file called mime_types.py.
class MIMETypes(object):
_types = Types(VERSION)
def __repr__(self):
return '<MIMETypes version:%s>' % VERSION
#classmethod
def load_from_file(cls, type_file):
data = open(type_file).read()
data = data.split('\n')
mime_types = Types()
for index, line in enumerate(data):
item = line.strip()
if not item:
continue
try:
ret = TEXT_FORMAT_RE.match(item).groups()
except Exception as e:
__parsing_error(type_file, index, line, e)
(unregistered, obsolete, platform, mediatype, subtype, extensions,
encoding, urls, docs, comment) = ret
if mediatype is None:
if comment is None:
__parsing_error(type_file, index, line, RuntimeError)
continue
extensions = extensions and extensions.split(',') or []
urls = urls and urls.split(',') or []
mime_type = Type('%s/%s' % (mediatype, subtype))
mime_type.extensions = extensions
...
mime_type.url = urls
mime_types.add(mime_type) # instance of Type() is being filled?
return mime_types
The function startup() is being run whenever mime_types.py is imported and it does this.
def startup():
global STARTUP
if STARTUP:
type_files = glob(join(DIR, 'types', '*'))
type_files.sort()
for type_file in type_files:
MIMETypes.load_from_file(type_file) # class method is filling Types?
STARTUP = False
This all seems pretty weird to me. The MIMETypes class first creates an instance of Types() on the first line. _types = Types(VERSION). It then seems to do nothing with this instance and only use the mime_types instance created in the load_from_file() class method. mime_types = Types().
This sort of thing vaguely reminds me of javascript class construction. How is the instance mime_types filling Types.type_variants so that when it is imported like this.
from mime import Type, Types
The class's type_variants defaultdict can be used. And why isn't this working in python 3?
EDIT:
Adding extra code to show how type_variants is filled
(In "Types" Class)
#classmethod
def add_type_variant(cls, mime_type):
cls.type_veriants[mime_type.simplified].append(mime_type)
#classmethod
def add(cls, *types):
for mime_type in types:
if isinstance(mime_type, Types):
cls.add(*mime_type.defined_types())
else:
mts = cls.type_veriants.get(mime_type.simplified)
if mts and mime_type in mts:
Warning('Type %s already registered as a variant of %s.',
mime_type, mime_type.simplified)
cls.add_type_variant(mime_type)
cls.index_extensions(mime_type)
You can see that MIMETypes uses the add() classmethod.
Without posting more of your code, it's hard to say. I will say that I was able to get that package ported to Python 3 with only a few changes (print statement -> function, basestring -> str, adding a dot before same-package imports, and a really ugly hack to compensate for their love of cmp:
def cmp(x,y):
if isinstance(x, Type): return x.__cmp__(y)
if isinstance(y, Type): return y.__cmp__(x) * -1
return 0 if x == y else (1 if x > y else -1)
Note, I'm not even sure this is correct.
Then
import mime
print(mime.Types.type_veriants) # sic
printed out a 1590 entry defaultdict.
Regarding your question about MIMETypes._types not being used, I agree, it's not.
Regarding your question about how the dictionary is being populated, it's quite simple, and you've identified most of it.
import mime
Imports the package's __init__.py which contains the line:
from .mime_types import MIMETypes, VERSION
And mime_types.py includes the lines:
def startup():
global STARTUP
if STARTUP:
type_files = glob(join(DIR, 'types', '*'))
type_files.sort()
for type_file in type_files:
MIMETypes.load_from_file(type_file)
STARTUP = False
startup()
And MIMETypes.load_from_file() has the lines:
mime_types = Types()
#...
for ... in ...:
mime_types.add(mime_type)
And Types.add(): has the line:
cls.add_type_variant(mime_type)
And that classmethod contains:
cls.type_veriants[mime_type.simplified].append(mime_type)
So I've written a module that contains a bunch of functions to easily interact with a subprocess. This subprocess has a whole bunch of settings that let you change how it formats and behaves. I realized that it'd be nice to have a convenience class that you could use as a handler to store the settings you prefer to use and pass them on to the module level functions. here's the example code I'm doing testing with:
import inspect
class MyHandler(object):
def __init__(self):
self.format_string='class format string'
self.database='class database'
self.mode = "class mode"
def rename(self, *args, **kwargs):
self._pass_to_function(rename, *args, **kwargs)
def _pass_to_function(self, function, *overrided_args, **overrided_kwargs):
# get the function's remaining arguments with the inspect module
functon_kwargs = inspect.getargspec(function)[0][len(overrided_args):]
handler_vars = vars(self)
kwargs_to_pass = {}
for arg in functon_kwargs:
if arg in handler_vars:
kwargs_to_pass[arg] = handler_vars[arg]
for arg in overrided_kwargs:
kwargs_to_pass[arg] = overrided_kwargs[arg]
return function(*overrided_args, **kwargs_to_pass)
def rename(targets, format_string=None, database=None, mode=None,
not_in_class='None'):
print 'targets = {}'.format(targets)
print 'format_string = {}'.format(format_string)
print 'database = {}'.format(database)
print 'mode = {}'.format(mode)
print 'not_in_class = {}\n'.format(not_in_class)
return
The thing I like about this solution is that it uses the attributes stored in the class, but you can easily override them by simply adding them to the method call if you want a one-off with a different setting. To do this I have the _pass_to_function as a kind of wrapper function to parse and fill in the needed settings and overrides. Here's how it looks:
>>> import argstest
>>> argstest.rename('some_file.avi', database='some database')
targets = some_file.avi
format_string = None
database = some database
mode = None
not_in_class = None
>>> tst = argstest.MyHandler()
>>> tst.rename('some_file.avi')
targets = some_file.avi
format_string = class format string
database = class database
mode = class mode
not_in_class = None
>>> tst.rename('some_file.avi', 'one off format string', not_in_class=True)
targets = some_file.avi
format_string = one off format string
database = class database
mode = class mode
not_in_class = True
Now in my real module I have dozens of module-level functions that I want to access from the handler class. Ideally they would generate automatically based on the functions in the module. Seeing as how all the methods are only going to be passing everything to _pass_to_function I get the sense that this shouldn't be very difficult but I'm having a lot of trouble figuring out exactly how.
I've read about using type to generate a meta-class, but I don't see how I would use it in this situation. Am I not seeing how I could use type? Should I use some sort of module level script that adds the functions with setattr? Is what I was doing the better/clearer way to do things?
Any and all advice would be appreciated.
Okay, I think I've answered my own question for now. This is how the module looks:
import inspect
import sys
from types import MethodType
class MyHandler(object):
def __init__(self):
self.format_string = 'class format string'
self.database = 'class database'
self.mode = "class mode"
self._populate_methods()
def _populate_methods(self):
to_add = inspect.getmembers(sys.modules[__name__], inspect.isfunction)
to_add = [x[0] for x in to_add if not x[0].startswith('_')]
for func_name in to_add:
func = getattr(sys.modules[__name__], func_name) # strings to functions
self._add_function_as_method(func_name, func)
def _add_function_as_method(self, func_name, func):
def f(self, *args, **kwargs): # the template for the method we'll add
return self._pass_to_function(func, *args, **kwargs)
setattr(MyHandler, func_name, MethodType(f, None, MyHandler))
def _pass_to_function(self, function, *overrided_args, **overrided_kwargs):
functon_kwargs = inspect.getargspec(function)[0][len(overrided_args):]
handler_vars = vars(self)
kwargs_to_pass = {}
for arg in functon_kwargs:
if arg in handler_vars:
kwargs_to_pass[arg] = handler_vars[arg]
for arg in overrided_kwargs:
kwargs_to_pass[arg] = overrided_kwargs[arg]
return function(*overrided_args, **kwargs_to_pass)
def rename(targets, format_string=None, database=None, mode=None,
not_in_class='None'):
print 'targets = {}'.format(targets)
print 'format_string = {}'.format(format_string)
print 'database = {}'.format(database)
print 'mode = {}'.format(mode)
print 'not_in_class = {}\n'.format(not_in_class)
return
def something_else():
print "this function should become a method"
def _not_a_member():
print "this function should not become a method"
I've added the _populate_methods and the _add_function_as_method member functions. the _populate_methods function gets the name of all "public" functions in the module, de-references them to their function and passes each one though _add_function_as_method. All this method does is use an internal function to capture arguments and sent them to _pass_to_function, and set that function as a method using setattr.
phew
so it works, but I'm still wondering if there isn't a clearer or more straight forward way to get this done. I'd be very grateful if anyone could chime in.