Python3 test cases (doctests) are failing with my sample code. But the same is working fine in Python2.
test.py
class Test(object):
def __init__(self, a=0):
self.a = a
def __getattr__(self, attr):
return Test(a=str(self.a) + attr)
tst.py
from test import Test
t = Test()
Run test cases: python3 -m doctest -v tst.py
Error:
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.6/doctest.py", line 2787, in <module>
sys.exit(_test())
File "/usr/lib/python3.6/doctest.py", line 2777, in _test
failures, _ = testmod(m, verbose=verbose, optionflags=options)
File "/usr/lib/python3.6/doctest.py", line 1950, in testmod
for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
File "/usr/lib/python3.6/doctest.py", line 933, in find
self._find(tests, obj, name, module, source_lines, globs, {})
File "/usr/lib/python3.6/doctest.py", line 992, in _find
if ((inspect.isroutine(inspect.unwrap(val))
File "/usr/lib/python3.6/inspect.py", line 513, in unwrap
raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
ValueError: wrapper loop when unwrapping <test.Test object at 0x7f6e80028550>
Can anyone please help in how to overcome this error.
Thanks.
For unittest.mock try importing items as
from unittest import mock
instead of
from unittest.mock import patch
This worked around the bug for me.
This is arguably a bug in doctest. What's happening is that doctest is searching for functions/methods/callables with a docstring, and while doing so it's unwrapping any decorators it finds. Why it does this, I have no idea. But anyway, doctest ends up calling inspect.unwrap(t) (where t is a Test instance), which is essentially equivalent to doing this:
while True:
try:
t = t.__wrapped__
except AttributeError:
break
Because t is a Test instance, accessing t.__wrapped__ calls __getattr__ and returns a new Test instance. This would go on forever, but inspect.unwrap is smart enough to notice that it's not getting anywhere, and throws an exception instead of entering an infinite loop.
As a workaround, you can rewrite your __getattr__ method to throw an AttributeError when __wrapped__ is accessed. Even better, throw an AttributeError when any dunder-attribute is accessed:
def __getattr__(self, attr):
if attr.startswith('__') and attr.endswith('__'):
raise AttributeError
return Test(a=str(self.a) + attr)
Related
I got an error for this code:
from pathos.multiprocessing import ProcessingPool
def diePlz(im):
print('Whoopdepoop!')
def caller():
im = 1
pool = ProcessingPool()
pool.map(diePlz,[im,im,im,im])
if __name__=='__main__':
caller()
when I ran it with the cProfiler: (python3 -m cProfile testProfiler.py)
multiprocess.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/rohit/.local/lib/python3.6/site-packages/pathos/helpers/mp_helper.py", line 15, in <lambda>
func = lambda args: f(*args)
File "testProfiler.py", line 3, in diePlz
print('Whoopdepoop!')
NameError: name 'print' is not defined
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.6/cProfile.py", line 160, in <module>
main()
File "/usr/lib/python3.6/cProfile.py", line 153, in main
runctx(code, globs, None, options.outfile, options.sort)
File "/usr/lib/python3.6/cProfile.py", line 20, in runctx
filename, sort)
File "/usr/lib/python3.6/profile.py", line 64, in runctx
prof.runctx(statement, globals, locals)
File "/usr/lib/python3.6/cProfile.py", line 100, in runctx
exec(cmd, globals, locals)
File "testProfiler.py", line 11, in <module>
caller()
File "testProfiler.py", line 8, in caller
pool.map(diePlz,[im,im,im,im])
File "/home/rohit/.local/lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map
return _pool.map(star(f), zip(*args)) # chunksize
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get
raise self._value
NameError: name 'print' is not defined
But when I ran it without the cProfiler:
$ python3 testProfiler.py
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!
The code that I've provided is a minimal working example for the problem. There is a much larger code that I want to debug, but am not able to do so because cProfiler keeps raising weird errors.
In this case, the point of importance is
NameError: name 'print' is not defined
which means python3 is not able to recognize print itself. In my code, it was not able to recognize range.
So, I realize this is a long time after the original post, but I have this exact same issue.
In my case I was getting the exact same error as the original post - python builtin functions such as print() or len() resulted in errors like this:
NameError: name 'len' is not defined
I'm currently running multiprocess version 0.70.11.1 and dill version 0.3.3 (components of pathos that make process based parallelism work).
Based on what I found in an issue comment: https://github.com/uqfoundation/pathos/issues/129#issuecomment-536081859 one of the package authors recommends trying:
import dill
dill.settings['recurse'] = True
At least in my case, the above fixed the error!
I am struggling trying to understand how to write tests in Flask.
I've inherited an app that already has a bunch of tests that hit routes like /login and test that the response is what's expected.
I have a substantially more complicated situation. I need to test a route/method that depending on the circumstances, hits an external api, figures out whether a path exists in the container the app itself is running in, starts a process that takes 10+ minutes to run on a different machine -- all manner of things. So I can't just hit the route and see if I got what I wanted; I need mocking and patching to mimic the effects of various external world states.
Right now I have a route defined like so in brain_db/views.py:
#app.route('/label_view/<int:scan_number>')
#login_required
def label_view(scan_number):
<so much complicated logic>
The first route defined in that same file, brain_db/views.py, is
#app.route('/surface_test')
def surface_test():
<some code>
Here is a simplified version of the file that's throwing the error:
import unittest
from mock import MagicMock, patch
from flask_brain_db.test_helpers import set_up, tear_down
from flask_brain_db.brain_db.models import Scan
from brain_db.views import label_view
class BrainDBTest(unittest.TestCase):
def setUp(self):
app, db = set_up()
scan = Scan(1, '000001', '000001_MR1', 'scan.nii.gz', scan_number=1)
db.session.add(scan)
scan = Scan.query.filter(Scan.scan_number == 1).first()
db.session.commit()
def tearDown(self):
tear_down()
def mock_volume_views_setup(self)
scan = Scan.query.filter(Scan.scan_number == 1).first()
container_file_path = '/path/to/file/in/container'
return scan, container_file_path
def mock_os_path_exists(self, arg):
return True
#patch('brain_db_helpers.volume_views_setup', mock_volume_views_setup)
#patch('os.path.exists', mock_os_path_exists)
def test_label_view(self):
rv = label_view(1)
assert(True) # I'll actually write tests when I figure out that I can!
print rv
Here is the error:
======================================================================
ERROR: brain_db.tests.test (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: brain_db.tests.test
Traceback (most recent call last):
File "/usr/local/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/local/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File "/usr/src/app/flask_brain_db/brain_db/tests/test.py", line 7, in <module>
from brain_db.views import label_view
File "/usr/src/app/flask_brain_db/brain_db/views.py", line 36, in <module>
#app.route('/surface_test')
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1250, in decorator
self.add_url_rule(rule, endpoint, f, **options)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 66, in wrapper_func
return f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1221, in add_url_rule
'existing endpoint function: %s' % endpoint)
AssertionError: View function mapping is overwriting an existing endpoint function: surface_test
What I have done to try to solve my problem: I've read a bunch of the posts on SO that quote that same AssertionError. E.g. 1, 2. I can see that the general shape of the problem is that my routes have already been defined, and
from brain_db.views import label_view
is executing the views module again, thus redefining the routes, so I get an error thrown.
What I don't understand is how exactly I should avoid this. I need to be able to import a method into another file to be able to test it. Are all the routes supposed to be wrapped in if __name__ == main? I am brand new to Flask development and haven't yet seen example code where this is the case; I'm dubious that this is the correct solution; it's just the only thing that's offered when you try to search for preventing code from being executed on import.
The way I'm running my tests right now is via the file manage.py in the top level of my application. It contains the following method:
#manager.command
def test():
"""Runs the tests without coverage"""
tests = unittest.TestLoader().discover(start_dir='.', pattern='test*.py')
res = unittest.TextTestRunner(verbosity=2).run(tests)
sys.exit(not res.wasSuccessful())
I run python manage.py test at the command line.
It also might be relevant that while I've put the test that's failing in a submodule within brain_db, several tests run before it that hit routes defined in the app and test for the expected result. However, commenting out those tests has no effect on the way my test is failing.
Finally, I was initially getting an error at the line from flask_brain_db.brain_db.models import Scan:
ERROR: brain_db.tests.test (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: brain_db.tests.test
Traceback (most recent call last):
File "/usr/local/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/local/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File "/usr/src/app/flask_brain_db/brain_db/tests/test.py", line 5, in <module>
from flask_brain_db.brain_db.models import Scan
File "/usr/src/app/flask_brain_db/brain_db/models.py", line 6, in <module>
class Scan(db.Model):
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 67, in __init__
super(NameMetaMixin, cls).__init__(name, bases, d)
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 121, in __init__
super(BindMetaMixin, cls).__init__(name, bases, d)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py", line 65, in __init__
_as_declarative(cls, classname, cls.__dict__)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 116, in _as_declarative
_MapperConfig.setup_mapping(cls, classname, dict_)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 144, in setup_mapping
cfg_cls(cls_, classname, dict_)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 172, in __init__
self._setup_table()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 465, in _setup_table
**table_kw)
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 90, in __table_cls__
return sa.Table(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 439, in __new__
"existing Table object." % key)
InvalidRequestError: Table 'scan' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
I made it go away by including
__table_args__ = {'extend_existing': True}
In the model definition, but I don't know if I should have done that and I suspect I was just postponing the same problem I have now. It seems like the fundamental problem is that I don't know how to write tests without redefining a bunch of things that have been already been defined.
What is the correct way to approach this? Please let me know if I need to provide any other information.
You should be able to access your all view functions from your app object. Try removing the line "from brain_db.views import label_view", and instead define your label_view method after running set_up() using the below:
label_view = app.view_functions["label_view"]
For some reason I can't run ddt.py. Could you explain what is the problem? In official documentation it looks like this.
#ddt
class TestPages(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome()
self.driver.set_page_load_timeout(10)
self.driver.maximize_window()
#file_data('client.json')
def test_sort_by_price(self, value):
print("\n" + str(test_cases(2)))
gallerypage = GalleryPage(self.driver, value)
result = gallerypage.sort_by_price()
self.assertTrue(result)
ERROR:
Error
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
yield
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 605, in run
testMethod()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 34, in testFailure
raise self._exception
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 168, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: type object 'TestPages' has no attribute 'test_sort_by_price'
The problem is that you run separate test, not the test class itself. Try to run test class, it worked for me.
first, rename ddt.py to test_dtt.py to avoid confusion of lib
second, find . -name "*dtt*.py*", rename them properly
last, find and remove any *.pyc files and rerun the tests
for file in `find . -name "*.pyc"`; do rm -vf $file; done
I want to test a custom child class of http.server.CGIHTTPRequestHandler. As the init method and all other methods of CGIHTTPRequestHandler do a lot of irrelevant stuff for my tests I want to mock it. But I don't get how to achieve that by decorating the setUp-method of my tests with #patch.
I don't get howto import/decorate a patched class correctly, so I created a minimal example:
/src/samplepackage/
tests/
__init__.py
sutclass_tests.py
__init__.py
parentclass.py
sutclass.py
parentclass.py:
class ParentClass():
def __init__(self, a, b, c):
print("__init__ doing much to much stuff with " + str(a) + " " + str(b) + " " + str(c))
sutclass.py:
from samplepackage.parentclass import ParentClass
class SUTClass (ParentClass):
def __init__(self, a, b, c, d):
super().__init__(a, b, c)
self.sutAttribute = d
def SUTmethod(self):
self.sutAttribute = self.sutAttribute * 2
return self.sutAttribute
sutclass_tests.py:
import unittest
from samplepackage.sutclass import SUTClass
from unittest.mock import patch
class SUTClassTests(unittest.TestCase):
#patch("samplepackage.sutclass.SUTClass.Parent.__init__")
def setUp(self):
unittest.TestCase.setUp(self)
self.sutClass = SUTClass(1,2,3,4)
def test_firstIteration(self):
self.assertEqual(self.sutClass.SUTmethod(), 8)
if __name__ == '__main__':
unittest.main()
Executing the sutclass_tests.py results in:
============================= ERRORS =============================
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1149, in patched
arg = patching.__enter__()
File "/usr/lib/python3.5/unittest/mock.py", line 1205, in __enter__
self.target = self.getter()
File "/usr/lib/python3.5/unittest/mock.py", line 1375, in <lambda>
getter = lambda: _importer(target)
File "/usr/lib/python3.5/unittest/mock.py", line 1062, in _importer
thing = _dot_lookup(thing, comp, import_path)
File "/usr/lib/python3.5/unittest/mock.py", line 1051, in _dot_lookup
__import__(import_path)
ImportError: No module named 'samplepackage.sutclass.SUTClass'; 'samplepackage.sutclass' is not a package
I get that 'samplepackage.sutclass' is not a package (didn't intent to use it as a package). Changing the #patch-decorator to
#patch("samplepackage.SUTClass.Parent.__init__")
yields:
============================= ERRORS =============================
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1049, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'samplepackage' has no attribute 'SUTClass'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1149, in patched
arg = patching.__enter__()
File "/usr/lib/python3.5/unittest/mock.py", line 1205, in __enter__
self.target = self.getter()
File "/usr/lib/python3.5/unittest/mock.py", line 1375, in <lambda>
getter = lambda: _importer(target)
File "/usr/lib/python3.5/unittest/mock.py", line 1062, in _importer
thing = _dot_lookup(thing, comp, import_path)
File "/usr/lib/python3.5/unittest/mock.py", line 1051, in _dot_lookup
__import__(import_path)
ImportError: No module named 'samplepackage.SUTClass'
So my question is howto patch the parent class (ParentClass) of my SUTClass in the setUp-method (as I want to reuse it in muliple testcases) correctly?
I found How to mock a base class with python mock library which solves my issue. Though not by decoration but by mock.patch.object().
I changed sutclass_tests.py to:
import unittest
from samplepackage.sutclass import SUTClass
from unittest.mock import patch
from unittest import mock
class SUTClassTests(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
patcher = patch.object(SUTClass, '__bases__', (mock.Mock,))
with patcher:
patcher.is_local = True
self.sut = SUTClass(1,2,3,4)
def test_firstIteration(self):
print(self.sut.SUTmethod())
if __name__ == '__main__':
unittest.main()
This solves my problem (no import errors) and shows the correct behaviour when testing my real scenario with my class derived from http.server.CGIHTTPRequestHandler
I use below method to generate cases automatically in python unittest.
import unittest
class Tests(unittest.TestCase):
def check(self, i):
self.assertNotEquals(0, i)
for i in [0, 1, 2]:
def ch(i):
return lambda self: self.check(i)
setattr(Tests, "test_%d" % i, ch(i))
if __name__ == "__main__":
unittest.main()
It works well when "python test.py" to run all cases together.
But fails to run a specific case, like "python test.py Tests.test_0"
Traceback (most recent call last):
File "test.py", line 12, in <module>
unittest.main()
File "/usr/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/usr/lib/python2.7/unittest/main.py", line 149, in parseArgs
self.createTests()
File "/usr/lib/python2.7/unittest/main.py", line 158, in createTests
self.module)
File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python2.7/unittest/loader.py", line 109, in loadTestsFromName
return self.suiteClass([parent(obj.__name__)])
File "/usr/lib/python2.7/unittest/case.py", line 191, in __init__
(self.__class__, methodName))
ValueError: no such test method in <class '__main__.Tests'>: <lambda>
Anyone can help?
It appears (from the error message) and from perusing the source of unittest, that the test runner gets a reference to the method and then uses the method's __name__ attribute to figure out which method to run. This is a bit silly (after all, we already picked up a reference to the method!), but it does have some benefits in terms of code simplicity. The quick fix is to make sure that you set the name on your lambda functions as well:
import unittest
class Tests(unittest.TestCase):
def check(self, i):
self.assertNotEquals(0, i)
for i in [0, 1, 2]:
def ch(i):
return lambda self: self.check(i)
f = ch(i)
name = "test_%d" % i
f.__name__ = name
setattr(Tests, name, f)
if __name__ == "__main__":
unittest.main()
Now everything works as it should:
mgilson$ python test.py Tests.test_0
F
======================================================================
FAIL: test_0 (__main__.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test.py", line 13, in <lambda>
return lambda self: self.check(i)
File "test.py", line 5, in check
self.assertNotEquals(0, i)
AssertionError: 0 == 0
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
and:
mgilson$ python test.py Tests.test_1
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
Note that it appears this this was a bug which was fixed. In the current HEAD, we can see that the name is parsed from the string as opposed to in the version which I have (and apparently you do to) where the name is picked up from the object (function) to be run.