I've moved recently from Python 2.7 to Python 3.8. There's a strange new phenomenon when running the tests, that can be reproduced with this simple example:
from django.test import TestCase
from users.models import User
class TestWTF(TestCase):
def setUp(self):
self.user = User.objects.create(email='admin#project.com')
def test_failure(self):
self.assertTrue(False)
def test_success(self):
pass
When test_failure() fails the self.user object doesn't get removed from the DB. It seems like the promised rollback feature just doesn't happen. test_success() and all subsequent tests in the same class will fail with UNIQUE constraint being violated when setUp() tries to create the object again.
It doesn't happen in Python 2.
A partial output from pytest I'm using:
$ pytest -W ignore -s
================================================= test session starts ==================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
django: settings: project_sites.settings.tests (from ini)
rootdir: /home/emes/devel/project/project, inifile: pytest.ini
plugins: django-3.8.0, env-0.6.2, case-1.5.3, cov-2.8.1
collected 2 items
[...]
=============================================== short test summary info ================================================
FAILED deals/tests/test_wtf.py::TestWTF::test_failure - AssertionError: False is not true
FAILED deals/tests/test_wtf.py::TestWTF::test_success - django.db.utils.IntegrityError: UNIQUE constraint failed: use...
================================================== 2 failed in 22.76s ==================================================
edit: I'm using Django-1.11.26
I had the same issue with Django 2.2 on Python 3.6.
I was using pytest 5.4.1 and pytest-django 3.8.0.
Updating pytest-django to version 3.9.0 resolved the issue, so I believe it was an issue in 3.8.0.
Related
I have some twisted.trial tests in my project, that is tests inheriting from twisted.trial.unittest.TestCase.
I need to pass some trial options to my test, specifically it is --reactor option of twisted.trial command line utility. Is there some way for me to pass it to pytest? My thinking is: I add something to pytest.ini, and pytest would somehow launch my trial unittest testcase with this option. Is that possible at the moment?
Sample to reproduce this. Take the following unit test:
# test_reactor.py
from twisted.trial.unittest import TestCase
class CrawlTestCase(TestCase):
def test_if_correct_reactor(self):
from twisted.internet import reactor
from twisted.internet.asyncioreactor import AsyncioSelectorReactor
assert isinstance(reactor, AsyncioSelectorReactor)
Now run it with trial with --reactor flag
python -m twisted.trial --reactor=asyncio test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [OK]
-------------------------------------------------------------------------------
Ran 1 tests in 0.042s
PASSED (successes=1)
Now run it without --reactor flag
python -m twisted.trial test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [ERROR]
===============================================================================
[ERROR]
Traceback (most recent call last):
File "/home/pawel/.../test_reactor.py", line 8, in test_if_correct_reactor
assert isinstance(reactor, AsyncioSelectorReactor)
builtins.AssertionError:
test_reactor.CrawlTestCase.test_if_correct_reactor
-------------------------------------------------------------------------------
Ran 1 tests in 0.081s
FAILED (errors=1)
Now run it with pytest
py.test test_reactor.py
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.9.4, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/pawel/../aaomw, configfile: pytest.ini
plugins: Faker-8.1.3, hypothesis-6.10.1, benchmark-3.4.1
collected 1 item
test_reactor.py F
Question is: how do I pass something to pytest so that it passes it to trial? Is there something I can put in pytest.ini so that reactor is passed to twisted trial?
If what I'm trying to do is not possible, please provide proof that it is not possible, this is also possibly accepted answer, perhaps something needs to be changed in pytest to make this kind of thing possible.
After installing pytest-twisted plugin I get --reactor flag and a proper option of installing and launching reactor.
Two different Terminal windows open. Both set to the same dir. The second one was created by doing a "New tab" while in the first one
In the first one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 12 items
test_MakeInfo.py ............ [100%]
================================================================================= 12 passed in 0.87s =================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
In the second one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 0 items / 1 error
======================================================================================= ERRORS =======================================================================================
_________________________________________________________________________ ERROR collecting test_MakeInfo.py __________________________________________________________________________
ImportError while importing test module '/Users/me/Documents/workspace-vsc/Pipeline/src/python/test_MakeInfo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/me/opt/anaconda3/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_MakeInfo.py:6: in <module>
from MakeInfo import main, makeInfo, makeTumorInfo, _getNormalTumorInfo
E ModuleNotFoundError: No module named 'MakeInfo'
============================================================================== short test summary info ===============================================================================
ERROR test_MakeInfo.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.17s ==================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
What environment variables should I be looking at for differences? Because so far as I can tell everything's the same between the two
You want to check your PYTHONPATH and PATH environment variables.
Depending on the shell you use, they may not have been set the same when opening a new tab.
For example, in bash, you could append the required directory to your path using ~/.bash_profile or ~/.bashrc
Suppose we have installed huge library like SageMath. Let consider trivial test file:
from sage.all_cmdline import * # import sage library
class TestClass:
def test_method(self):
assert True
It runs for about 1.5 sec with Nosetest
$ time nosetests test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
nosetests test.py 1.38s user 0.14s system 97% cpu 1.567 total
Whereas with pytest it runs for ~4.5 sec!
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/user/pytest, inifile: pytest.ini
plugins: profiling-1.7.0
collecting 1 item
/usr/lib/python3.8/site-packages/sage/misc/sage_unittest.py:20:
PytestCollectionWarning: cannot collect test class 'TestSuite' because it has a __init__ constructor (from: test.py)
class TestSuite(object):
collected 1 item
test.py . [100%]
====================================================================== 1 passed in 3.26s ======================================================================
pytest test.py 3.86s user 0.46s system 101% cpu 4.253 total
It looks (according to the warning) like pytest collects some tests from the library itself or may be something else.
The question is: how to speedup pytest startup in the cases like this with huge library to load? And how to avoid loading tests from that huge library?
P.S. See detailed discussion on the subject: https://github.com/pytest-dev/pytest/issues/7111
i am trying to execute my following test suite:
import unittest
from Login_Page import LoginPageAndLogout
def test_suite():
# get all tests from classes
login_test = unittest.TestLoader().loadTestsFromTestCase(LoginPageAndLogout)
# create a test suite
all_tests = unittest.TestSuite([
login_test
])
# run the suite
unittest.TextTestRunner(verbosity=2).run(all_tests)
from Pycharm's terminal using the command :
sudo pytest selenium-tests/testSuite.py -vvv -s
and a part of the output is the following:
============================================================================================================ test session starts ============================================================================================================
platform linux2 -- Python 2.7.14, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
rootdir: /home/osboxes/PycharmProjects/WebTesting, inifile:
plugins: cov-2.5.1
collected 3 items
selenium-tests/testSuite.py::LoginPageAndLogout::test_failed_login <- selenium-tests/Login_Page.py PASSED
selenium-tests/testSuite.py::LoginPageAndLogout::test_login <- selenium-tests/Login_Page.py FAILED
selenium-tests/testSuite.py::test_suite test_failed_login (Login_Page.LoginPageAndLogout) ... ok
test_login (Login_Page.LoginPageAndLogout) ... ok
----------------------------------------------------------------------
Ran 2 tests in 55.993s
The structure of my Login_Page.py file is:
class LoginPageAndLogout(unittest.TestCase):
def setUp(self):
# ...
# login with incorrect credentials to get error message
def test_failed_login(self):
# ...
# login with correct credentials
def test_login(self):
# ...
def tearDown(self):
# ...
As you can see from the output, I have 2 tests but the terminal collects three things instead and run each test twice. Is there a way to execute only the PASSED/FAILED execution, not the ... ok ?
If I comment out unittest.TextTestRunner(verbosity=2).run(all_tests) my tests executed only once but I get the ... ok result instead of the PASSED/FAILED which is the one I want; so I see the pytest execution results instead of the unittests runner results.
How can I run from the terminal my suite using the unitest runner only?
The solution to this was quite easy as I had just misunderstood how my unit test was being executed all this time.
The only thing I had to do was to comment out the whole test_suite class from my testSuite.py file and just import at the top of this file the classes from the test scripts i wanted to execute.
Now my tests run only once and I can still execute all my scripts all at once without typing them in my command one by one using the exact same command:
sudo pytest selenium-tests/testSuite.py -vvv -s
The output of that command is now:
osboxes#osboxes:~/PycharmProjects/WebTesting$ sudo pytest selenium-tests/testSuite.py -vvv -s
========================================================================================================================= test session starts ==========================================================================================================================
platform linux2 -- Python 2.7.14, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
rootdir: /home/osboxes/PycharmProjects/WebTesting, inifile:
plugins: cov-2.5.1
collected 2 items
selenium-tests/testSuite.py::LoginPageAndLogout::test_failed_login <- selenium-tests/Login_Page.py PASSED
selenium-tests/testSuite.py::LoginPageAndLogout::test_login <- selenium-tests/Login_Page.py PASSED
====================================================================================================================== 2 passed in 58.81 seconds =======================================================================================================================
osboxes#osboxes:~/PycharmProjects/WebTesting$
I am using someone else's code, available on GitHub. To run their code I created a virtualenv and installed all the dependencies listed - both python libraries and clones of other repositories. When I proceed to run the included tests, I get an ImportError:
Namespace(all=False, regr=False, sci=False, unit=True)
[localhost] local: py.test -x -v engine/test
==================================================================================== test session starts =====================================================================================
platform linux2 -- Python 2.7.6, pytest-2.8.2, py-1.4.31, pluggy-0.3.1 -- /home/compomics/local/METASPACE/SM_distributed/SM_engine/bin/python
cachedir: engine/test/.cache
rootdir: /home/compomics/local/METASPACE/SM_distributed/engine/test, inifile:
collecting 6 items / 1 errors
=========================================================================================== ERRORS ===========================================================================================
_______________________________________________________________________ ERROR collecting test_formula_img_validator.py _______________________________________________________________________
engine/test/test_formula_img_validator.py:7: in <module>
from engine.formula_img_validator import filter_sf_images,get_compute_img_measures, ImgMeasures
engine/formula_img_validator.py:7: in <module>
from pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
E ImportError: cannot import name measure_of_chaos
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.99 seconds ===================================================================================
Fatal error: local() encountered an error (return code 2) while executing 'py.test -x -v engine/test'
However, if I open the python interpreter and try to do the exact same imports, it does it just fine without any error. Similar questions suggested:
adding an empty __init__.py to the test directory
making sure pytest is installed in the virtualenv
I did both these things, and the error persists.
I added to the beginning of the test script:
import os
print(os.environ["PYTHONPATH"].split(os.pathsep))
print(os.listdir("."))
and confirmed that the folder from where I'm trying to import is indeed in the resulting list.
Not sure how to proceed. Would appreciate any help I can get :)
In file formula_img_validator.py change
from pyIMS.image_measures import measure_of_chaos,isotope_image_correlation, isotope_pattern_match
to
from engine.pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
That'll solve the problem. For complete solution go to GitHub for new updated code.
there was a conflict with other library
EDIT - this was my own stupidity for not remembering I had cloned a previous version of the dependent repos, which was also on my path, and that did not include the function this code was trying to load. Sorry for not having deleted the question when I noticed, I couldn't for the life of me find the delete button :)