I try to collect my tests with py.test but it doesn't do so.
Do I have to provide additional options at the command line?
Py.test was executed in the directory of my .py-file. Are there any other requirements?
Are my tests named correctly? In my code I used 'Test-' for classes and 'test_' for methods.
The results from the terminal:
> py.test
=============================== in 0.00 seconds ============================
user#host:~/workspace/my_project/src/html_extractor$ py.test
============================= test session starts ===========================
platform linux2 -- Python 2.7.3 -- pytest-2.3.4
plugins: xdist
collected 0 items
My code under test:
class Factories(object):
TEST_LINKS = ['http://www.burgtheater.at',
'http://www.musikverein.at/']
def links(self):
for link in self.TEST_LINKS:
yield link
class TestHTMLExtractor(unittest.TestCase):
def setUp(self):
self.factory = Factories()
self.extractor = HTMLExtractor()
def test_extract_site_return(self):
for link in self.factory.links():
raw_html_page = self.extractor.run(link)
def test_whatever():
pass
if __name__ == '__main__':
unittest.main()
In the default configuration, the test file should be named test_<something>.py. See Changing standard (Python) test discovery.
Related
I'm working on the Code-dojo Tennis Kata and I'm stuck with the silliest problem. My pytest stops after running the first test in the file. My test file looks like this:
from tennis import game
import re
def test_game_start(capsys):
MaxVsMimi = game()
MaxVsMimi.result()
out, err = capsys.readouterr()
assert bool(re.search(r'Love : Love', out))
def p1_scores_once(capsys):
MaxVsMimi = game()
MaxVsMimi.score("p1")
MaxVsMimi.result()
out, err = capsys.readouterr()
assert bool(re.match(r'^abc', out))
This is the code:
class game:
def __init__(self):
self.scorep1 = "Love"
self.scorep2 = "Love"
def result(self):
print(self.scorep1, ":", self.scorep2)
and the output:
:stdout:
============================= test session starts ==============================
platform linux -- Python 3.8.6, pytest-6.0.2, py-1.9.0, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /sandbox
collecting ... collected 1 item
test_tennis.py::test_game_start PASSED [100%]
============================== 1 passed in 0.01s ===============================
Why does it not run the second "p1_scores_once" test?
Thanks!
Pytest, like many testing libraries including unittest require all test functions to have the word test in the start of the name. You can fix your code simply by changing the name of p1_scores_once:
def test_p1_scores_once(capsys):
...
Now the automatic test-finding system will treat the function like a test case.
I'm trying to use pytest to test if my function is logging the expected text, such as addressed this question (the pyunit equivalent would be assertLogs). Following the pytest logging documentation, I am passing the caplog fixture to the tester. The documentation states:
Lastly all the logs sent to the logger during the test run are made available on the fixture in the form of both the logging.LogRecord instances and the final log text.
The module I'm testing is:
import logging
logger = logging.getLogger(__name__)
def foo():
logger.info("Quinoa")
The tester is:
def test_foo(caplog):
from mwe16 import foo
foo()
assert "Quinoa" in caplog.text
I would expect this test to pass. However, running the test with pytest test_mwe16.py shows a test failure due to caplog.text being empty:
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /tmp
plugins: mock-1.12.1, cov-2.8.1
collected 1 item
test_mwe16.py F [100%]
=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________
caplog = <_pytest.logging.LogCaptureFixture object at 0x7fa86853e8d0>
def test_foo(caplog):
from mwe16 import foo
foo()
> assert "Quinoa" in caplog.text
E AssertionError: assert 'Quinoa' in ''
E + where '' = <_pytest.logging.LogCaptureFixture object at 0x7fa86853e8d0>.text
test_mwe16.py:4: AssertionError
============================== 1 failed in 0.06s ===============================
Why is caplog.text empty despite foo() sending text to a logger? How do I use pytest such that caplog.text does capture the logged text, or otherwise verify that the text is being logged?
The documentation is unclear here. From trial and error, and notwithstanding the "all the logs sent to the logger during the test run are made available" text, it still only captures logs with certain log levels. To actually capture all logs, one needs to set the log level for captured log messages using caplog.set_level or the caplog.at_level context manager, so that the test module becomes:
import logging
def test_foo(caplog):
from mwe16 import foo
with caplog.at_level(logging.DEBUG):
foo()
assert "Quinoa" in caplog.text
Now, the test passes:
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /tmp
plugins: mock-1.12.1, cov-2.8.1
collected 1 item
test_mwe16.py . [100%]
============================== 1 passed in 0.04s ===============================
In the logger set up, please set logger.propagate=True
This is a limitation of Pytest, cf. the feature request. I've resorted to creating a fixture based on _pytest.logging.LogCaptureFixture with a context manager and using it instead of caplog.
Some code:
from _pytest.logging import LogCaptureHandler, _remove_ansi_escape_sequences
class CatchLogFixture:
"""Fixture to capture logs regardless of the Propagate flag. See
https://github.com/pytest-dev/pytest/issues/3697 for details.
"""
#property
def text(self) -> str:
return _remove_ansi_escape_sequences(self.handler.stream.getvalue())
#contextmanager
def catch_logs(self, level: int, logger: logging.Logger) -> LogCaptureHandler:
"""Set the level for capturing of logs. After the end of the 'with' statement,
the level is restored to its original value.
"""
self.handler = LogCaptureHandler()
orig_level = logger.level
logger.setLevel(level)
logger.addHandler(self.handler)
try:
yield self
finally:
logger.setLevel(orig_level)
logger.removeHandler(self.handler)
#pytest.fixture
def capture_log():
return CatchLogFixture().catch_logs
In the future, I'll need to add many identical tests with different parameters. Now I am making a sample test suite:
import unittest
class TestCase(unittest.TestCase):
def __init__(self, methodName='runTest', param=None):
super(TestCase, self).__init__(methodName)
self.param = param
def test_something(self):
print '\n>>>>>> test_something: param =', self.param
self.assertEqual(1, 1)
if __name__ == "__main__":
suite = unittest.TestSuite()
testloader = unittest.TestLoader()
testnames = testloader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
unittest.TextTestRunner(verbosity=2).run(suite)
It gets discovered by VS Code:
start
test.test_navigator.TestCase.test_something
When I run the tests, I don't receive the parameter:
test_something (test.test_navigator.TestCase) ...
>>>>>> test_something: param = None
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
If I run this file directly, everything works as expected (note param = 42 part)
test_something (__main__.TestCase) ...
>>>>>> test_something: param = 42
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
So it looks like VS Code is running the tests on its own just by using the discovered classes and ignoring TestSuite completely?
What am I doing wrong?
Thanks.
The problem is your code is in a if __name__ == "__main__" block which is only executed when you point Python directly at the file. So when the extension asks unittest to get all the tests and then run them for us it doesn't run the code in your if __name__ == "__main__" block (which is why it can find it but it doesn't do anything magical).
If you can get it to work using unittest's command-line interface then the extension should run it as you want it to.
The key is to implement the load_tests function:
def load_tests(loader, tests, pattern):
suite = unittest.TestSuite()
testnames = loader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
suite.addTest(TestCase(name, param=84))
return suite
The documentation says:
If load_tests exists then discovery does not recurse into the package, load_tests is responsible for loading all tests in the package.
Now my tests run as expected.
P.S. Thanks to Brett Cannon for pointing me to Unit testing framework documentation
I am trying to write a py.test test with selenium to test a complex website. Because the setup is complex, the test can fail even before the actual test. But in such a case I want to be able to call a 'debug' function (maybe its a teardown function which I can use to debug things.
Example:
The test uses a fixture which returns a selenium webdriver
def test1(driver):
driver.get("my page")
...
other tests, login, etc etc
But now the call driver.get fails because of any reason. But in such a case I want to be able to investigate things like
def debug(driver):
driver.screenshot(...)
print(driver.page_source)
...
driver.quit()
(including the shutdown of the driver, as the browser would stay open) with the same driver instance as has been used in the test method.
Is there a way to do that?
The trickiest part is to pass the test result into the fixture, the rest is pretty much trivial. Following the pytests example Making test result information available in fixtures, add a custom hook in your conftest.py:
import pytest
#pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
outcome = yield
rep = outcome.get_result()
setattr(item, "rep_" + rep.when, rep)
return rep
Now you can enhance your driver fixture with custom teardown logic in case the test fails:
# test_selenium.py
import pytest
from selenium import webdriver
def debug(driver):
print('debug: ', driver.title)
#pytest.fixture
def driver(request):
driver = webdriver.Firefox()
yield driver
if request.node.rep_setup.passed and request.node.rep_call.failed:
# this is the teardown code executed on test failure only
debug(driver)
# this is the teardown code that is always executed
driver.quit()
def test_fail(driver):
driver.get('http://wtf')
def test_ok(driver):
driver.get('https://www.google.de')
Running the tests yields:
$ pytest -sv
=============================== test session starts ===============================
platform darwin -- Python 3.6.3, ...
cachedir: .cache
rootdir: /Users/hoefling/projects/private/stackoverflow/so-48521762, inifile:
plugins: ...
collecting ... collected 2 items
test_spam.py::test_fail FAILED [ 50%]
debug Server Not Found
test_spam.py::test_ok PASSED [100%]
==================================== FAILURES =====================================
____________________________________ test_fail ____________________________________
driver = <selenium.webdriver.firefox.webdriver.WebDriver( ...
...
------------------------------- Captured log setup --------------------------------
remote_connection.py 474 DEBUG POST http://127.0.0.1:50319/session { ...
...
remote_connection.py 561 DEBUG Finished Request
======================== 1 failed, 1 passed in 7.23 seconds =======================
I am using py.test to write some tests and in my tests I utilize funcargs. These funcargs have their own setups and teardowns defined in the conftest.py like this:
conftest.py:
def pytest_funcarg__resource_name(request):
def setup():
# do setup
def teardown():
# do teardown
My problem is when someone uses CTRL+C to stop the test executions it leaves everything un-teardowned.
I know there is a hook pytest_keyboard_interrupt but I dont know what to do from there.
Sorry for the noobish question.
You don't provide a full example so maybe i am missing something. But here is an example of how it can work, using the request.cached_setup() helper:
def pytest_funcarg__res(request):
def setup():
print "res-setup"
def teardown(val):
print "res-teardown"
return request.cached_setup(setup, teardown)
def test_hello(res):
raise KeyboardInterrupt()
If you run this with "py.test" you get:
============================= test session starts ==============================
platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev4
plugins: xdist, bugzilla, pep8, cache
collected 1 items
tmp/test_keyboardinterrupt.py res-setup
res-teardown
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/home/hpk/p/pytest/tmp/test_keyboardinterrupt.py:10: KeyboardInterrupt
which shows that setup and teardown are called if a KeyboardInterrupt occurs during test execution.