I'm unable to get logging output from my unit tests and it looks like the reason is that all loggers except for the root logger have propagate set to false when I run pytest. I can run the following file, using pytest and python test_file.py. In the former logging.getLogger('one').propagate == True while in the latter logging.getLogger('one').propagate == False. Why is this?
test_file.py
import logging
def test_function():
print()
print('root', logging.getLogger().propagate)
print('one', logging.getLogger('one').propagate)
if __name__ == '__main__':
test_function()
How do I get all my loggers to propagate? Searching the internet only turns up questions about how to turn off propagation as if most people have the opposite experience that I do.
This isn't a problem with pytest. I searched the cpython and pytest source code for anything that would do this and found nothing. Then I stepped through the getLogger code and found that something was calling the logging.setLoggerClass function and replacing the logger class with a subclass that sets propagate to false. I set a breakpoint on setLoggerClass and found that a pytest plugin provided by ROS2 was doing this. I added the following to pytest.ini and now everything works great.
[pytest]
addopts = -p no:launch -p no:launch_ros
Related
I am running a particular unit test in my library, and I would like to get more logging from it. I am running it in the following way:
python -m unittest my_module.my_submodule_test.MyTestClass.test_my_testcase
Because I'm doing this, the my_submodule_test.py file does not run the bottom part where I set the log level like so:
if __name__ == '__main__':
logging.getLogger().setLevel(logging.DEBUG)
unittest.main()
How can I programatically set the log level so that I can get more detailed logging from my test?
Also, I'd like to be able to set the logging via a command-line argument. Is there a way to do that?
One way in which this can be done is by doing it in the setUp code of your TestCase subclass. You would do the following:
class MyTestClass(unittest.TestCase):
def setUp(self):
logging.basicConfig(level=logging.DEBUG)
logging.getLogger().setLevel(logging.DEBUG)
You can use either logging.basicConfig(level=logging.DEBUG), or logging.getLogger().setLevel(logging.DEBUG).
This should activate logging for your whole project (unless you are changing the level for loggers further down in the hierarchy that you care about).
I do not know of a way to do this from the command line, unfortunately.
Robot Framework has a great set of unit tests which are implemented using Python's unittest module. I wonder if these tests can be run with Pytest and if somebody has already tried to do so. At least Pytest's docu says that it can deal with regular Python unittest.
EDIT: To be more precise. I would like to run Robot's own over 1000 unit tests with Pytest instead of with Python's unittest module. E.g. now you have to run python run.py inside the utest folder of RF's repo to execute all unit tests. So what I am actually asking for is how to modify run.py so that it uses the Pytest framework instead of the unittest framework?
I think the most tricky part is here:
if __name__ == '__main__':
docs, vrbst = parse_args(sys.argv[1:])
tests = get_tests()
suite = unittest.TestSuite(tests)
runner = unittest.TextTestRunner(descriptions=docs, verbosity=vrbst)
result = runner.run(suite)
rc = len(result.failures) + len(result.errors)
if rc > 250:
rc = 250
sys.exit(rc)
especially:
suite = unittest.TestSuite(tests)
runner = unittest.TextTestRunner(descriptions=docs, verbosity=vrbst)
I already can run single test by pointing pytest to concrete test file, e.g. pytest utest/api/test_exposed_api.py. But if I try to run all unit test in utest folder with pytest utest/ I just get errors and warnings :(
Short answer: Yes it is! :)))
Long answer: I had to edit only two lines in run.py and was almost happy with the result
add import pytest below import unittest
edit the if __name__ == '__main__': section so that it looks like that:
if __name__ == '__main__':
docs, vrbst = parse_args(sys.argv[1:])
tests = get_tests()
pytest.main()
Then in utest folder just call python run.py and the tests run :)))
1532 passed, 45 warnings, 1 error in 9.70 seconds
Obviously there is more stuff which is not necessary for pytest in run.py and could be wiped out ... but me no python expert (yet)
I've installed new pytest plugin (pytest-catchlog==1.2.2) and as much as I like it, it breaks my unit tests for logging module (e.g ValueError: I/O operation on closed file).
I would like to disable that plugin for test_logging.py file (or even a class or method), but I can't find any information on it.
The only option I found so far is to execute pytest twice: first time for test_logging.py onlywith catchlog disabled (py.test -p no:catchlog test_logging.py), and second time for all other test files.
Please let me know if I missed a pytest decorator, or any other way of disabling plugins in runtime.
You cannot selectively disable arbitrary plugins for selected tests. The plugins are loaded at a much earlier stage — when the pytest starts. And the plugins actually define what pytest does and how (i.e., command line options, test collection, filtering, etc).
In other words, it is too late to redefine the pytest's internal structure when it gets to the test execution.
Your best case is, indeed, to mark your tests with #pytest.mark.nocatchlog, and execute them separately:
pytest -m 'nocatchlog' -p no:catchlog # problematic tests with no plugin
pytest -m 'not nocatchlog` # all other tests
If those tests not under your control, i.e. if you cannot add marks, then you can only filter by expressions like -k test_logging or -k 'not test_logging' (i.e. by part of their node id).
Specifically for this pytest-catchlog plugin, you can make the same hooks as it does, and remove its log handler from the root logger (assuming that no other loggers were used explicitly):
conftest.py:
import pytest
def _disable_catchlog(item):
logger = logging.getLogger()
if item.catch_log_handler in logger.handlers:
logger.handlers.remove(item.catch_log_handler)
#pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_setup(item):
_disable_catchlog(item)
yield
#pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_call(item):
_disable_catchlog(item)
yield
#pytest.hookimpl(hookwrapper=True, trylast=True)
def pytest_runtest_teardown(item):
_disable_catchlog(item)
yield
I'm using nose to run my "unittest" tests and have nose-cov to include coverage reports. These all work fine, but part of my tests require running some code as a multiprocessing.Process. The nose-cov docs state that it can do multiprocessing, but I'm not sure how to get that to work.
I'm just running tests by running nosetests and using the following .coveragerc:
[run]
branch = True
parallel = True
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain about missing debug-only code:
def __repr__
#if self\.debug
# Don't complain if tests don't hit defensive assertion code:
raise AssertionError
raise NotImplementedError
# Don't complain if non-runnable code isn't run:
if 0:
if __name__ == .__main__.:
def __main__\(\):
omit =
mainserver/tests/*
EDIT:
I fixed the parallel switch in my ".coveragerc" file. I've also tried adding a sitecustomize.py like so in my site-packages directory:
import os
import coverage
os.environ['COVERAGE_PROCESS_START']='/sites/metrics_dev/.coveragerc'
coverage.process_startup()
I'm pretty sure it's still not working properly, though, because the "missing" report still shows lines that I know are running (they output to the console). I've also tried adding the environment variable in my test case file and also in the shell before running the test cases. I also tried explicitly calling the same things in the function that's called by multiprocessing.Process to start the new process.
First, the configuration setting you need is parallel, not parallel-mode. Second, you probably need to follow the directions in the Measuring Subprocesses section of the coverage.py docs.
Another thing to consider is if you see more than one coverage file while running coverage. Maybe it's only a matter of combining them afterwards.
tl;dr — to use coverage + nosetests + nose’s --processes option, set coverage’s --concurrency option to multiprocessing, preferably in either .coveragerc or setup.cfg rather than on the command-line (see also: command line usage and configuration files).
Long version…
I also fought this for a while, having followed the documentation on Configuring Python for sub-process coverage to the letter. Finally, upon re-examining the output of coverage run --help a bit more closely, I stumbled across the --concurrency=multiprocessing option, which I’d never used before, and which seems to be the missing link. (In hindsight, this makes sense: nose’s --processing option uses the multiprocessing library under the hood.)
Here is a minimal configuration that works as expected:
unit.py:
def is_even(x):
if x % 2 == 0:
return True
else:
return False
test.py:
import time
from unittest import TestCase
import unit
class TestIsEvenTrue(TestCase):
def test_is_even_true(self):
time.sleep(1) # verify multiprocessing is being used
self.assertTrue(unit.is_even(2))
# use a separate class to encourage nose to use a separate process for this
class TestIsEvenFalse(TestCase):
def test_is_even_false(self):
time.sleep(1)
self.assertFalse(unit.is_even(1))
setup.cfg:
[nosetests]
processes = 2
verbosity = 2
[coverage:run]
branch = True
concurrency = multiprocessing
parallel = True
source = unit
sitecustomize.py (note: located in site-packages)
import os
try:
import coverage
os.environ['COVERAGE_PROCESS_START'] = 'setup.cfg'
coverage.process_startup()
except ImportError:
pass
$ coverage run $(command -v nosetests)
test_is_even_false (test.TestIsEvenFalse) ... ok
test_is_even_true (test.TestIsEvenTrue) ... ok
----------------------------------------------------------------------
Ran 2 tests in 1.085s
OK
$ coverage combine && coverage report
Name Stmts Miss Branch BrPart Cover
-------------------------------------------
unit.py 4 0 2 0 100%
I'm sure this is a simple fix, but I'd like to view log messages in the PyCharm console while running a unit test. The modules I'm testing have their own loggers, and normally I'd set a root logger to catch the debugging messages at a certain level, and pipe the other logs to a file. But I can't figure out how this works with unit tests.
I'm using the unittest2 module, and using PyCharm's automatic test discovery (which probably is based on nose, but I don't know).
I've tried fooling with the run configurations, but there doesn't seem to be a straightforward way to do this.
The PyCharm documentation isn't particularly helpful here either, if any of you work there.
In edit: It DOES appear that the console catches critical level log messages. I want to know if there is a way to configure this to catch debug level messages.
This post (Pycharm unit test interactive debug command line doesn't work) suggests adding the -s option to the build configuration, which does not produce the desired result.
The only solution I've found is to do the normal logging setup at the top of the file with tests in it (assuming you're just running one test or test class): logging.basicConfig(level=logging.DEBUG). Make sure you put that before most import statements or the default logging for those modules will have already been set (that was a hard one to figure out!).
I needed to add too the option --nologcapture to nosetests as param in pycharm 2016.3.2
Run>Edit COnfigurations > Defaults > Python Tests > Nosetests : activate the check for Prams option and add --nologcapture
In Edit Configurations:
delete old configurations
go to Defaults / Python tests / NoseTests
add --nologcapture to "additional Arguments"
worked like a "charm" for me in pyCharm 2017.2.3
thanks bott
My problem was similar, I only saw messages with level WARNING or higher while running tests in PyCharm. A colleague suggested to configure the logger in __init__.py which is in the directory of my tests.
# in tests/__init__.py
import logging
import sys
# Reconfiguring the logger here will also affect test running in the PyCharm IDE
log_format = '%(asctime)s %(levelname)s %(filename)s:%(lineno)d %(message)s'
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, format=log_format)
Then I can simply log in the test code like:
import logging
logging.info('whatever')
I experienced this problem all of a sudden after updating Pycharm. I had been running my tests with the Unittest runner, and all of a sudden Pycharm decided the default should be the pytest runner. When I changed the default test runner back to Unittest the logs appeared as I expected they would.
You can change the default test runner in Project Settings/Preferences >> Tools >> Python Integrated Tools
I assume adding the -nologcapture flag as noted in other answers probably would have worked in my case as well, but I prefer a solution that's exposed by the IDE rather than a manual override.
BTW choosing Unittest also solves an additional error that I got once when trying to run tests - No module named 'nose'
The -s option mentioned in the question in combination with adding the StreamHandler using stream=sys.stdout did work for me.
import logging
import sys
logger = logging.getLogger('foo')
logger.addHandler(logging.StreamHandler(stream=sys.stdout))
logger.warning('foo')
Pycharm unit test interactive debug command line doesn't work
Add a stream handler, pointing to sys.stdout if doctest or pytest is running:
import logging
import sys
logger = logging.getLogger()
def setup_doctest_logger(log_level:int=logging.DEBUG):
"""
:param log_level:
:return:
>>> logger.info('test') # there is no output in pycharm by default
>>> setup_doctest_logger()
>>> logger.info('test') # now we have the output we want
test
"""
if is_pycharm_running():
logger_add_streamhandler_to_sys_stdout()
logger.setLevel(log_level)
def is_pycharm_running()->bool:
if ('docrunner.py' in sys.argv[0]) or ('pytest_runner.py' in sys.argv[0]):
return True
else:
return False
def logger_add_streamhandler_to_sys_stdout():
stream_handler=logging.StreamHandler(stream=sys.stdout)
logger.addHandler(stream_handler)