It seems that ignoring warnings using warnings.filterwarnings is not respected by pytest. For example:
$ cat test.py
import warnings
warnings.filterwarnings('ignore', category=UserWarning)
def test_warnings_filter():
warnings.warn("This is a warning", category=UserWarning)
When I run this I expect that my explicitly ignored warning will by ignored by pytest. Instead I get this:
$ pytest test.py
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0
rootdir: /Users/aldcroft/tmp/pytest
plugins: anyio-3.6.2
collected 1 item
test.py . [100%]
================================================================================ warnings summary =================================================================================
test.py::test_warnings_filter
/Users/aldcroft/tmp/pytest/test.py:7: UserWarning: This is a warning
warnings.warn("This is a warning", category=UserWarning)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================== 1 passed, 1 warning in 0.01s ===========================================================================
I know about pytest.ini config file and the -W flag and #pytest.mark.filterwarnings, but these don't work well
for my use case of integration testing a large number of installed packages via <pkg_name>.test(), where there
are at least a dozen 3rd party warnings that need to be ignored for a clean output.
Any ideas on how to make this work?
You can use #pytest.mark.filterwarnings annotation to ignore warnings from specific test functions or entire test classes.
import warnings
import pytest
#pytest.mark.filterwarnings("ignore:warning")
def test_warnings_filter():
warnings.warn("This is a warning", category=UserWarning)
https://docs.pytest.org/en/7.1.x/how-to/capture-warnings.html#pytest-mark-filterwarnings
Related
I have some twisted.trial tests in my project, that is tests inheriting from twisted.trial.unittest.TestCase.
I need to pass some trial options to my test, specifically it is --reactor option of twisted.trial command line utility. Is there some way for me to pass it to pytest? My thinking is: I add something to pytest.ini, and pytest would somehow launch my trial unittest testcase with this option. Is that possible at the moment?
Sample to reproduce this. Take the following unit test:
# test_reactor.py
from twisted.trial.unittest import TestCase
class CrawlTestCase(TestCase):
def test_if_correct_reactor(self):
from twisted.internet import reactor
from twisted.internet.asyncioreactor import AsyncioSelectorReactor
assert isinstance(reactor, AsyncioSelectorReactor)
Now run it with trial with --reactor flag
python -m twisted.trial --reactor=asyncio test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [OK]
-------------------------------------------------------------------------------
Ran 1 tests in 0.042s
PASSED (successes=1)
Now run it without --reactor flag
python -m twisted.trial test_reactor
test_reactor
CrawlTestCase
test_if_correct_reactor ... [ERROR]
===============================================================================
[ERROR]
Traceback (most recent call last):
File "/home/pawel/.../test_reactor.py", line 8, in test_if_correct_reactor
assert isinstance(reactor, AsyncioSelectorReactor)
builtins.AssertionError:
test_reactor.CrawlTestCase.test_if_correct_reactor
-------------------------------------------------------------------------------
Ran 1 tests in 0.081s
FAILED (errors=1)
Now run it with pytest
py.test test_reactor.py
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.9.4, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/pawel/../aaomw, configfile: pytest.ini
plugins: Faker-8.1.3, hypothesis-6.10.1, benchmark-3.4.1
collected 1 item
test_reactor.py F
Question is: how do I pass something to pytest so that it passes it to trial? Is there something I can put in pytest.ini so that reactor is passed to twisted trial?
If what I'm trying to do is not possible, please provide proof that it is not possible, this is also possibly accepted answer, perhaps something needs to be changed in pytest to make this kind of thing possible.
After installing pytest-twisted plugin I get --reactor flag and a proper option of installing and launching reactor.
Two different Terminal windows open. Both set to the same dir. The second one was created by doing a "New tab" while in the first one
In the first one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 12 items
test_MakeInfo.py ............ [100%]
================================================================================= 12 passed in 0.87s =================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
In the second one:
me $ pytest test_MakeInfo.py
================================================================================ test session starts =================================================================================
platform darwin -- Python 3.7.4, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /Users/me/Documents/workspace-vsc/Pipeline/src/python
plugins: arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0, openfiles-0.4.0
collected 0 items / 1 error
======================================================================================= ERRORS =======================================================================================
_________________________________________________________________________ ERROR collecting test_MakeInfo.py __________________________________________________________________________
ImportError while importing test module '/Users/me/Documents/workspace-vsc/Pipeline/src/python/test_MakeInfo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/me/opt/anaconda3/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_MakeInfo.py:6: in <module>
from MakeInfo import main, makeInfo, makeTumorInfo, _getNormalTumorInfo
E ModuleNotFoundError: No module named 'MakeInfo'
============================================================================== short test summary info ===============================================================================
ERROR test_MakeInfo.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.17s ==================================================================================
me $ which pytest
/Users/me/opt/anaconda3/bin/pytest
What environment variables should I be looking at for differences? Because so far as I can tell everything's the same between the two
You want to check your PYTHONPATH and PATH environment variables.
Depending on the shell you use, they may not have been set the same when opening a new tab.
For example, in bash, you could append the required directory to your path using ~/.bash_profile or ~/.bashrc
Suppose we have installed huge library like SageMath. Let consider trivial test file:
from sage.all_cmdline import * # import sage library
class TestClass:
def test_method(self):
assert True
It runs for about 1.5 sec with Nosetest
$ time nosetests test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
nosetests test.py 1.38s user 0.14s system 97% cpu 1.567 total
Whereas with pytest it runs for ~4.5 sec!
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/user/pytest, inifile: pytest.ini
plugins: profiling-1.7.0
collecting 1 item
/usr/lib/python3.8/site-packages/sage/misc/sage_unittest.py:20:
PytestCollectionWarning: cannot collect test class 'TestSuite' because it has a __init__ constructor (from: test.py)
class TestSuite(object):
collected 1 item
test.py . [100%]
====================================================================== 1 passed in 3.26s ======================================================================
pytest test.py 3.86s user 0.46s system 101% cpu 4.253 total
It looks (according to the warning) like pytest collects some tests from the library itself or may be something else.
The question is: how to speedup pytest startup in the cases like this with huge library to load? And how to avoid loading tests from that huge library?
P.S. See detailed discussion on the subject: https://github.com/pytest-dev/pytest/issues/7111
When I run python setup.py test command in my project folder to test my package with setup.cfg configuration I got this warning message.
How can I disable it?
python setup.py test acts like pytest --flake8 command.
============================================================================================== warnings summary ===============================================================================================
c:\users\yedhrab\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:65
c:\users\yedhrab\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:65: PytestDeprecationWarning: direct construction of Flake8Item has been deprecated, please use Flake8Item.from_parent
return Flake8Item(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================================================================================== short test summary info ===========================================================================================
SKIPPED [1] c:\users\***\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:106: file(s) previously passed FLAKE8 checks
=================================================================================== 9 passed, 1 skipped, 1 warning in 0.33s ===================================================================================
If you do not want to disable all warnings but only a specific pytest warning, use the -W parameter:
pytest -W ignore::pytest.PytestDeprecationWarning
You should use the --disable-warnings flag to silence the warnings for a particular test if that's really what you want.
pytest --disable-warnings
See for more details by running pytest --help.
I solved this issue via adding these lines to my setup.cfg
[tool:pytest]
# ...
filterwarnings =
ignore::DeprecationWarning
I just ran py.test on my code and got the following output:
================== 6 passed, 2 pytest-warnings in 40.79 seconds =======================
However, I cannot see what py.test would like to warn me about. How can I turn on warning output to the console?
py.test --help offers me the --strict flag:
--strict run pytest in strict mode, warnings become
errors.
However I just want to see the output, not make my tests fail.
I checked pytest.org and this question but they are only concerned with asserting warnings in python, not showing warnings generated on the commandline.
In this case, pytest-warnings are warnings which were generated for pytest and/or it's plugins. These warnings were not generated for your code. In order to list them in the report, you will need to use option -r w. Here is portion of py.test --help:
-r chars show extra test summary info as specified by chars (f)ailed,
(E)error, (s)skipped, (x)failed, (X)passed
(w)pytest-warnings (a)all.
This will allow to show warnings in the report (top portion of the record) will list which pytest plugins use deprecated arguments (in my case bellow):
...
================================ pytest-warning summary ================================
WI1 /Projects/.tox/py27/lib/python2.7/site-packages/pytest_timeout.py:68 'pytest_runtest_protocol' hook uses deprecated __multicall__ argument
WI1 /Projects/.tox/py27/lib/python2.7/site-packages/pytest_capturelog.py:171 'pytest_runtest_makereport' hook uses deprecated __multicall__ argument
...