I'm getting deprecation warning from my pipelines at circleci.
Message.
/home/circleci/evobench/env/lib/python3.7/site-packages/_pytest/junitxml.py:436: PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0.
Command
- run:
name: Tests
command: |
. env/bin/activate
mkdir test-reports
python -m pytest --junitxml=test-reports/junit.xml
How should I modify command to use xunit?
Is it possible to a default tool, as it is mentioned in the message?
I mean without specyfing xunit or junit.
Here's full pipeline.
Run your command in this ways.
with xunit2
python -m pytest -o junit_family=xunit2 --junitxml=test-reports/junit.xml
with xunit1
python -m pytest -o junit_family=xunit1 --junitxml=test-reports/junit.xml or
python -m pytest -o junit_family=legacy --junitxml=test-reports/junit.xml
This here describes the change in detail:
The default value of junit_family option will change to xunit2 in
pytest 6.0, given that this is the version supported by default in
modern tools that manipulate this type of file.
In order to smooth the transition, pytest will issue a warning in case
the --junitxml option is given in the command line but junit_family is
not explicitly configured in pytest.ini:
PytestDeprecationWarning: The `junit_family` default value will change to 'xunit2' in pytest 6.0. Add `junit_family=legacy` to your
pytest.ini file to silence this warning and make your suite
compatible.
In order to silence this warning, users just need to configure the
junit_family option explicitly:
[pytest]
junit_family=legacy
In your pytest.ini file add the following line:
junit_family=legacy
If you want to keep the default behavior of the --junitxml option. Or you can accept the new version, xunit2 but not explicitly defining the junit_family variable.
Essentially what the warning is saying is you are giving the --junitxml option in your
run
name: Tests
section not specifying the junit_family variable. You need to start to explicitly defining it to remove the warning or accept the new default.
This thread goes into more details about where to find the .ini file for pytest.
For official statement/documentation about moving from xunit1 to xunit2 read: docs.pytest.org
Also if your project contains pytest.ini file you can set junit_family usage directly from the file like:
# pytest.ini
[pytest]
minversion = 6.0
junit_family=xunit2
junit_suite_name = Pytest Tests
addopts = -ra -q -v -s --junitxml=path/to/pytest_results/pytest.xml
The other answers pretty much covered the means of specifying the junit family, either within pytest.ini or at the commandline with an ini option override.
It's worth looking at the differences between xunit1 and xunit2 for a concrete xml file. Doing a quick spot check on the differences I found these differences show in the image below for the following test module...
# test_stub.py
import sys
import pytest
def test_pass():
assert True
def test_fail():
assert False
if __name__ == "__main__":
sys.exit(pytest.main([__file__] + sys.argv[1:]))
Pytest was ran under three separate configurations (which match the vertical order in the image)
# Default execution
pytest test_stub.py --junit-xml=out.xml
pytest test_stub.py --junit-xml=out_xunit2.xml -o junit_family=xunit2
# Junit prefix execution
pytest test_stub.py --junit-prefix=FOOOP --junit-xml=out_prefix.xml
pytest test_stub.py --junit-prefix=FOOOP --junit-xml=out_prefix_xunit2.xml -o junit_family=xunit2
# Junit suite execution
pytest -o junit_suite_name=SUITE test_stub.py --junit-xml=out_suite.xml
pytest -o junit_suite_name=SUITE test_stub.py --junit-xml=out_suite_xunit2.xml -o junit_family=xunit2
All of the diffs pretty much highlight the fact that xunit2 omits the file and line attributes that showed up previously in xunit1. The other diffs were merely timestamp differences. Both junit_suite_name and junit-prefix behave as before.
Another major difference is that for some reason record_property has been deprecated under the xunit2 schema.
PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest.junitxml.record_property
Related
I have project using python3.6 which in I use pytest for testing. Everything was working fine but suddenly when I run py.test I get this warning message
=============== warnings summary===========
/usr/local/lib/python3.6/dist-packages/gevent/monkey.py:685
/usr/local/lib/python3.6/dist-packages/gevent/monkey.py:685:
DeprecationWarning: inspect.getargspec() is deprecated since Python
3.0, use inspect.signature() or inspect.getfullargspec()
patch_all_args = getargspec(patch_all)[0] # pylint:disable=deprecated-method
-- Docs: https://docs.pytest.org/en/latest/warnings.html
First of all, I haven't changed anything. And secondly the warning is for monkey.py in python3.6.
Why should I get this warning in my project and how can I fix this?
Referring to the documents, I've found out that I have two ways to disable this warning:
Passing -p no:warnings to the py.test in command line:
py.test -p no:warnings
Disable only deprecation warnings once and for all by modifying pytest.ini:
[pytest]
filterwarnings =
ignore:.*U.*mode is deprecated:DeprecationWarning
In pytest-cov documentation it says:
Note that this plugin controls some options and setting the option in
the config file will have no effect. These include specifying source
to be measured (source option) and all data file handling (data_file
and parallel options).
However it doesn't say how to change these options. Is there a way to change it (parallel=True)?
I want to change this because after coverage is upgraded from < 5 to latest (5.1) I got these:
Failed to generate report: Couldn't use data file '/path/to/jenkins/workspace/pr/or/branch/.coverage': no such table: line_bits
Note: using coverage < 5 do not have this problem
I have also tried adding .coveragerc with the following but still get the same issue.
[run]
parallel = True
The way it is run in jenkins:
pytest ./tests --mpl -n 4 \
--junitxml=pyTests.xml --log-cli-level=DEBUG -s \
--cov=. --cov-report --cov-report html:coverage-reports
This is due to pytest-cov using coverage combine, which combines all coverage results: In parallel it mixes results from other runs, that may or may not be completed, and in any cases are irrelevant.
I think if you're having the issue, it may be because you're running multiple tests in parallel, like multiple versions of Python.
In which case it's easily solved by specifying a unique COVERAGE_FILE for each run, like:
export COVERAGE_FILE=.coverage.3.7
for the Python 3.7 run, an so on.
See: https://github.com/nedbat/coveragepy/issues/883#issuecomment-650562896
When I run python setup.py test command in my project folder to test my package with setup.cfg configuration I got this warning message.
How can I disable it?
python setup.py test acts like pytest --flake8 command.
============================================================================================== warnings summary ===============================================================================================
c:\users\yedhrab\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:65
c:\users\yedhrab\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:65: PytestDeprecationWarning: direct construction of Flake8Item has been deprecated, please use Flake8Item.from_parent
return Flake8Item(
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================================================================================== short test summary info ===========================================================================================
SKIPPED [1] c:\users\***\appdata\local\programs\python\python38\lib\site-packages\pytest_flake8.py:106: file(s) previously passed FLAKE8 checks
=================================================================================== 9 passed, 1 skipped, 1 warning in 0.33s ===================================================================================
If you do not want to disable all warnings but only a specific pytest warning, use the -W parameter:
pytest -W ignore::pytest.PytestDeprecationWarning
You should use the --disable-warnings flag to silence the warnings for a particular test if that's really what you want.
pytest --disable-warnings
See for more details by running pytest --help.
I solved this issue via adding these lines to my setup.cfg
[tool:pytest]
# ...
filterwarnings =
ignore::DeprecationWarning
I am trying to use tox automating testing in my project. But I am not able to figure out where the logs or prints from my test_methods in python file goes while using tox. I also grepped the entire tox directory for logs but couldn't find it.
Questions
1) How to configure log directory in tox ?
2) What is the default configuration for logs ?
Any Pointers to the documentation and examples ?
My tox.ini file
[tox]
minversion = 1.6
skipsdist = True
[testenv]
skip_install = True
setenv =
VIRTUAL_ENV={envdir}
deps = -r{toxinidir}/test-requirements.txt
passenv = LANG
[testenv:test]
commands = ./test.sh --slowest --testr-args='{posargs}'
it's not really an answer, but I ended up using asserts.. so instead of
print(type(x))
I do
assert type(x)==1
which gives me
E AssertionError: assert <class 'tuple'> == 1
... so it's a tuple.. A bit crap but it works
There's envlogdir exactly for that:
envlogdir=path
defines a directory for logging where tox will put logs of tool invocation.
default: {envdir}/log
By default your testrunner's logs are in ./.tox/envname/log/envname-k.log where envname is environment name (e.g. py27) and k is number of the command which actually runs your testrunner (python -m testtools.run). k==0 is envname creation log, k==1 is log for first dependency installation, etc.
You can use --verbose and throw error intentionaly to print out log
Example:
print('HERE')
assertTrue(False)
tox -e unit-tests --verbose
If you use Linux, you can simply log the test using:
tox > toxlog.txt
And the output will be in the toxlog.txt file
How can I generate test report using pytest? I searched for it but whatever i got was about coverage report.
I tried with this command:
py.test sanity_tests.py --cov=C:\Test\pytest --cov-report=xml
But as parameters represents it generates coverage report not test report.
Ripped from the comments: you can use the --junitxml argument.
$ py.test sample_tests.py --junitxml=C:\path\to\out_report.xml
You can use a pytest plugin 'pytest-html' for generating html reports which can be forwarded to different teams as well
First install the plugin:
$ pip install pytest-html
Second, just run your tests with this command:
$ pytest --html=report.html
You can also make use of the hooks provided by the plugin in your code.
import pytest
from py.xml import html
def pytest_html_report_title(report)
report.title = "My very own title!"
Reference: https://pypi.org/project/pytest-html/
I haven't tried it yet but you can try referring to https://github.com/pytest-dev/pytest-html. A python library that can generate HTML output.
py.test --html=Report.html
Here you can specify your python file as well. In this case, when there is no file specified it picks up all the files with a name like 'test_%' present in the directory where the command is run and executes them and generates a report with the name Report.html
You can also modify the name of the report accordingly.