I am trying to use tox automating testing in my project. But I am not able to figure out where the logs or prints from my test_methods in python file goes while using tox. I also grepped the entire tox directory for logs but couldn't find it.
Questions
1) How to configure log directory in tox ?
2) What is the default configuration for logs ?
Any Pointers to the documentation and examples ?
My tox.ini file
[tox]
minversion = 1.6
skipsdist = True
[testenv]
skip_install = True
setenv =
VIRTUAL_ENV={envdir}
deps = -r{toxinidir}/test-requirements.txt
passenv = LANG
[testenv:test]
commands = ./test.sh --slowest --testr-args='{posargs}'
it's not really an answer, but I ended up using asserts.. so instead of
print(type(x))
I do
assert type(x)==1
which gives me
E AssertionError: assert <class 'tuple'> == 1
... so it's a tuple.. A bit crap but it works
There's envlogdir exactly for that:
envlogdir=path
defines a directory for logging where tox will put logs of tool invocation.
default: {envdir}/log
By default your testrunner's logs are in ./.tox/envname/log/envname-k.log where envname is environment name (e.g. py27) and k is number of the command which actually runs your testrunner (python -m testtools.run). k==0 is envname creation log, k==1 is log for first dependency installation, etc.
You can use --verbose and throw error intentionaly to print out log
Example:
print('HERE')
assertTrue(False)
tox -e unit-tests --verbose
If you use Linux, you can simply log the test using:
tox > toxlog.txt
And the output will be in the toxlog.txt file
Related
I'm getting deprecation warning from my pipelines at circleci.
Message.
/home/circleci/evobench/env/lib/python3.7/site-packages/_pytest/junitxml.py:436: PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0.
Command
- run:
name: Tests
command: |
. env/bin/activate
mkdir test-reports
python -m pytest --junitxml=test-reports/junit.xml
How should I modify command to use xunit?
Is it possible to a default tool, as it is mentioned in the message?
I mean without specyfing xunit or junit.
Here's full pipeline.
Run your command in this ways.
with xunit2
python -m pytest -o junit_family=xunit2 --junitxml=test-reports/junit.xml
with xunit1
python -m pytest -o junit_family=xunit1 --junitxml=test-reports/junit.xml or
python -m pytest -o junit_family=legacy --junitxml=test-reports/junit.xml
This here describes the change in detail:
The default value of junit_family option will change to xunit2 in
pytest 6.0, given that this is the version supported by default in
modern tools that manipulate this type of file.
In order to smooth the transition, pytest will issue a warning in case
the --junitxml option is given in the command line but junit_family is
not explicitly configured in pytest.ini:
PytestDeprecationWarning: The `junit_family` default value will change to 'xunit2' in pytest 6.0. Add `junit_family=legacy` to your
pytest.ini file to silence this warning and make your suite
compatible.
In order to silence this warning, users just need to configure the
junit_family option explicitly:
[pytest]
junit_family=legacy
In your pytest.ini file add the following line:
junit_family=legacy
If you want to keep the default behavior of the --junitxml option. Or you can accept the new version, xunit2 but not explicitly defining the junit_family variable.
Essentially what the warning is saying is you are giving the --junitxml option in your
run
name: Tests
section not specifying the junit_family variable. You need to start to explicitly defining it to remove the warning or accept the new default.
This thread goes into more details about where to find the .ini file for pytest.
For official statement/documentation about moving from xunit1 to xunit2 read: docs.pytest.org
Also if your project contains pytest.ini file you can set junit_family usage directly from the file like:
# pytest.ini
[pytest]
minversion = 6.0
junit_family=xunit2
junit_suite_name = Pytest Tests
addopts = -ra -q -v -s --junitxml=path/to/pytest_results/pytest.xml
The other answers pretty much covered the means of specifying the junit family, either within pytest.ini or at the commandline with an ini option override.
It's worth looking at the differences between xunit1 and xunit2 for a concrete xml file. Doing a quick spot check on the differences I found these differences show in the image below for the following test module...
# test_stub.py
import sys
import pytest
def test_pass():
assert True
def test_fail():
assert False
if __name__ == "__main__":
sys.exit(pytest.main([__file__] + sys.argv[1:]))
Pytest was ran under three separate configurations (which match the vertical order in the image)
# Default execution
pytest test_stub.py --junit-xml=out.xml
pytest test_stub.py --junit-xml=out_xunit2.xml -o junit_family=xunit2
# Junit prefix execution
pytest test_stub.py --junit-prefix=FOOOP --junit-xml=out_prefix.xml
pytest test_stub.py --junit-prefix=FOOOP --junit-xml=out_prefix_xunit2.xml -o junit_family=xunit2
# Junit suite execution
pytest -o junit_suite_name=SUITE test_stub.py --junit-xml=out_suite.xml
pytest -o junit_suite_name=SUITE test_stub.py --junit-xml=out_suite_xunit2.xml -o junit_family=xunit2
All of the diffs pretty much highlight the fact that xunit2 omits the file and line attributes that showed up previously in xunit1. The other diffs were merely timestamp differences. Both junit_suite_name and junit-prefix behave as before.
Another major difference is that for some reason record_property has been deprecated under the xunit2 schema.
PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest.junitxml.record_property
After some time I finally managed to successfully install python and pip and run it on my machine using Visual Studio Code.
I am working in virtual environment in python and we have a Makefile with following statement:
test:
source .env && PYTHONPATH=. PY_ENV=testing py.test ${ARGS} --duration=20
File .env lives in the main directory next to Makefile. It contains some environmental variables needed for testing certain APIs.
When I take the line out of the file and run it in my terminal, everything works fine and all tests are running etc.
However if I call the following: make test I am getting this error:
$ make test
source .env && PYTHONPATH=. PY_ENV=testing py.test --duration=20
/usr/bin/sh: line 0: source: .env: file not found
make: *** [test] Error 1
(venv)
To me it looks like when running this command from within Makefile it can't see the .env file but have no idea how to solve it.
The source command isn't looking up the file in the current working directory. As mentioned in man source:
Read and execute commands from filename in the current shell
environment and return the exit status of the last command executed
from filename. If filename does not contain a slash, filenames in
PATH are used to find the directory containing filename.
Change the file path like so:
test:
source ./.env && PYTHONPATH=. PY_ENV=testing py.test ${ARGS} --duration=20
Note that this error does not occur in bash version < 4. This is due to an implementation bug when run under POSIX mode (what make uses, since its default shell is sh, which is usually bash --posix). The correct behaviour was first mentioned in the documentation of bash-2.05 (revision 28ef6c31, file doc/bashref.info):
When Bash is not in POSIX mode, the current directory is searched if
FILENAME is not found in `$PATH'.
These older versions searched the current directory regardless of POSIX mode. It was only in bash-4.0-rc1 (revision 3185942a, file general.c) that this was corrected. Running git diff 3185942a~ 3185942a general.c outputs this section:
## -69,6 +69,7 ## posix_initialize (on)
if (on != 0)
{
interactive_comments = source_uses_path = expand_aliases = 1;
+ source_searches_cwd = 0;
}
Currently using tox to test a python package, and using a python library (chromedriver-binary) to install chromedriver.
This library creates a script (chromedriver-path) which when called outputs the PATH where chromedriver is installed. The usual way to use this is to run:
export PATH=$PATH:`chromedriver-path`
I've tried the following without success in tox.ini
setenv=
PATH = {env:PATH}{:}`chromedriver-path`
This errors as expected:
FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver': 'chromedriver'
Implying that the setenv command is never called/run.
commands=
export PATH=$PATH:`chromedriver-path
This fails with:
ERROR: InvocationError for command could not find executable export
How do I make this work?
Commands can't change their parent processes' environment variables, and thus can't change the environment variables of subsequent commands launched by forking that parent; they can only set environment variables for themselves or their own children.
If you were able to collect the output of chromedriver-path before starting tox, this would be moot. If it's only available in an environment tox itself creates, then things get a bit more interesting.
One approach you can follow is to wrap the commands that need this path entry in a shim that adds it. Consider changing:
commands=
py test ...
to:
commands=
sh -c 'PATH=$PATH:$(chromedrive-path); exec "$#"' _ py test ...
I would like to use different command line arguments for py.test, depending on environment: running locally there should be only default ones, but on Jenkins I would like to add --junitxml=junit-{envname}.xml, so the test results could be published in nice format.
I know from documentation, that there is special [tox:jenkins] section, which should be used in case there is defined 'JENKINS_URL' or 'HUDSON_URL'. So now I created simple tox.ini file:
[tox]
envlist = py27, py35
[tox:jenkins]
commands = echo "We are in JENKINS!"
[testenv]
setenv =
PYTHONPATH = {toxinidir}:{toxinidir}/my_module
commands = python setup.py test
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/requirements_test.txt
And have defined JENKINS_URL environment variable:
export JENKINS_URL=true
I expect, that if I run tox, then my original command will be substituted with echo, but it doesn't work, instead I end with original command been executed.
Could someone help me with this problem?
Okay, found the solution myself:
[tox]
envlist = py27, py35
[testenv]
setenv =
PYTHONPATH = {toxinidir}:{toxinidir}/my_module
commands = py.test {env:CUSTOM_ARGS} ; <== use environment variable here
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/requirements_test.txt
and in Jenkins just define CUSTOM_ARGS environment variable like this:
export CUSTOM_ARGS="--junitxml=junit.xml"
You could have used positional arguments.
Like this:
commands = py.test --mandatory-flag {posargs:--default-flag}
And in Jenkins you can call tox like this:
tox -- --override-flag
Note the separation with two dashes --.
I am working in windows and attempting to run a git diff command in the pre-commit script (Python) of a repository. My Python call looks like this:
repo_dir = 'D:/git/current_uic/src/gtc/resource'
cmd = ['diff', '--name-only']
print(Popen(['git', '--git-dir={}'.format(repo_dir + '/.git'),
'--work-tree={}'.format(repo_dir)] + cmd,
stdin=PIPE, stdout=PIPE).communicate())
Whenever I go to commit in the "D:/git/current_uic/src/gtc" repo, I get the following:
fatal: unable to read 6ff96bd371691b9e93520e133ebc4d84c74cd0f6
Note that this is a pre-commit hook for the 'D:/git/current_uic/src/gtc' repository and that 'D:/git/current_uic/src/gtc/resource' is a submodule of 'D:/git/current_uic/src/gtc'. Also note that if I pop open Git bash and run the following:
git --git-dir=D:/git/current_uic/src/gtc/resource/.git
--work-tree=D:/git/current_uic/src/gtc/resource diff --name-only
or if I just run the script straight from Git bash I get exactly what I want, regardless of working directory.
Any ideas as to what is going on here?
The Problem:
Upon running a hook, Git sets some environment variables that are accessible by the hook script. The problem is that Git itself uses these environment variables, and the normal way in which Git sets/uses them seems to be overridden by the values set when the hook gets fired off. In this particular instance, the environment variable GIT_INDEX_FILE has been set to the path to the index file corresponding to the repository which had called the hook (D:/git/current_uic/src/.git/modules/gtc/index), causing a mismatch between the (incorrect) index and the (correct) change tree.
The Fix:
In the hook script, set the environment variable GIT_INDEX_FILE to the correct value before making any git calls. In this case, you could do the following:
set GIT_INDEX_FILE=D:/git/current_uic/src/.git/modules/gtc/modules/resource/index
git --git-dir=D:/git/current_uic/src/gtc/resource/.git
--work-tree=D:/git/current_uic/src/gtc/resource diff --name-only
Additional Info
More information about these Git environment variables and which hooks set them can be found here.
Got exactly same issue but using gitpython.
I solved it like this:
repo = git.Repo()
for submodule in repo.submodules:
back_index = os.getenv('GIT_INDEX_FILE')
os.environ['GIT_INDEX_FILE'] = submodule.module().index.path
commit = submodule.module().head.commit
print([item.a_path for item in commit.diff(None)])
os.environ['GIT_INDEX_FILE'] = back_index