After I implemented my test, which is using pandas, my build is failing with "ModuleNotFoundError: No module named 'pandas'" error, however, I added pandas to the testenv deps and in the log file I also see that it is installed. I got the same error in case of boto3 but after I added it to the deps, it solved the problem but in case of pandas it doesn't work.
tox.ini
[tox]
envlist=flake8
py36_tests
[testenv]
deps=pytest
flake8
[testenv:py36_tests]
basepython=python3.6
deps=boto3
pandas
commands=py.test -s -v tests --junitxml=report.xml
[testenv:flake8]
commands=flake8 --exclude=.git,__pycache__,__init__.py data_collector/
flake8 tests/
flake8 setup.py
flake8 setup-cy.py
flake8 Docker/startup_scripts/
flake8 bin/data_collector
Log:
$ tox -e py36_tests
GLOB sdist-make: /builds/<path>/<my_package>/setup.py
py36_tests create: /builds/<path>/<my_package>/.tox/py36_tests
py36_tests installdeps: boto3, pandas
py36_tests inst: /builds/<path>/.tox/.tmp/package/1/<my_package>-0.0.0.zip
py36_tests installed: <many packages>, pandas==1.1.1, <many packages>
y36_tests run-test-pre: PYTHONHASHSEED='3745093701'
py36_tests run-test: commands[0] | py.test -s -v tests --junitxml=report.xml
WARNING: test command found but not installed in testenv
cmd: /usr/local/bin/py.test
env: /builds/<path>/<my_package>/.tox/py36_tests
Maybe you forgot to specify a dependency? See also the allowlist_externals envconfig setting.
DEPRECATION WARNING: this will be an error in tox 4 and above!
============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /usr/bin/python3.6
cachedir: .tox/py36_tests/.pytest_cache
rootdir: /builds/<path>/<my_package>
collecting ... collected 10 items / 1 error / 9 selected
==================================== ERRORS ====================================
________________ ERROR collecting tests/ingestion/test_task.py _________________
ImportError while importing test module '/builds/<path>/tests/ingestion/test_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.6/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/ingestion/test_task.py:3: in <module>
import pandas as pd
E ModuleNotFoundError: No module named 'pandas'
- generated xml file: /builds/<path>/<my_package>/report.xml -
=========================== short test summary info ============================
ERROR tests/ingestion/test_task.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.24s ===============================
ERROR: InvocationError for command /usr/local/bin/py.test -s -v tests --junitxml=report.xml (exited with code 2)
___________________________________ summary ____________________________________
ERROR: py36_tests: commands failed
While I hadn't got any import in my test it worked well. What is the problem?
See this
WARNING: test command found but not installed in testenv
cmd: /usr/local/bin/py.test
You have not installed py.test into the virtual environment so tox found a global one. And of course the global one runs with a global Python and doesn't know anything about your virtual environments in tox.
To fix it install pytest into the virtual environment:
[testenv:py36_tests]
deps=boto3
pandas
pytest
I used a virtual env after the tox call, the solution was to move the tox call when the virtual env was already activated.
Question
How to use tox to install a the local package without it refering to a random time when I installed the pacakge from GitHub?
Context
In this PR, I'm trying to build and test my package, dbt-synapse, before publishing a new minor version to PyPI. At some point (rookie move), I think I tried installing the package directly from a specific GitHub commit with the below line.
pip install -e git+https://github.com/dbt-msft/dbt-synapse.git#345d7cafcb08bac25d23867a2e22c0e9b741f603#egg=dbt_synapse
Here's a gist of the requirements.txt, tox.ini, setup.py and some other files
When I call tox -e integration-synapse to kick off the tests, having listing the actual package in the deps as -e. compared to just ., causes the package to be installed from GitHub (see the integration-synapse installed: line below, shortened for clarity)!
when deps has just ., however it lists this as having been installed (which also doesn't seem right): dbt-synapse # file:///Users/anders.swanson/repos/dbt-synapse.
Unfortunately, neither are giving me what I know should be happening.
integration-synapse recreate: /Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse
integration-synapse installdeps: -rrequirements.txt, -e.
integration-synapse installed: [...],-e git+https://github.com/dbt-msft/dbt-synapse.git#345d7cafcb08bac25d23867a2e22c0e9b741f603#egg=dbt_synapse,[...]
integration-synapse runtests: PYTHONHASHSEED='1932234374'
integration-synapse runtests: commands[0] | /bin/bash -c '/Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse/bin/python -m pytest -v test/integration/synapse.dbtspec'
======================================================================================================================================== test session starts =========================================================================================================================================
platform darwin -- Python 3.6.12, pytest-6.2.1, py-1.10.0, pluggy-0.13.1 -- /Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse/bin/python
cachedir: .pytest_cache
rootdir: /Users/anders.swanson/repos/dbt-synapse
plugins: dbt-adapter-0.4.0
collected 1 item
tox.ini
[tox]
skipsdist = True
envlist = integration-synapse
[testenv:integration-synapse]
basepython = python3
commands = /bin/bash -c '{envpython} -m pytest -v test/integration/synapse.dbtspec'
passenv = DBT_SYNAPSE_DB DBT_SYNAPSE_PORT DBT_SYNAPSE_PWD DBT_SYNAPSE_SERVER DBT_SYNAPSE_UID
deps =
-rrequirements.txt
-e.
If you want to start from a clean state, you can you can delete the hidden .tox folder and start from scratch.
Also, you can run tox -r which will recreate the environemnts.
In order to test the code from your local development environment, you do not have to include your package in the deps list (-e.). tox will build the package from your setup.py.
Well.. it would. But in your tox.ini you specified skipsdist = True which afaik means tox should not build the package.
There are some more issues in your tox.ini, e.g. unit and flake8 envs do nothing.
Why do you use the /bin/bash -c construct?
Unfortunately, I am missing some header files, so I cannot run your tox.ini file, but I would create a tox.ini file like this...
[tox]
envlist = py36,py37,py38,py39,integration-synapse,flake8
[testenv]
deps = whatever needed for your unit tests
commands = python -m pytest ...unit tests...
[testenv:integration-synapse]
deps =
pytest
pytest-dbt-adapter
commands =
python -m pytest -v test/integration/synapse.dbtspec
passenv = DBT_SYNAPSE_DB DBT_SYNAPSE_PORT DBT_SYNAPSE_PWD DBT_SYNAPSE_SERVER DBT_SYNAPSE_UID
[testenv:flake8]
skipsdist = True
deps = flake8
commands = flake8 setup.py ... (and more)
I'm developing a custom layer for the Keras framework and would like to extend their existing testsuite.
Since I'm working on the keras sourcecode, I have installed it like this:
pip install keras-preprocessing
pip install keras-applications
git clone https://github.com/keras-team/keras
cd keras
export PYTHONPATH=$PWD:$PYTHONPATH
After these commands, you are in the cloned keras repository, this is the working directory for the code samples below.
To begin with, I would like to run the existing tests and see what they do. It looks like they can simply be run as a python file:
if __name__ == '__main__':
pytest.main([__file__])
But this:
python tests/keras/layers/wrappers_test.py
produces the following output, and doesn't run any tests
Using TensorFlow backend.
usage: wrappers_test.py [options] [file_or_dir] [file_or_dir] [...]
wrappers_test.py: error: unrecognized arguments: -n tests/keras/layers/wrappers_test.py
inifile: /home/lhk/programming/keras/pytest.ini
rootdir: /home/lhk/programming/keras
So I tried explicitly calling pytest next:
pytest tests/keras/layers/wrappers_test.py
Exactly the same response (without the tensorflow log):
usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: -n tests/keras/layers/wrappers_test.py
inifile: /home/lhk/programming/keras/pytest.ini
rootdir: /home/lhk/programming/keras
I'm using PyCharm, and if I run the tests from within pycharm (which I admittedly tried first), it only produces the messages above.
Apparently, I'm not configuring it correctly. Pytest is not able to pick up the test suite. To find a reference configuration, I looked at the Keras CI setup. They use Travis and the config is opensource: https://travis-ci.org/keras-team/keras/jobs/442252422/config
It looks like I have all the dependencies installed. And the actual test command is basically what I have already tried:
PYTHONPATH=$PWD:$PYTHONPATH py.test tests/ --ignore=tests/integration_tests --ignore=tests/test_documentation.py --ignore=tests/keras/legacy/layers_test.py --cov-config .coveragerc --cov=keras tests/
It produces the exact same output as above.
I assume that the problem lies with pytest. But the installation section of their automated testing simply shows pip install pytest pytest-pip8. I ran this, but sure enough, requirement already satisfied.
How can I execute the keras pytests.
I'm running ubuntu 18.04.1, python 3.6.5 with anaconda 64bit.
Running the keras tests is mentioned on the Contributing page:
Run our test suite locally. It's easy: from the Keras folder, simply run: py.test tests/.
You will need to install the test requirements as well: pip install -e .[tests].
If you don't want to install the package in editable mode, just install all of the test dependencies explicitly. Looking at keras' setup script, the command for that would be:
$ pip install pytest pytest-pep8 pytest-xdist pytest-cov pytest-timeout pandas requests
Package Settings
I have built a Python package which uses nose for testing. Therefore, setup.py contains:
..
test_suite='nose.collector',
tests_require=['nose'],
..
And python setup.py test works as expected:
running test
...
----------------------------------------------------------------------
Ran 3 tests in 0.065s
OK
Running with XUnit output
Since I'm using Jenkins CI, I would like to output the nose results to JUnit XML format:
nosetests <package-name> --with-xunit --verbose
However, python setup.py test is far more elegant, and it installs the test requirements without having to build a virtual environment.
Is there a way to pass the --with-xunit (or any other parameter) to nose, when calling nose via python setup.py test?
You can set nosetests option using setup.cfg
For example in you setup.cfg
[nosetests]
with-xunit=1
Further information can be found at http://nose.readthedocs.io/en/latest/api/commands.html
Nose provides its own setuptools command (nosetests) which accepts command line arguments:
python setup.py nosetests --with-xunit
More information can be found here:
http://nose.readthedocs.io/en/latest/setuptools_integration.html
subtitle: Not only sdist
I am trying to get the setup.py file of a package I'm working on to play nicely with sdist. The relevant parts of the setup.py file are:
from setuptools.command.test import test
[...]
class Tox(test):
"as described in
http://tox.readthedocs.org/en/latest/example/basic.html?highlight=setuptools#integration-with-setuptools-distribute-test-commands"
[...]
def run_tests(self):
if self.distribution.install_requires:
self.distribution.fetch_build_eggs(
self.distribution.install_requires)
if self.distribution.tox_requires:
self.distribution.fetch_build_eggs(self.distribution.tox_requires)
# import here, cause outside the eggs aren't loaded
import tox
import shlex
args = self.tox_args
if args:
args = shlex.split(self.tox_args)
else:
args = ""
errno = tox.cmdline(args=args)
sys.exit(errno)
entry_points ={}
distutils_ext = {'distutils.setup_keywords': [
"tox_requires = setuptools.dist:check_requirements", ]
}
entry_points.update(distutils_ext)
setup(
install_requires=['six', 'numpy', 'matplotlib', 'scipy', 'astropy>=1',
'Pillow', ],
cmdclass={
'test': PyTest, # this is to run python setup.py test
'tox': Tox,
},
# list of packages and data
packages=find_packages(),
# tests
tests_require=['pytest', 'pytest-cov'],
tox_requires=['tox'],
# other keywords, mostly metadata
)
If I run python setup.py sdist, I get a warning at the beginning:
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'tox_requires'
warnings.warn(msg)
but then the sdist works fine and it creates a tar.gz file that I can use to install my package.
But if I run it a second time, it starts with (it's the beginning of Pillow building):
warning: no previously-included files found matching '.editorconfig'
Building using 4 processes
_imaging.c: In function ‘getink’:
and begins building all the required packages into .eggs directory.
If I remove the *egg-info directory I can rerun the command.
If I comment out the tox_requires=[...] line, I can build the sdist as many times as I want.
Now according to the setuptools documentation the command above should be the correct way to run add new arguments to the setup function.
As per the subtitle, the problem is not only with sdist but it's probably due to my non-understanding on how setuptools and requirements work.
If I run python setup.py tox in a place without tox already installed I get, after the installation of some testing package it should not install (namely pytest and pytest-cov):
Traceback (most recent call last):
[...]
File "/usr/lib/python2.7/dist-packages/setuptools/command/test.py", line 127, in with_project_on_sys_path
func()
File "setup.py", line 65, in run_tests
if self.distribution.tox_requires:
AttributeError: Distribution instance has no attribute 'tox_requires'
[Update] The tox_requires also confuse very badly pip during installation. If it is commented out I can install the package without any issue; otherwise it begins to compile the source of the packages and it systematically fails because it doesn't find numpy while building stuff like scipy
How can I get setuptools to recognize and properly use tox_requires?
Once this issue is fixed, I think that I can get rid of the spurious installations here doing a better job at implementing the Tox class, maybe overriding more things from test or deriving it directly from Command
Complete (working) solution described below consist of 8 files (incl. short
README.rst) and has in total 43 lines of code. This is less then code in your
original question.
Despite of being so short, it supports many development and testing scenarios
in very convenient way.
Anyway, it does not answer exactly your question, but I am sure, it fulfils the
requirements which were behind it.
Three lines long setup.py
Technically it may be possible to put test command including tox automation into your setup.py, however, the result may be very messy and difficult to understand.
The same result can be achieved in much simpler way:
for developer assume:
using git
having tox installed into system
for package user:
there are no special requirements to install the resulting package
(optional) if you want your users to test the package by single command and keep test reports collected in central server:
install devpi-server and give your users access to it
ask your users to install $ pip install devpi
The solution builds on following tools and packages:
pbr: simplify package creation incl. versioning via git tags and
creation of AUTHORS and ChangeLog from git commit messages.
pytest: excelent testing framework, but any other framework can
be used instead of it.
tox: excellent build and test automation tool.
coverage: tools to measure test coverage (working simpler then
pytest-cov)
Optionally you may also use:
devpi-server: private PyPi server with password protected
access. Allows simple testing and provides test reports collection.
devpi: tool similar to pip. Apart from installation also supports
running tox defined tests (install, run tests, publish reports in on step).
Authoring the package
Create new project directory and initialize git:
$ mkdir francesco
$ cd francesco
$ git init
Create a package or module
Here we create single module francesco, but the same works for more
modules or packages.
francesco.py
def main():
print("Hi, it is me, Francesco, keeping things simple.")
requirements.txt
Create list of packages for actual installation of the package:
six
test_requirements.txt
Define packages required for testing:
pytest
coverage
tests/test_it.py
Initiate the test suite:
from francesco import main
def test_this():
main()
print("All seems fine to me")
assert True
setup.py
Did you dream of stupid simple setup.py? Here it goes:
from setuptools import setup
setup(setup_requires=["pbr"], pbr=True)
setup.cfg
Metadata belong to configuration file:
[metadata]
name = francesco
author = Francesco Montesano
author-email = fm#acme.com
summary = Nice and simply installed python module supporting testing in different pythons
description-file = README.rst
[files]
modules=francesco
[entry_points]
console_scripts =
francesco = francesco:main
tox.ini
To configure tox automated builds and tests:
[tox]
envlist = py27, py34
[testenv]
commands =
coverage run --source francesco -m pytest -sv tests
coverage report
coverage html
deps =
-rtest_requirements.txt
README.rst
Never forget README.rst:
===========================================
Complex package with 3 line long `setup.py`
===========================================
Can we keep`setup.py` simple and still support automated testing?
...
tox: build sdist and run tests in all supported python versions
Being in the project directory root, just run single command tox:
$ tox
GLOB sdist-make: /home/javl/sandbox/setuppy/setup.py
py27 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py27 runtests: PYTHONHASHSEED='2409409075'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile:
collecting ... collected 1 items
tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED
=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name Stmts Miss Cover
----------------------------------
francesco.py 2 0 100%
py27 runtests: commands[2] | coverage html
py34 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py34 runtests: PYTHONHASHSEED='2409409075'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile:
collecting ... collected 1 items
tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED
=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name Stmts Miss Cover
----------------------------------
francesco.py 2 0 100%
py34 runtests: commands[2] | coverage html
___________________________________ summary ____________________________________
py27: commands succeeded
py34: commands succeeded
congratulations :)
Getting the sdist
ls .tox/dist
francesco-0.0.0.zip
Developing in Python 2.7 virtualenv
Activate Python 2.7 virtualenv
$ source .tox/py27/bin/activate
Run tests
(py27) $ py.test -sv tests
==============================================================================================
test session starts
===============================================================================================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
-- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7 cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: collected 1 items
tests/test_it.py::test_this Hi, it is me, Francesco, keeping things
simple. All seems fine to me PASSED
============================================================================================
1 passed in 0.01 seconds
============================================================================================
Measure test coverage
(py27)$ coverage run --source francesco -m pytest -sv tests
.....
(py27)$ coverage report
Name Stmts Miss Cover
----------------------------------
francesco.py 2 0 100%
View coverage report in web browser
(py27)$ coverage html
(py27)$ firefox htmlcov/index.html
Release new package version
(optional) Install local devpi-server
The installation of devpi-server is not covered here, but is very
simple, especially, if you install only to your local machine for your
personal testing.
Commit source code, assign version tag
Make sure, all your source code is commited.
Assing version tag:
$ git tag -a 0.1
Rerun the tests by tox and build sdist
Make sure, you have deactivated virtualenv (otherwise it conflicts with
tox):
(py27)$ deactivate
Run the tox:
$ tox
.....
...it builds as usual, may fail, if you have forgotten to commit some changes or files...
Find the sdist for new version of your package:
$ ls .tox/dist/francesco-0.1.0.
.tox/dist/francesco-0.1.0.zip
You are done. You may distribute your new tested versions of your package
to users as usually.
(optional) Upload the sdist to devpi-server and test it locally
Following steps assume, you have devpi-server installed and running.
$ devpi login javl
...enter your password...
$ devpi upload .tox/dist/francesco-0.1.0.zip
Test the package in clean environment
(deactivate virtualenv if active) :
$ cd /tmp
$ mkdir testing
$ cd testing
$ devpi test francesco
received http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
unpacking /tmp/devpi-test6/downloads/francesco-0.1.0.zip to /tmp/devpi-test6/zip
/tmp/devpi-test6/zip/francesco-0.1.0$ tox --installpkg /tmp/devpi-test6/downloads/francesco-0.1.0.zip -i ALL=http://localhost:3141/javl/dev/+simple/ --recreate --result-json /tmp/devpi-test6/zip/toxreport.json
-c /tmp/devpi-test6/zip/francesco-0.1.0/tox.ini
py27 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27
py27 installdeps: -rtest_requirements.txt
py27 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py27 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py27 runtests: PYTHONHASHSEED='3916044270'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items
tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED
=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name Stmts Miss Cover
----------------------------------
francesco.py 2 0 100%
py27 runtests: commands[2] | coverage html
py34 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34
py34 installdeps: -rtest_requirements.txt
py34 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py34 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py34 runtests: PYTHONHASHSEED='3916044270'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items
tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED
=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name Stmts Miss Cover
----------------------------------
francesco.py 2 0 100%
py34 runtests: commands[2] | coverage html
____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
py27: commands succeeded
py34: commands succeeded
congratulations :)
wrote json report at: /tmp/devpi-test6/zip/toxreport.json
posting tox result data to http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
successfully posted tox result data
You may check the test results in web browser:
$ firefox http://localhost:3141
then search for "francesco" package, click the package name, find in table
column named "tox results", click there to show environment set up and test
results.
Let your users test the package
Let's assume, your devpi-server is running and your user has access to it.
The user shall have devpi command installed:
$ pip install devpi
(note, this tool is not installing anything from the devpi-server)
Help your user to gain access to devpi-server (not covering here).
Then the user just runs the test:
$ devpi test francesco
After the test is run (it is automatically using tox, but user does not have
to care about that), you will find test results on the same place on devpi web
interface as you found yours before.