I have 2 similar plugins I wrote:
import pytest
#pytest.fixture
def pyplug<ID>_fixture():
pass
where <ID> is replaced with the number I gave the plugin, i.e. '0', '1'...
I created a setup.py for each plugin:
from setuptools import setup
setup(
name='pyplug<ID>',
packages=['pyplug<ID>'],
entry_points={'pytest11': ['pkg = pyplug<ID>.plugin']}
)
and inside each package, I created another dir (same name) with plugin.py file, containing the fixture described earlier.
This is my dir tree AFTER I run python setup.py bdist_wheel for each plugin:
src/
├── pyplug0
│ ├── build
│ │ ├── bdist.linux-x86_64
│ │ └── lib
│ │ └── pyplug0
│ │ └── plugin.py
│ ├── dist
│ │ └── pyplug0-0.0.0-py3-none-any.whl
│ ├── pyplug0
│ │ └── plugin.py
│ ├── pyplug0.egg-info
│ │ ├── dependency_links.txt
│ │ ├── entry_points.txt
│ │ ├── PKG-INFO
│ │ ├── SOURCES.txt
│ │ └── top_level.txt
│ └── setup.py
└── pyplug1
├── build
│ ├── bdist.linux-x86_64
│ └── lib
│ └── pyplug1
│ └── plugin.py
├── dist
│ └── pyplug1-0.0.0-py3-none-any.whl
├── pyplug1
│ └── plugin.py
├── pyplug1.egg-info
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ └── top_level.txt
└── setup.py
This is my pip freeze:
attrs==19.3.0
importlib-metadata==1.6.0
more-itertools==8.2.0
packaging==20.3
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.7
pyplug0==0.0.0
pyplug1==0.0.0
pytest==5.4.1
pytest-html==2.1.1
pytest-metadata==1.8.0
pytest-tldr==0.2.1
six==1.14.0
wcwidth==0.1.9
zipp==3.1.0
and this is my test file content:
def test_plugin(pyplug0_fixture):
pass
def test_plugin2(pyplug1_fixture):
pass
The problem is, the plugins are not registered OK.
When I install only one (doesn't matter which) and test it, it's OK. When I install both, one is overridden by the other, which makes no sense IMO, since they have different names (for both the fixture, and the plugin)
I run py3.6.10 on Ubuntu16.04
Found it.
Both setup.py had the same entry point:
entry_points={"pytest11": ["KEY = VAL"]}
the VAL was different, but I kept using the same KEY, so the latest always won
Related
I am playing around with Sphinx 4.4.0 in a test project. I use sphinx.ext.autodoc and sphinx-apidoc.
But not all py files are recognized. It seams like that files with a starting _ in filename are ignored (e.g. _mypackage.py). But i am not sure if that character is the cause or something else.
Are you aware of an option to modify that behavior?
This is the project structure
sphinx_versuch
├── docs
│ ├── make.bat
│ ├── Makefile
│ └── source
│ ├── conf.py
│ ├── index.rst
│ ├── modules.rst
│ ├── mypackage.rst
│ ├── _static
│ └── _templates
├── LICENSE
├── README.md
├── src
│ ├── mypackage
│ │ ├── a.py
│ │ ├── b.py
│ │ ├── __init__.py
│ │ └── _mypackage.py
│ ├── setup.cfg
│ └── setup.py
└── tests
├── __init__.py
├── test_a.py
├── test_b.py
└── test_mypackage.py
The option --private or -P for sphinx-apidoc make sure that private files (which start with an underscore by convention) are included also.
See also
https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html#cmdoption-sphinx-apidoc-P
I'm trying to run my tests using python -m pytest but I get an error that
ModuleNotFoundError: No module named 'sample'
When using nosetests or anything else it works fine, but when trying to use pytest, it doesn't work.
My tree looks like below, do you have any advice why it doesn't work?
├── LICENSE.txt
├── README.md
├── data
│ └── data_file
├── exported_register.csv
├── pyproject.toml
├── requirements.txt
├── setup.cfg
├── setup.py
├── src
│ └── sample
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-39.pyc
│ │ ├── dziennik.cpython-39.pyc
│ │ ├── przedmiot.cpython-39.pyc
│ │ ├── simple.cpython-39.pyc
│ │ └── uczen.cpython-39.pyc
│ ├── dziennik.py
│ ├── package_data.dat
│ ├── przedmiot.py
│ ├── simple.py
│ └── uczen.py
├── tests
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-39.pyc
│ │ ├── test_ASSERTPY_uczen.cpython-39-pytest-6.2.1.pyc
│ │ ├── test_ASSERTPY_uczen.cpython-39-pytest-6.2.5.pyc
│ │ ├── test_ASSERTPY_uczen.cpython-39.pyc
│ │ ├── test_PYHAMCREST_uczen.cpython-39-pytest-6.2.1.pyc
│ │ ├── test_PYHAMCREST_uczen.cpython-39-pytest-6.2.5.pyc
│ │ ├── test_PYHAMCREST_uczen.cpython-39.pyc
│ │ ├── test_UNITTEST_register.cpython-39-pytest-6.2.1.pyc
│ │ ├── test_UNITTEST_register.cpython-39-pytest-6.2.5.pyc
│ │ ├── test_UNITTEST_register.cpython-39.pyc
│ │ ├── test_UNITTEST_uczen.cpython-39-pytest-6.2.1.pyc
│ │ ├── test_UNITTEST_uczen.cpython-39-pytest-6.2.5.pyc
│ │ ├── test_UNITTEST_uczen.cpython-39.pyc
│ │ ├── test_simple.cpython-39-pytest-6.2.1.pyc
│ │ ├── test_simple.cpython-39-pytest-6.2.5.pyc
│ │ └── test_simple.cpython-39.pyc
│ ├── test_ASSERTPY_uczen.py
│ ├── test_PYHAMCREST_uczen.py
│ ├── test_UNITTEST_register.py
│ ├── test_UNITTEST_uczen.py
│ └── test_simple.py
└── tox.ini
When you run pytest with python -m pytest it uses the current directory as it its working dir, which doesn't contain the sample module (located inside ./src). The way I deal with this is I have a conftest.py inside my tests directory where I add my source dir to python path something like this:
import sys
from pathlib import Path
source_path = Path(__file__).parents[1].joinpath("src").resolve()
sys.path.append(str(source_path))
I've recently started using pytorch and have had similar problems. Couple steps come to mind:
How are you writing the .py file that contains the tests? It may simply be that you need to change up how you import sample within the unit test file. I would expect that you need something like import src.sample.simple. In other words, could be just a pathing issue.
Try a (much) simpler folder structure and try again. If that doesn't work, try to just copy an example of a simple scheme that someone has posted. That is, just get python -m pytest to run somehow, then start slowly adding the complexities of your project.
My python project installs via setup.py. The project structure looks like:
├── Makefile
├── README.rst
├── circle.yml
├── docs
│ ├── Makefile
│ ├── conf.py
│ ├── deps.txt
│ ├── guide_installation.rst
│ ├── guide_model.rst
│ ├── guide_transliteration.rst
│ ├── index.rst
│ ├── make.bat
│ └── module_trans.rst
├── indictrans
│ ├── __init__.py
│ ├── _decode
│ ├── _utils
│ ├── base.py
│ ├── iso_code_transformer.py
│ ├── libindic_
│ ├── mappings
│ ├── models
│ ├── polyglot_tokenizer
│ ├── script_transliterate.py
│ ├── test.py
│ ├── tests
│ ├── transliterator.py
│ ├── trunk
│ └── unicode_marks.py
├── requirements.txt
├── setup.cfg
├── setup.py
├── test-requirements.txt
└── tox.ini
where the subfolder indictrans/models looks like
├── ben-eng
│ ├── classes.npy
│ ├── coef.npy
│ ├── intercept_final.npy
│ ├── intercept_init.npy
│ ├── intercept_trans.npy
│ └── sparse.vec
├── ben-guj
│ ├── classes.npy
│ ├── coef.npy
│ ├── intercept_final.npy
│ ├── intercept_init.npy
│ ├── intercept_trans.npy
│ └── sparse.vec
so I have .npy and .vec files to be included in the project.
In my setup.py I'm trying to explicitly include this folder models via the include_package_data directive like:
setup(
setup_requires=['pbr'],
pbr=True,
packages=find_packages(),
include_package_data=True,
package_data={'models': ['*.npy','*.vec']},
ext_modules=cythonize(extensions)
)
and in the setup.cfg I have
[files]
packages =
indictrans
but running python setup.py install does not copy the models folder to the installation folder /usr/local/lib/python2.7/dist-packages/indictrans/.
If I print the it is the output of the find_packages I get
['indictrans', 'indictrans.tests', 'indictrans.libindic_', 'indictrans._utils', 'indictrans._decode', 'indictrans.polyglot_tokenizer', 'indictrans.models', 'indictrans.trunk', 'indictrans.libindic_.utils', 'indictrans.libindic_.soundex', 'indictrans.libindic_.utils.tests', 'indictrans.libindic_.soundex.utils', 'indictrans.libindic_.soundex.tests', 'indictrans.libindic_.soundex.utils.tests', 'indictrans.polyglot_tokenizer.tests', 'indictrans.trunk.tests']
so I will assume that indictrans/models would be included, but it is not.
Add include_package_data=True to your setup-function (you already did that).
Create a file MANIFEST.in in the same directory as setup.py
MANIFEST.in can look as follows:
include indictrans/models/ben-eng/*
include indictrans/models/ben-guj/*
You don't need setup.cfg for doing this.
Source: This great writeup of python packaging
EDIT about recursive-include:
According to the documentation this should also work:
recursive-include indictrans/models *.npy *.vec
include_package_data=True requires MANIFEST.in.
To include data for the module indictrans.models you have to provide the full name:
package_data={'indictrans.models': ['*.npy','*.vec']},
I'm trying to use Django + Haystack + Elasticsearch.
So I installed Elasticsearch 2.4 with pip becouse Django gave me errors about that it can't import Elasticsearch. Now it can and I can run ./manage.py rebuild_index in my Django project and it gives this output:
Indexing 23 journal entrys
GET /haystack/_mapping [status:404 request:0.006s]
but only if I somehove run elasticsearch. So I installed elsasticsearch2 from AUR packages as well and run that. But as I suspected when I try to get all documents by running: curl -X GET "localhost:9200/_cat/indices?v" which returns:
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open haystack 5 1 0 0 795b 795b
If I understand correctly it is empty.
I went where pip installs packages(/usr/lib/python3.6/site-packages) and found two folders related to Elasticsearch:
elasticsearch
├── client
│ ├── cat.py
│ ├── cluster.py
│ ├── indices.py
│ ├── ingest.py
│ ├── __init__.py
│ ├── nodes.py
│ ├── __pycache__
│ │ ├── cat.cpython-36.pyc
│ │ ├── cluster.cpython-36.pyc
│ │ ├── indices.cpython-36.pyc
│ │ ├── ingest.cpython-36.pyc
│ │ ├── __init__.cpython-36.pyc
│ │ ├── nodes.cpython-36.pyc
│ │ ├── snapshot.cpython-36.pyc
│ │ ├── tasks.cpython-36.pyc
│ │ └── utils.cpython-36.pyc
│ ├── snapshot.py
│ ├── tasks.py
│ └── utils.py
├── compat.py
├── connection
│ ├── base.py
│ ├── http_requests.py
│ ├── http_urllib3.py
│ ├── __init__.py
│ ├── pooling.py
│ └── __pycache__
│ ├── base.cpython-36.pyc
│ ├── http_requests.cpython-36.pyc
│ ├── http_urllib3.cpython-36.pyc
│ ├── __init__.cpython-36.pyc
│ └── pooling.cpython-36.pyc
├── connection_pool.py
├── exceptions.py
├── helpers
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-36.pyc
│ │ └── test.cpython-36.pyc
│ └── test.py
├── __init__.py
├── __pycache__
│ ├── compat.cpython-36.pyc
│ ├── connection_pool.cpython-36.pyc
│ ├── exceptions.cpython-36.pyc
│ ├── __init__.cpython-36.pyc
│ ├── serializer.cpython-36.pyc
│ └── transport.cpython-36.pyc
├── serializer.py
└── transport.py
elasticsearch-2.4.1.dist-info
├── DESCRIPTION.rst
├── INSTALLER
├── METADATA
├── metadata.json
├── pbr.json
├── RECORD
├── top_level.txt
└── WHEEL
I don't see a start_elasticsearch.sh or bin/elasticsearch so how can I start it?
I want to make python package with C extensions. I want this to be done with cython. My structure is:
.
├── build
│ ├── lib.linux-i686-2.7
│ │ └── pyA13SOM
│ │ ├── cython
│ │ │ └── spi.so
│ │ └── __init__.py
│ └── temp.linux-i686-2.7
│ └── pyA13SOM
│ └── cython
│ ├── my_test.o
│ └── spi.o
├── CHANGES.txt
├── Makefile
├── MANIFEST
├── pyA13SOM
│ ├── cython
│ │ ├── clibraries
│ │ │ └── spi_test.c
│ │ ├── __init__.py
│ │ ├── __init__.pyc
│ │ ├── spi.c
│ │ ├── spi.pyx
│ │ └── spi.so
│ ├── gpio
│ │ ├── gpio.c
│ │ ├── gpio_lib.c
│ │ ├── gpio_lib.h
│ │ ├── __init__.py
│ │ └── __init__.pyc
│ ├── i2c
│ │ ├── i2c.c
│ │ ├── i2c_lib.c
│ │ ├── i2c_lib.h
│ │ ├── __init__.py
│ │ └── __init__.pyc
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── spi
│ │ ├── __init__.py
│ │ ├── __init__.pyc
│ │ ├── spi.c
│ │ ├── spi_lib.c
│ │ └── spi_lib.h
│ └── utilities
│ └── color.h
├── README.txt
└── setup.py
My setup file is:
from distutils.core import setup
from distutils.core import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
module_gpio = Extension('pyA13SOM.gpio',
sources=['pyA13SOM/gpio/gpio_lib.c', 'pyA13SOM/gpio/gpio.c'])
module_i2c = Extension('pyA13SOM.i2c',
sources=['pyA13SOM/i2c/i2c_lib.c', 'pyA13SOM/i2c/i2c.c'])
module_spi = Extension('pyA13SOM.spi',
define_macros=[('CYTHON_IN_USE', '1')],
sources=['pyA13SOM/spi/spi_lib.c', 'pyA13SOM/spi/spi.c'])
setup(
name='pyA13SOM',
version='0.2.0',
packages=['pyA13SOM'],
# ext_modules=[module_gpio, module_i2c, module_spi],
cmdclass={'build_ext': build_ext},
ext_modules=cythonize("pyA13SOM/cython/*.pyx"),
)
The tree is in ~/mydir/. I go to ~/mydir/ and do: python setup.py install.
Everything in the build process is OK. Next I try to test import. When I import pyA13SOM.cython.spi, it should give me "Hello world" message. And it does.
~/mydir/$ **python -c "import pyA13SOM.cython.spi"**
Test:
Hellowwwwwwwwwww!
But when I do this from another directory:
~/someotherdir/$ **python -c "import pyA13SOM.cython.spi"**
ImportError: No module named cython.spi
Any idea why does this happen?
You might need to include the directory in which your newly built .spi file is located into your $PYTHONPATH. Otherwise python cannot find the file to import it. While you are in ~/mydir/, python searches the local path if I am not mistaken...
Depending on the shell you are using, you can include the ~/mydir/ directory into the pythonpath with the following:
for the bash and sh shells:
PYTHONPATH=$PYTHONPATH:~/mydir/
export $PYTHONPATH
for the csh/tcsh environment:
set PYTHONPATH = ($PYTHONPATH ~/mydir/)
These two commands add the ~/mydir/ temporarily to your $PYTHONPATH. If you want to add the path permanently, you will have to add the above commands to your ~/.bashrc or ~/.tcshrc, respectively.
Hope this helps...