I am having some issues running pytest through tox due to it not recognising submodules.
The error that keeps coming up is ModuleNotFoundError: No module named 'app.api'
I'm not really sure how to fix this because it is working fine when I install the package and run pytest myself.
Any help would be greatly appreciated.
My directory structure:
|- project
|- src
|- app
|- api
|- core
|- tests
|- setup.py
|- tox.ini
|- setup.cfg
tox.ini
[tox]
minversion = 3.27.0
envlist = py311
isolated_build = True
[testenv]
setenv =
PYTHONPATH = {toxinidir}
deps =
-r{toxinidir}/requirements_dev.txt
-r{toxinidir}/requirements.txt
commands =
pytest --basetemp={envtmpdir}
setup.py
from setuptools import find_packages, setup
if __name__ == "__main__":
setup(packages=find_packages(exclude=["tests"]))
setup.cfg
[metadata]
name = app
version = 0.0.1
[options]
packages = app
python_requires = >=3.11
package_dir =
=src
zip_safe = no
[options.extras_require]
testing =
pytest>=6.0
pytest-cov>=2.0
tox>=3.24
So after spending some time (reluctantly) reading the tox docs I discovered that the package is built as defined in the pyproject.toml where I had it set in setup.cfg
The text I had to add to pyproject.toml was this:
[tool.setuptools.packages.find]
where = ["src"]
include = ["app*"]
exclude = ["app.tests*"]
Tox now builds the app all good.
Related
For traditional Python projects with a setup.py, there are various ways of ensuring that the version string does not have to be repeated throughout the code base. See PyPA's guide on "Single-sourcing the package version" for a list of recommendations.
Many are trying to move away from setup.py to setup.cfg (probably under the influence of PEP517 and PEP518; setup.py was mostly used declaratively anyway, and when there was logic in setup.py, it was probably for the worse.) This means that most the suggestions won't work anymore since setup.cfg cannot contain "code".
How can I single-source the package version for Python projects that use setup.cfg?
There are a couple of ways to do this (see below for the project structure used in these examples):
1.
setup.cfg
[metadata]
version = 1.2.3.dev4
src/my_top_level_package/__init__.py
import importlib.metadata
__version__ = importlib.metadata.version('MyProject')
2.
setup.cfg
[metadata]
version = file: VERSION.txt
VERSION.txt
1.2.3.dev4
src/my_top_level_package/__init__.py
import importlib.metadata
__version__ = importlib.metadata.version('MyProject')
3.
setup.cfg
[metadata]
version = attr: my_top_level_package.__version__
src/my_top_level_package/__init__.py
__version__ = '1.2.3.dev4'
And more...
There are probably other ways to do this, by playing with different combinatons.
References:
https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html
https://docs.python.org/3/library/importlib.metadata.html
Structure assumed in the previous examples is as follows...
MyProject
├── setup.cfg
├── setup.py
└── src
└── my_top_level_package
└── __init__.py
setup.py
#!/usr/bin/env python3
import setuptools
if __name__ == '__main__':
setuptools.setup(
# see 'setup.cfg'
)
setup.cfg
[metadata]
name = MyProject
# See above for the value of 'version = ...'
[options]
package_dir =
= src
packages = find:
[options.packages.find]
where = src
$ cd path/to/MyProject
$ python3 setup.py --version
1.2.3.dev4
$ python3 -m pip install .
# ...
$ python3 -c 'import my_top_level_package; print(my_top_level_package.__version__)'
1.2.3.dev4
$ python3 -V
Python 3.6.9
$ python3 -m pip list
Package Version
------------- ----------
MyProject 1.2.3.dev4
pip 20.0.2
pkg-resources 0.0.0
setuptools 45.2.0
wheel 0.34.2
zipp 3.0.0
If I have a tree that looks like:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── setup.py
├── env
└── setup.py
Is there a way to include the nested setup.py in the install for the top setup.py? I want to avoid this:
pip install -e . ; cd project/package ; pip install -e .
The solution is to have two separate projects: a main project (usually an application) and a sub-project (usually a library). The main application has a dependency to the library.
Tree structure and setup.py
The main project can have the following structure:
your_app/
|-- setup.py
ˋ-- src/
ˋ-- your_app/
|-- __init__.py
|-- module1.py
ˋ-- ...
The setup.py of your application can be:
from setuptools import find_packages
from setuptools import setup
setup(
name='Your-App',
version='0.1.0',
install_requires=['Your-Library'],
packages=find_packages('src'),
package_dir={'': 'src'},
url='https://github.com/your-name/your_app',
license='MIT',
author='Your NAME',
author_email='your#email.com',
description='Your main project'
)
You can notice that:
The name of your application can be slightly different to the name of your package;
This package has a dependency to "Your-Library", defined below;
You can put your source in the src directory, but it is optional. A lot of project have none.
The sub-project can have the following structure:
your_library/
|-- setup.py
ˋ-- src/
ˋ-- your_library/
|-- __init__.py
|-- lib1.py
ˋ-- ...
The setup of you library can be:
from setuptools import find_packages
from setuptools import setup
setup(
name='Your-Library',
version='0.1.0',
packages=find_packages('src'),
package_dir={'': 'src'},
url='https://github.com/your-name/your_library',
license='MIT',
author='Your NAME',
author_email='your#email.com',
description='Your sub-project'
)
Putting all things together
Create a virtualenv for your application and activate it
Go in the your_library/ directory and run:
pip install -e .
Then, go in your_app/ directory and run:
pip install -e .
You are now ready to code. Have fun!
See the Hitchhiker's Guide to Python: “Structuring Your Project”.
I have a python project that I want to distribute. I read multiple tutorials on how to write my setup.py file and how to install the produced wheel: sample project example, setup.py tutorial, wheel doc, wheel install or wheel install.
The structure of my project is:
project_name
|_ lib
|_ project_folder
|_ py modules
|_ test
|_ setup.py
|_README.rst
I build my wheel like this python setup.py bdist_wheel and then I take the produced wheel into another folder outside my project and do pip install my_wheel. I tried also pip install --no-index --find-links=my_wheel project_name
The problem is that when I look into my python site-packages folder, instead of having:
python folders
project_name
project_name-2.0.0.dist-info
the project_name folder is broken into lib and test:
python folders
lib
project_name-2.0.0.dist-info
test
I don't understand why my project_name isn't like the other python folders, grouped. Can someone help me understand better?
setup.py:
from setuptools import setup, find_packages
from codecs import open
from os import path
root_folder = path.abspath(path.dirname(__file__))
with open(path.join(root_folder, "README.rst"), encoding="utf-8") as f:
long_description = f.read()
setup(
name = "project",
version = "2.0.0",
description = "My project is cool",
long_description = long_description,
packages = find_packages(),
include_package_data = True
)
find_packages() determines packages by the __init__.py files. It looks like your lib and tests directories have __init__.py files in them.
Neither your lib or tests directories are packages, remove the __init__.py files from those. That way find_packages() will only include project_folder() in the resulting distribution (source, binary or wheel).
The reusable app docs (https://docs.djangoproject.com/en/1.9/intro/reusable-apps/) tells you to list template and static files in MANIFEST.in, but it doesn't look like python setup.py bdist_wheel looks at that file at all.
I've seen references to data_files but those files go in directories relative to the python installation (sys.prefix) and not the package installation (and sys.prefix isn't uniformly related to site-packages across systems).
Am I correct in assuming that myapp/templates/myapp/foo.html should end up in .../site-packages/myapp/templates/myapp/foo.html and similarly for static files, and that the user needs to run a manage.py collectstatic after pip install myapp?
Update (example):
The following structure:
(build2) go|c:\srv\tmp\myapp> tree
.
|-- MANIFEST.in
|-- myapp
| |-- static
| | `-- myapp
| | `-- foo.css
| |-- templates
| | `-- myapp
| | `-- foo.html
| |-- urls.py
| `-- views.py
`-- setup.py
5 directories, 6 files
setup.py
import setuptools
from distutils.core import setup
setup(
name='myapp',
version='0.1.0',
packages=['myapp']
)
MANIFEST.in
recursive-include myapp/templates *
recursive-include myapp/static *
running python setup.py sdist and python setup.py bdist_wheel creates the following files bin myapp/dist:
2016-06-18 13:47 2,073 myapp-0.1.0-py2-none-any.whl
2016-06-18 13:46 2,493 myapp-0.1.0.zip
if you look inside the .zip file, you'll find the templates and static folders, if you rename the .whl file to .zip and look inside it, the directories are not included.
Update 2 (solution):
Changing the MANIFEST.in file to
recursive-include myapp *
and setup.py to
from setuptools import find_packages, setup
setup(
name='myapp',
version='0.1.0',
include_package_data=True,
packages=['myapp'],
zip_safe=False,
)
then running python setup.py bdist_wheel will create a .whl file that installs myapp/templates and myapp/static in the expected places.
The MANIFEST.in file needs to be changed to:
recursive-include myapp *
This includes everything under myapp/myapp with the correct paths. In particular, this includes myapp/myapp/templates, which is necessary.
The declaration above also includes myapp/myapp/static which could be useful if you plan to run manage.py collectstatic after installing the .whl file.
In setup.py, the setup function needs to be imported from setuptools (and not distutils), i.e.:
from setuptools import find_packages, setup
setup(
name='myapp',
version='0.1.0',
include_package_data=True,
packages=['myapp'],
zip_safe=False,
)
When you now run python setup.py bdist_wheel it will create a .whl file that installs myapp/templates and myapp/static in the expected places.
I followed the instructions here to publish a simple project onto PyPI. I succeeded after some struggles.
However, after I installed the project thru pip install my-project, I cannot import it in Python like this: ImportError: No module named my-project...
Is it because of my file structure? It is like this:
My-project-folder
|- setup.py
|- test.py
|- README
|- my-project
~~|- file.py
~~|- file2.py
You're missing __init__.py inside my-project!