We Keep Coding

How do I build multiple wheel files from a single setup.py?

In my project, I have a single setup.py file that builds multiple modules using the following namespace pattern:
from setuptools import setup
setup(name="testmoduleserver",
packages=["testmodule.server","testmodule.shared"],
namespace_packages=["testmodule"])
setup(name="testmoduleclient",
packages=["testmodule.client","testmodule.shared"],
namespace_packages=["testmodule"])
I am trying to build wheel files for both packages. However, when I do:
python -m pip wheel .
It only ever builds the package for one of the definitions.
Why does only one package get built?

You cannot call setuptools.setup() more than once in your setup.py, even if you want to create several packages out of one codebase.
Instead you need to separate everything out into separate namespace packages, and have one setup.py for each (they all can reside in one Git repository!):
testmodule/
testmodule-client/
setup.py
testmodule/
client/
__init__.py
testmodule-server/
setup.py
testmodule/
server/
__init__.py
testmodule-shared/
setup.py
testmodule/
shared/
__init__.py
And each setup.py contains something along the lines
from setuptools import setup
setup(
name='testmodule-client',
packages=['testmodule.client'],
install_requires=['testmodule-shared'],
...
)
and
from setuptools import setup
setup(
name='testmodule-server',
packages=['testmodule.server'],
install_requires=['testmodule-shared'],
...
)
and
from setuptools import setup
setup(
name='testmodule-shared',
packages=['testmodule.shared'],
...
)
To build all three wheels you then run
pip wheel testmodule-client
pip wheel testmodule-server
pip wheel testmodule-shared

Install modules with setup.py and setup.cfg

I have the following directory structure:
/modules/
/modules/setup.py
/modules/setup.cfg
/modules/module1/
/modules/module1/__init__.py
/modules/module1/tool1/__init__.py
/modules/module1/tool1/tool1.py
/modules/module2/
/modules/module2/__init__.py
/modules/module2/tool2/__init__.py
/modules/module2/tool2/tool2.py
/modules/module2/tool3/__init__.py
/modules/module2/tool3/tool3.py
And I want to install these modules using setup.py and setup.cfg and import them later on like this:
import my_project.module1.tool1
import my_project.module2.tool2
import my_project.module2.tool3
These are my installation files:
setup.py
import setuptools
setuptools.setup(
setup_requires=['paramiko>=2.0.1'],
paramiko=True)
setup.cfg
[metadata]
name = my_project
summary = my project modules
[files]
packages =
module1
module2
It fails when I try to install the packages:
/modules# pip install -e .
Obtaining file:///modules
Installing collected packages: UNKNOWN
Found existing installation: UNKNOWN 0.0.0
Can't uninstall 'UNKNOWN'. No files were found to uninstall.
Running setup.py develop for UNKNOWN
Successfully installed UNKNOWN

Try using the find_packages function:
from setuptools import setup, find_packages
...
setup(
...
packages=find_packages(),
...

Tox can't copy non-python file while installing the module

This is the tree structure of the module I'm writing the setup.py file for:
ls .
LICENSE
README.md
bin
examples
module
scratch
setup.py
tests
tox.ini
I configured my setup.py as follows:
from setuptools import setup, find_packages
setup(
name="package_name",
version="0.1",
packages=find_packages(),
install_requires=[
# [...]
],
extras_require={
# [...]
},
tests_require={
'pytest',
'doctest'
},
scripts=['bin/bootstrap'],
data_files=[
('license', ['LICENSE']),
],
# [...]
# could also include long_description, download_url, classifiers, etc.
)
If I install the package from my python environment (also a virtualenv)
pip install .
the LICENSE file gets correctly installed.
But running tox:
[tox]
envlist = py27, py35
[testenv]
deps =
pytest
git+https://github.com/djc/couchdb-python
docopt
commands = py.test \
{posargs}
I get this error:
running install_data
creating build/bdist.macosx-10.11-x86_64/wheel/leafline-0.1.data
creating build/bdist.macosx-10.11-x86_64/wheel/leafline-0.1.data/data
creating build/bdist.macosx-10.11-x86_64/wheel/leafline-0.1.data/data/license
error: can't copy 'LICENSE': doesn't exist or not a regular file
Removing the data_files part from the setup.py makes tox running correctly.

Your issue here is that setuptools is not able to find the 'LICENSE' file in the files that have been included for building the source distribution. You have 2 options, to tell setuptools to include that file (both have been pointed to here):
Add a MANIFEST.in file (like https://github.com/pypa/sampleproject/)
Use include_package_data=True in your setup.py file.
Using MANIFEST.in is often simpler and easier to verify due to https://pypi.org/project/check-manifest/, making it possible to use automation to verify that things are indeed correct (if you use a VCS like Git or SVN).
pip install . builds a wheel using python setup.py bdist_wheel which is installed by simply unpacking it appropriately, as defined in the Wheel Specification: https://www.python.org/dev/peps/pep-0427/
tox builds a source distribution using python setup.py sdist, which is then unpacked and installed using python setup.py install.
That might be a reason for the difference in behavior for you.

I have some resource files inside my packages which I use during the execution. To make setup store them in a package with python code, I use include_package_data=True and I access them using importlib.resources. You can use backport for an older Python version than 3.7 or another library.
Before each release I have a script which verifies, that all files I need are placed inside a bdist wheel to be sure that everything is on the place.

Why does setuptools not understand git+https URLs?

According to Dependency section in the setuptools manual git repository URLs can be specified in the dependency_links argument to setup with git+URL. Yet,
cd /tmp
mkdir py-test
cd py-test
touch __init__.py
and creation of a setup.py file with
from setuptools import setup, find_packages
from pkg_resources import parse_version
setup(
name = "py-test",
version = "1.0",
packages = ["."],
dependency_links = [
"git+https://github.com/wxWidgets/wxPython.git"
],
install_requires = ["wxPython"],
)
causes the error Download error on git+https://github.com/wxWidgets/wxPython.git: unknown url type: git+https -- Some packages may not be found! when I run python setup.py build && sudo setup.py install.
The installation of the package python-setuptools-git doesn't help.
I'm using setuptools 18.2 with python 2.7 on Ubuntu 15.04.

From the setuptools docs:
In the case of a VCS checkout, you should also append #egg=project-version in order to identify for what package that checkout should be used
So the fix is just to append the #egg=wxPython fragment onto the end:
dependency_links = [
"git+https://github.com/wxWidgets/wxPython.git#egg=wxPython"
]

Pip installing from github installs only __init__.py file

I'm trying to grasp the git(hub) way of managing software. I have a repository:
https://github.com/pythonishvili/django-inguri
And I try to pip install it with this command
pip install git+git://github.com/pythonishvili/django-inguri.git
The response I get:
Downloading/unpacking git+git://github.com/pythonishvili/django-inguri.git
Cloning git://github.com/pythonishvili/django-inguri.git to /tmp/pip-bv5r89-build
Running setup.py egg_info for package from git+git://github.com/pythonishvili/django-inguri.git
Installing collected packages: inguri
Running setup.py install for inguri
Successfully installed inguri
Cleaning up...
But installation went clearly wrong because all I get in my virtualenv (/home/username/.virtualenvs/envname/lib/python2.7/site-packages/inguri) are two files:
__init__.py
__init__.pyc
What did I do wrong? How do I make this work?

I believe you need to add all the subdirectories of your project to the packages option of your setup.py file. Right now, you have just the outermost directory - inguri. You would need to add inguri.ads, inguri.ads.migrations and so forth (as they contain .py files too which you want to include in your distribution).
You also need to add the following line in your manifest file: recursive-include inguri *