Is there a way to specify optional dependencies for a Python package that should be installed by default from pip but for which an install should not be considered a failure if they cannot be installed?
I know that I can specify install_requires so that the packages will be installed for the 90% of users using OSes that can easily install certain optional dependencies, and I also know I can specify extra_require to specify that users can declare they want a full install to get these features, but I haven't found a way to make a default pip install try to install the packages but not complain if they cannot be installed.
(The particular package I'd like to update the setuptools and setup.py for is called music21 for which 95% of the tools can be run without matplotlib, IPython, scipy, pygame, some obscure audio tools etc. but the package gains extra abilities and speed if these packages are installed, and I'd prefer to let people have these abilities by default but not report errors if they cannot be installed)
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
#classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling said above, this will only work if you publish a source distribution (.tar.gz or .zip). It will not work if you publish your package as a built distribution (.whl).
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
Related
I want to make a distributable package.
And my package depends on some OS package
Here what I want to install:
def install_libmagic():
if sys.platform == 'darwin':
subprocess.run(['brew', 'install', 'libmagic'])
elif sys.platform == 'linux':
subprocess.run(['apt-get', 'update'])
subprocess.run(['apt-get', 'install', '-y', 'libmagic1'])
else:
raise Exception(f'Unknown system: {sys.platform}, can not install libmagic')
I want this code to be executed only when smb call:
pip install mypacakge
I don't want it to be executed when I run: python setup.py bdist_wheel
How can I achieve this?
I tried this:
setup(
...
install_requires=install_libmagic(),
)
Also tried to override install command:
from setuptools.command.install import install
class MyInstall(install):
def run(self):
install_libmagic()
install.run(self)
setup(
...
cmdclass={'install': MyInstall}
)
But the function was executed on python setup.py bdist_wheel, which is not what I am trying to achieve.
I think you're mixing up the behaviors of built distributions (wheels) and source distributions.
If your goal is run some subprocesses at install time, then you can't do this with a built distribution. A built distribution executes no Python code at install time. It only executes setup.py at build time, which is why you're seeing your functions executed when you call python setup.py bdist_wheel.
On the other hand, a source distribution (python setup.py sdist) does execute the setup.py file at both build time and install time (roughly the same as python setup.py install) and would give you the behavior you're looking for.
However, as the comments have already mentioned, this is going to be very fragile and not very user-friendly or portable. What you're describing is really a distro/OS package that contains some Python module, and you'd probably be better off with that instead.
Is there a way to make certain dependencies for a python package optional ? Or conditional on the installation of another dependency failing.
The two cases I have are:
I want to install dependency x AND y. However, in case either of the install fails, the package can probably work alright with just one of them, so installation should complete with a warning.
I want to install dependency x IF the installation of y failed.
You can have conditional dependencies, but not based on the success/failure of the installation of other dependencies.
You can have optional dependencies, but an install will still fail if an optional dependency fails to install.
The simplest way to do have dependencies be optional (i.e. won't fail the main installation if they fail to install) or conditional (based on the success/failure of other packages) is by adding a custom install command that manually shells out to pip to install the individual packages, and checks the result of each call.
In your setup.py:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
# Attempt to install optional dependencies
try:
subprocess.check_call([sys.executable, "-m", "pip", "install", "dependency-y"])
except subprocess.CalledProcessError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "dependency-x"])
# Continue with regular installation
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Note that this will only work if you publish a source distribution (.tar.gz or .zip). It will not work if you publish your package as a built distribution (.whl).
# setup.py
from setuptools import setup
from setuptools.command.install import install
from subprocess import check_call
class CustomInstall(install):
def run(self):
check_call("./build.sh")
install.run(self)
setup(
name='customlib',
packages=['customlib'],
version='0.0.1',
...
cmdclass={'install': CustomInstall}
)
build.sh contains a make & make install step which takes more than 10 minutes to finish.
Is there a PyPi way to "package" the output of build.sh to speed up the pip install process?
Use wheel. A wheel is a great standard format for passing around Python packages, and it can contain C code compiled for various architectures. PyPI supports uploading wheels for your project, and pip will download them when available.
Very useful docs can be found here: https://packaging.python.org/tutorials/distributing-packages/#packaging-your-project
I'm trying to add a post-install step into the setup.py. The installation works but the code inside run(self) is never executed.
Things I've tried (with no result):
Install it using both "pip install (-e) ." and "python setup.py (develop)" [and later uninstall it, reinstall it, delete .egg-info folder,...]
Variations using: do_egg_install, build, bdist_egg,...
Versions:
pip 8.1.2 (for Python 3.5)
setuptools-20.2.2
Toy example:
from setuptools import setup
from setuptools.command.install import install
class MyCommand(install):
def run(self):
print("Hello, developer, how are you? :)")
install.run(self)
if __name__ == '__main__':
setup(
cmdclass={
'install': MyCommand,
}
)
I need to add rjsmin to my dependencies via install_require in a setup.py.
rjsmin offers a way to disable the c-extension by using the --without-c-extensions switch like following
python setup.py install --without-c-extensions
I wonder, how to add this switch to the install_require string.
I solved my problem of installing dependencies with global-options by sub-classing setuptools.command.install class and overriding its run() method, like following code -
from setuptools import setup
from setuptools.command.install import install
from subprocess import call
class CustomInstall(install):
def run(self):
install.run(self)
call(['pip', 'install', 'pycurl', '--global-option=--with-nss'])
setup( ...
cmdclass={
'install': CustomInstall,
},
)
Here, I am installing pycurl with global option --with-nss
You need to provide an --install-option or --global-option along with the requirement text.
You can refer the doc here