Does anybody encounter this warning when executing python setup.py install of a PyPI package?
install_requires defines what the package requires. A lot of PyPI packages have this option. How can it be an "unknown distribution option"?
python setup.py uses distutils which doesn't support install_requires. setuptools does, also distribute (its successor), and pip (which uses either) do. But you actually have to use them. I.e. call setuptools through the easy_install command or pip install.
Another way is to import setup from setuptools in your setup.py, but this not standard and makes everybody wanting to use your package have to have setuptools installed.
This was the first result on my google search, but had no answer.
I found that upgrading setuptools resolved the issue for me (and pip for good measure)
pip install --upgrade pip
pip install --upgrade setuptools
Hope this helps the next person to find this link!
I'm on a Mac with python 2.7.11. I have been toying with creating extremely simple and straightforward projects, where my only requirement is that I can run python setup.py install, and have setup.py use the setup command, ideally from distutils. There are literally no other imports or code aside from the kwargs to setup() other than what I note here.
I get the error when the imports for my setup.py file are:
from distutils.core import setup
When I use this, I get warnings such as
/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
If I change the imports (and nothing else) to the following:
from distutils.core import setup
import setuptools # noqa
The warnings go away.
Note that I am not using setuptools, just importing it changes the behavior such that it no longer emits the warnings. For me, this is the cause of a truly baffling difference where some projects I'm using give those warnings, and some others do not.
Clearly, some form of monkey-patching is going on, and it is affected by whether or not that import is done. This probably isn't the situation for everyone researching this problem, but for the narrow environment in which I'm working, this is the answer I was looking for.
This is consistent with the other (community) comment, which says that distutils should monkeypatch setuptools, and that they had the problem when installing Ansible. Ansible appears to have tried to allow installs without having setuptools in the past, and then went back on that.
https://github.com/ansible/ansible/blob/devel/setup.py
A lot of stuff is up in the air... but if you're looking for a simple answer for a simple project, you should probably just import setuptools.
ATTENTION! ATTENTION! Imperfect answer ahead. To get the "latest memo" on the state of packaging in the Python universe, read this fairly detailed essay.
I have just ran into this problem when trying to build/install ansible. The problem seems to be that distutils really doesn't support install_requires. Setuptools should monkey-patch distutils on-the-fly, but it doesn't, probably because the last release of setuptools is 0.6c11 from 2009, whereas distutils is a core Python project.
So even after manually installing the setuptools-0.6c11-py2.7.egg running setup.py only picks up distutils dist.py, and not the one from site-packages/setuptools/.
Also the setuptools documentation hints to using ez_setup and not distutils.
However, setuptools is itself provided by distribute nowadays, and that flavor of setup() supports install_requires.
This is a warning from distutils, and is a sign that you do not have setuptools installed.
Installing it from http://pypi.python.org/pypi/setuptools will remove the warning.
In conclusion:
distutils doesn't support install_requires or entry_points, setuptools does.
change from distutils.core import setup in setup.py to from setuptools import setup or refactor your setup.py to use only distutils features.
I came here because I hadn't realized entry_points was only a setuptools feature.
If you are here wanting to convert setuptools to distutils like me:
remove install_requires from setup.py and just use requirements.txt with pip
change entry_points to scripts (doc) and refactor any modules relying on entry_points to be full scripts with shebangs and an entry point.
sudo apt-get install python-dev # for python2.x installs
sudo apt-get install python3-dev # for python3.x installs
It will install any missing headers. It solved my issue
As far as I can tell, this is a bug in setuptools where it isn't removing the setuptools specific options before calling up to the base class in the standard library: https://bitbucket.org/pypa/setuptools/issue/29/avoid-userwarnings-emitted-when-calling
If you have an unconditional import setuptools in your setup.py (as you should if using the setuptools specific options), then the fact the script isn't failing with ImportError indicates that setuptools is properly installed.
You can silence the warning as follows:
python -W ignore::UserWarning:distutils.dist setup.py <any-other-args>
Only do this if you use the unconditional import that will fail completely if setuptools isn't installed :)
(I'm seeing this same behaviour in a checkout from the post-merger setuptools repo, which is why I'm confident it's a setuptools bug rather than a system config problem. I expect pre-merge distribute would have the same problem)
I've now seen this in legacy tools using Python2.7, where a build (like a Dockerfile) installs an unpinned dependancy, for example pytest. PyTest has dropped Python 2.7 support, so you may need to specify version < the new package release.
Or bite the bullet and convert that app to Python 3 if that is viable.
It works fine if you follow the official documentation:
import setuptools
setuptools.setup(...)
Source: https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py
Related
There is a locally built package (eg main-0.1.tar.gz). There is another package (for example base-0.1) that requires main-0.1 as a dependency.
It is necessary that during the subsequent installation of the base-0.1 package, the main-0.1 package is also installed.
Those. You can specify only packages with PyPI in install_requires, but local adding packages to the assembly is not clear how.
You can add the package main-0.1.tag.gz to the base-0.1 archive using MANIFEST.in (include main-0.1.tag.gz). But further dependency_links, for example, does not work correctly.
How do I add a local package to the build of another package and then install it along with another package, as if it were pulled from PyPI?
You might want to look at:
PEP 440 ("File URLs")
PEP 508
import setuptools
setuptools.setup(
# [...]
install_requires = [
'main # file:///path/to/main-0.1.tar.gz'
# [...]
],
)
Alternatively (probably better actually), use some combination of pip install options:
pip install --no-index --find-links '/path/to/distributions' main base
Reference:
https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages
Found a rough solution. I don't know how much it is for Feng Shui, but it works.
Add include main-0.1.tar.gz to MANIFEST.in
In setup.py, at the end of the file (after calling setup ()), add:
if 'sdist' not in sys.argv[1]:
os.system('pip install main-0.1.tar.gz')
The condition may be different if, for example, sdist is not used for building (python setup.py sdist). The main thing is to somehow determine that this is running setup for assembly, and not for installation (pip install base-0.1.tar.gz in the future).
In this case, we copy the local dependent package into the archive of the package being built, and it is distributed, accordingly, along with it. And installed the same way.
The importlib_resources backport for Python < 3.7 of the importlib.resources standard library module has the following section in the setup.cfg file:
[options]
python_requires = >=2.7,!=3.0,!=3.1,!=3.2,!=3.3
setup_requires =
setuptools
wheel
install_requires =
pathlib2; python_version < '3'
typing; python_version < '3.5'
packages = find:
Why does setup_requires include setuptools? This does not seem to make sense since:
the first line of the setup.py file imports setuptools, so by the time the setup function is called and reads the setup.cfg file that instructs to install setuptools it is already too late to install setuptools:
from setuptools import setup
setup()
setuptools is already installed on any fresh Python installation (well, only tested on Windows 10 and MacOS 10.15 with Python 3.8.0):
$ python -V
Python 3.8.0
$ pip list
Package Version
---------- -------
pip 19.2.3
setuptools 41.2.0
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
No, setuptools should not be included in setup_requires, according to PEP 518 (bold emphasis mine):
Setuptools tried to solve this with a setup_requires argument to its
setup() function [3]. This solution has a number of issues, such as:
No tooling (besides setuptools itself) can access this information without executing the setup.py, but setup.py can't be executed without having these items installed.
While setuptools itself will install anything listed in this, they won't be installed until during the execution of the setup() function, which means that the only way to actually use anything added here is through increasingly complex machinations that delay the import and usage of these modules until later on in the execution of the setup() function.
This cannot include setuptools itself nor can it include a replacement to setuptools, which means that projects such as numpy.distutils are largely incapable of utilizing it and projects cannot take advantage of newer setuptools features until their users naturally upgrade the version of setuptools to a newer one.
The items listed in setup_requires get implicitly installed whenever you execute the setup.py but one of the common ways that the setup.py is executed is via another tool, such as pip, who is already managing dependencies. This means that a command like pip install spam might end up having both pip and setuptools downloading and installing packages and end users needing to configure both tools (and for setuptools without being in control of the invocation) to change settings like which repository it installs from. It also means that users need to be aware of the discovery rules for both tools, as one may support different package formats or determine the latest version differently.
The accepted answer is mostly correct, but where PEP 518 says.
[The setup_requires mechanism] cannot include setuptools itself...
It's technically incorrect, and as importlib_resources demonstrates, it can actually include setuptools. The problem is that including setuptools in setup_requires serves mostly as documentation. It declares that setuptools is a build requirement (required to run setup.py), but it won't be capable of satisfying that requirement if it's not already satisfied.
But, the presence of setuptools in setup_requires is technically correct and does serve the purpose of declaring the requirement and asking setuptools to verify that the requirement is in fact installed (alongside other setup-time requirements).
It is, however, just a legacy artifact and doesn't provide that much value, and as can be seen in the question and answers, it does lead to confusion. The recommended, proper, approach is to use PEP 517 and 518 declarations and builders, but that part of the ecosystem hasn't matured yet, so setuptools vestiges will remain. Try not to let them bother you.
Why does setup_requires includes setuptools? This does not seem to make sense
Does not make sense at all. On the other hand it doesn't hamper anything so why not?
I ran into a problem when installing a package which depended on python-daemon. I ultimately traced it to the latest version of the package python-daemon (2.0.3) released yesterday. Testing in a virtual environment on an Ubuntu 14.04 machine and issuing the following commands:
(venv) $ pip list
argparse (1.2.1)
pip (1.5.6)
setuptools (3.6)
wsgiref (0.1.2)
(venv) $ pip install redis
... works fine ....
(venv) $ pip install python-daemon
...
snip
...
File "/home/pwj/.virtualenvs/venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2147, in load
['__name__'])
ImportError: No module named version
(venv)02:15 PM tmp$ pip list
argparse (1.2.1)
lockfile (0.10.2)
pip (1.5.6)
python-daemon (2.0.3)
setuptools (3.6)
wsgiref (0.1.2)
So the install of python-daemon seemed to work but something affected pip or setuptools because other packages (celery, flask), I try to install with pip after this gives me the same traceback:
...
snip
...
File "/home/pwj/.virtualenvs/venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2147, in load
['__name__'])
ImportError: No module named version
If I uninstall python-daemon with pip things again and packages that weren't installing now install fine. Has anyone else come across this or something similar with a different project? My solution was to pip install the previous version
(venv) $ pip install python-daemon==2.0.2
... works ...
but was wondering what might be causing such an error.
(This behaviour is corrected in python-daemon version 2.0.4 and later.)
There are two sides to this:
Setuptools assumes it is the centre of everything.
Version 2.0.3 of python-daemon doesn't take that into account.
A more detailed explanation: There is some complex code using Docutils involved in the python-daemon build process, that isn't needed after install and isn't part of the library code.
It's too complex to leave in the un-importable (and therefore not-unit-testable) setup.py, so that build code is shunted to a separate testable module, version (in the file version.py), which itself uses Docutils.
But then the setup.py has a circular dependency: How to import version, when Docutils isn't yet installed? How to use Setuptools to ensure Docutils is installed, when running setup.py to completion will need version? All the feasible solutions are ugly and confusing.
The approach taken in ‘python-daemon’ 2.0.3 is to declare Docutils required for setup, and declare a Setuptools entry point for the work that needs version. That way setup.py gets to install Docutils before any of the entry points that will use version.
But now we come to the first point, that Setuptools arrogates itself as the centre of everything. By declaring an entry point, setup.py has modified every Setuptools action thereafter, and every package will fail if it can't find the entry points. And, since most of them don't have version or the specified functions in that module, they crash Setuptools.
What is essentially a bug to be fixed, reveals a poorly-understood corner case in Setuptools. So I'm voting your question up.
There doesn't seem to be a good solution to this: having modules available for setup.py but ensuring requirements are met first. Setuptools assumes it is the only build system needed to satisfy all dependencies for everything, and when that assumption fails it's very difficult to get around.
Thanks to the Python Packaging Authority folks, and the distutils-sig forum, for explaining this to me.
I have a program that uses dateutil from the package index. I would like to have setup.py check for for its presence and try to get it using easy_install if it is not there.
The documentation for distutils seems to indicate that this can be done using the requires keyword in setup(), but when I try, it installs on a system without dateutil without giving a warning or installing the required package.
The only thing I could find on google was this blog post about the same issue which did not have any answer either.
Am I using distutils wrong? Do I need to subclass distutils.command.install and do the checking/installing myself?
Automatic downloading of dependencies is a feature introduced by setuptools which is a third-party add-on to distutils, in particular, the install_requires argument it adds. See the setuptools documentation for more information.
Another option is to use requirements.txt file with pip rather than using easy_install as a package installer. pip has now become the recommended installer; see the Python Packaging User Guide for more information.
Update [2015-01]: The previous version of this answer referred to the distribute fork of setuptools. The distribute fork has since been merged back into a newer active setuptools project. distribute is now dead and should no longer be used. setuptools and pip are now very actively maintained and support Python 3.
The argument install_requires in setup function from distutils work for me well, only if I create sdist distributive, like: python setup.py sdist
This is partly a question, partly my own findings on what I found to be the issue when I encountered this error:
(cdbak)USER-MBP-2:.virtualenvs <YOUR_USER_NAME>$ pip
Traceback (most recent call last):
File "/Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/bin/pip", line 6, in <module>
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources
This issue arose when I tried to install pypsum via pip in my virtual environment for use with django.
(cdbak)USER-MBP-2:.virtualenvs <YOUR_USER_NAME>$ pip install pypsum
I have been working in virtual environments, so I was fortunate that after it broke, I could just reset my virtual environment with the script I had written.
I copied the output from the installation process and started looking closer into it and it seems like there is something that goes wrong in setuptools.
The installation process tries to build the package but it does not find build_py in the setuptools which causes it to 'patch' the setuptools installation by renaming the currently installed setuptools.
This is the part where I think funky stuff starts to happen:
Setuptools installation detected at /Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
Egg installation
Patching...
Renaming /Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg into /Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg.OLD.1321360113.04
And then it goes on to try and install another version of setuptools or so it seems:
After install bootstrap.
Creating /Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info
Creating /Users/<YOUR_USER_NAME>/.virtualenvs/cdbak/lib/python2.7/site-packages/setuptools.pth
But it does not seem to install the setuptools package correctly in its current location and then results in a missing pkg_resources module (In fact, it is missing a lot of other things too)
[Setup]
OS: Mac OS X Lion
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
virtualenv v1.6.1
Fresh virtual environment using virtualenvwrapper using requirements file to install these packages:
mercurial==1.9.3
Django>=1.3.1
MySQL-python>=1.2.3
Sphinx
wsgiref
pylint
yolk
dbgp
django-debug-toolbar
south
I was able to use pip to install other packages just fine but for some reason it seems to break with this installation.
[Question]
- Do people think this is a mistake with the installation process of this package?
- Or, is it a mistake with setuptools?
- Or, am I just installing it incorrectly?
[Extra Note]
I can attach the file with the entire output but its a long file and I decided to only extract the segments I felt were relevant. If you would like to view the full file, I can upload that too.
I'm not sure about all of your problems but there is at least one problem caused by the loremipsum package that is a dependency for the pypsum package. For some reason, in the setup.py file of loremipsum, the author includes specific requirements for the distribute package:
egg = {
'name': name,
'version': module.__version__,
'author': author,
'author_email': email.strip('<>'),
'url': url,
'description': "A Lorem Ipsum text generator",
'long_description': long_description,
'classifiers': module.__classifiers__,
'keywords': ['lorem', 'ipsum', 'text', 'generator'],
'setup_requires': ['distribute'],
'install_requires': ['distribute'],
'packages': [name],
# 'package_dir': {'': '.'},
# 'package_data': {'': 'default/*.txt'},
# 'data_files': [(name, ('default/dictionary.txt', 'default/sample.txt'))],
'include_package_data': True,
'test_suite': 'tests.suite' }
distribute, as you may know, is a fork of the setuptools package; there's a long history behind this. Since distribute is supposed to be an almost plug-compatible replacement for setuptools, it will try to masquerade as setuptools and disable any existing setuptools already installed in that Python instance. Thus, putting distribute in a setup.py file as a requirement is usually not a good idea. By default, virtualenv will install a version of setuptools but it does have an option to use distribute instead. The Apple-supplied system Pythons in OS X 10.6 and 10.7 already come with versions of setuptools pre-installed and, because they are in non-standard system directories, cannot be so easily patched around. A simple workaround when using virtualenv on OS X seems to be to also use its no-site-packages option which will prevent the setuptools version from the system Python interfering with the required distribute in the virtualenv. No doubt the confusion between distribute and setuptools is causing the problems seen with pkg_resources since it is also supplied by both of them.
So try re-creating your virtualenv this way:
virtualenv --distribute --no-site-packages /path/to/ve
That will also have the side-effect of not including the third-party packages that Apple ships with the system Python. You can add them back in with PYTHONPATH if you really need them but it's probably better to install separate versions.