Currently, we are setting\installing up some packages on system by mentioning their version and dependencies in setup.py under install_requires attribute. Our system requires Python 2.7. Sometimes, users are having multiple versions of Python on their systems, say 2.6.x and 2.7, some packages it says are available already but actually on the system available under 2.6 site packages list. Also some users have 2.6 only, how to enforce from setup.py or is there any other way to say to have only Python 2.7 and all packages which we want setup.py to update are for only 2.7. We require minimum 2.7 on the machine to run our code.
Thanks!
Santhosh
The current best practice (as of this writing in March 2018) is to add a python_requires argument directly to the setup() call in setup.py:
from setuptools import setup
[...]
setup(name="my_package_name",
python_requires='>3.5.2',
[...]
Note that this requires setuptools>=24.2.0 and pip>=9.0.0; see the documentation for more information.
As the setup.py file is installed via pip (and pip itself is run by the python interpreter) it is not possible to specify which Python version to use in the setup.py file.
Instead have a look at this answer to setup.py: restrict the allowable version of the python interpreter which has a basic workaround to stop the install.
In your case the code would be:
import sys
if sys.version_info < (2,7):
sys.exit('Sorry, Python < 2.7 is not supported')
Related
I have a Python package containing a number of C/C++ extensions built as a single wheel. I'm trying to understand how to ensure the wheel and shared libraries it contains correctly advertise that they use the stable ABI at a particular API version. I build the package using a setup.py that I run this way.
% python setup.py bdist_wheel --py-limited-api=cp34
I think the cp34 part is how I indicate that I'm using the stable ABI and at most the Python 3.4 API. The resulting wheel is named goober-1.2-cp34-abi3-linux_x86_64.whl. The highlighted part shows the Python and ABI tags. Without the --py-limited-api, that part is cp38-cp38, matching my Python 3.8. Is that enough to advertise that my wheel should work with all Python 3.x starting from 3.4, without recompiling? I guess I'd specify cp3 to indicate all 3.x versions.
For the shared libraries, I compile the C/C++ source this way.
% gcc ... -DPy_LIMITED_API=0x03040000 ... blooper.c
In this case, the shared library is named blooper.cpython-38-x86_64-linux-gnu.so, with nothing indicating it supports the stable ABI and the 3.4 API. From PEP 3149 I expected to see that somewhere in the name. Otherwise, won't Python 3.8 be the only version willing to import this module?
Thanks.
It might be surprising, but adding --py-limited-api=cp34 only changes the name of the wheel, but not its content - i.e. it will be still "usual" version pinned to the Python version which with it has been built.
The first step is to create a setup.py which would produce a C-extension which uses stable-API and which declares it as well. To my knowledge distutils has no support for stable C-API, so setuptools should be used.
There is a minimal example:
from setuptools import setup, Extension
my_extension = Extension(
name='foo',
sources = ["foo.c"],
py_limited_api = True,
define_macros=[('Py_LIMITED_API', '0x03040000')],
)
kwargs = {
'name':'foo',
'version':'0.1.0',
'ext_modules': [my_extension],
}
setup(**kwargs)
Important details are:
py_limited_api should be set to True, thus the resulting extension will have the correct suffixes (e.g. abi3), once build.
Py_LIMITED_API macro should be set to the correct value, otherwise non-stable or wrong stable C-API will be used.
The resulting suffix of the extension might also be surprising. The CPython documentation states:
On some platforms, Python will look for and load shared library files
named with the abi3 tag (e.g. mymodule.abi3.so). It does not check if
such extensions conform to a Stable ABI. The user (or their packaging
tools) need to ensure that, for example, extensions built with the
3.10+ Limited API are not installed for lower versions of Python.
"Some platforms" are Linux and MacOS, one can check it by looking at
from importlib.machinery import EXTENSION_SUFFIXES
print(EXTENSION_SUFFIXES)
# ['.cpython-38m-x86_64-linux-gnu.so', '.abi3.so', '.so'] on Linux
# ['.cp38-win_amd64.pyd', '.pyd'] on Windows
that means on Linux, the result will be foo.abi3.so and just foo.pyx on Windows (see e.g. this code in setuptools).
Now, just running
python setup.py bdist_wheel
would build an extension, which could be used with any Python version>= 3.4, but pip would not install it for anything else than CPython-3.8 with pymalloc on Linux (because the name of wheel is foo-0.1.0-cp38-cp38m-linux_x86_64.whl). This is the part, from the documentation, where the packaging system needs to ensure, that it doesn't come to version mismatch.
To allow pip to install for multiple python versions, the wheel should be created with --py-limited-api-version:
python setup.py bdist_wheel --py-limited-api=cp34
due to the resulting name foo-0.1.0-cp34-abi3-linux_x86_64.whl, pip will know, it is safe to install for CPython>=3.4.
To make clear: CPython doesn't really know, that the c-extension with suffix abi3.so (or .pyx on Windows) can really be used by the interpreter (it just assumes in good faith) - it is pip who ensures, that right version is installed.
What is the purpose of the namespace_packages argument in setup.py when working with PEP420 namespace packages (the ones without __init__.py)?
I played with it and saw no difference whether I declared the namespace packages or not.
"setup.py install" and "pip install ." worked in any case.
I am building an automatic setup.py code generator and would be happy not to handle this if this is not necessary.
As long as you:
aim for Python 3.3 and newer or Python 2.7 with importlib2 dependency installed (a backport of importlib for Python 2),
use a recent version of setuptools for packaging (I think it should be 28.8 or newer)
and use a recent pip version for installing (9.0 and newer will be fine, 8.1.2 will probably also work, but you should test that yourself),
you are on the safe side and can safely omit the namespace_packages keyword arg in your setup scripts.
There is a PyPA's official repository named sample-namespace-packages on GitHub that contains a suite of tests for different possible scenarios of distributions installed that contain namespace packages of each kind. As you can see, the sample packages using the implicit namespace packages don't use namespace_packages arg in their setup scripts (here is one of the scripts) and all of the tests of types pep420 and cross_pep420_pkgutil pass on Python 3; here is the complete results table.
Namespace packages are separate packages that are installed under one top-level name.
Usually two different packages (for example SQLObject and Cheetah3) install two (or more) different top-level packages (sqlobject and Cheetah in my examples).
But what if I have a library that I want to split into parts and allow to install these parts without the rest of the library? I use namespace packages. Example: these two packages are 2 parts of one library: m_lib and m_lib.defenc. One installs m_lib/defenc.py which can be used separately, the other installs the rest of the m_lib library. To install the entire library at once I also provide m_lib.full.
PS. All mentioned packages are mine. Source code is provided at Github or my personal git hosting.
We have an application that uses pyenv/virtualenv to manage python dependencies. We want to ensure that everyone who works on the application will have the same python version. Coming from ruby, the analog is Gemfile. To a certain degree, .ruby-version.
What's the equivalent in python? Is it .python-version? I've seen quite a few .gitignore that have that in it and usually under a comment ".pyenv". What's the reason for that? And what's the alternative?
Recent versions of setuptools (24.2.0+) allow you to control Python version at the distribution level.
For example, suppose you wanted to allow installation only on a (compatible) version of Python 3.6, you could specify:
# in setup.py
from setuptools import setup
setup(
...
python_requires='~=3.6',
...
)
The distribution built by this setup would have associated metadata which would prevent installation on incompatible Python version. Your clients need a current version of pip for this feature to work properly, older pip (<9.0.0) will not check this metadata.
If you must extend the requirement to people using older version of pip, you may put an explicit check on sys.version somewhere in the module level of the setup.py file. However, note that with this workaround, the package will still be downloaded by pip - it will fail later, on a pip install attempt with incorrect interpreter version.
This question tells me how to check the version of Python. For my package I require at least Python 3.3:
MIN_VERSION_INFO = 3, 3
import sys
if not sys.version_info >= MIN_VERSION_INFO:
exit("Python {}.{}+ is required.".format(*MIN_VERSION_INFO))
but where/when should this check occur?
I want to produce the clearest possible error message for users installing via pip (sdist and wheel) or python setup.py install. Something like:
$ pip -V
pip x.x.x from ... (Python 3.2)
$ pip install MyPackage
Python 3.3+ is required.
$ python -V
Python 3.2
$ python setup.py install
Python 3.3+ is required.
The primary point of a compatibility check is to have this check either elegantly handle a compatibility issue or gracefully exit with an explanation before the incompatibility causes problems.
I'd put it near the top of setup.py, or the first script that belongs to you that will be called. Best practice is not to include code in __init__.py (unless you're making a MAJOR package, I'd opine), but if you already have a lot of code there, it's fine.
The quickest point of reference I can find: The old Python 2.6 unittest module had a test for it near the top, naturally, after the module docstring, imports, and __all__.
Another point of reference shows a compatibility check in an __init__.py, again, near the top, though here it is immediately after the docstring and an import of sys, required for the check. There are other similar examples of this usage in __init__.py in this same set of libraries.
I need to support some software that is using an old Python version (2.4). So I have downloaded and compiled Python 2.4 and installed it in a virtualenv. So far, all OK and normal procedure.
But the software is trying to import an rpm module. And I cannot find a source for that module (it is not part of the standard Python library, afaict).
Typically, once the virtualenv is enabled (source env/bin/activate) I can install required software using easy_install. But easy_install rpm is failing to find anything. There is a pyrpm module, but it is not the same thing (it installs a module called "pyrpm"). And google searches are useless, as they all link to articles on how to build rpms...
If I were using the system python (on Ubuntu) I could install the python-rpm package. But that is for Python 2.7. How do I install the equivalent for Python 2.4?
[My impression is that the rpm libraries, used by many Linux systems, include a Python library, which is packaged as python-dev by the distro. But I can't see how to access that for an arbitrary python version.]
I AM NOT LOOKING FOR AN RPM THAT CONTAINS PYTHON 2.4. I AM LOOKING FOR A MODULE NAMED rpm THAT IS USED BY SOFTWARE WRITTEN FOR PYTHON 2.4.
It's right there, in the python-rpm RPM package:
http://rpmfind.net/linux/rpm2html/search.php?query=python-rpm
You will probably want to download the package contents, extract them, and then use
python setup.py install
From your active environment.
Of course, as it's pre compiled, you might have trouble getting the C extension to run.
I'm not familiar enough with RPM's to know whether you can get the source from there.
No guarantees the package will work with your python version though.
there's no simple way to do this; the python library is part of the system rpm package and interfaces to C code, so is closely tied to the rpm package installed on your machine.
instead, it's much simpler to install an old OS in a VM (eg CentOS 5) that uses Python 2.4. then everything is consistent and works.
the sources for the rpm module can be found here: http://www.rpm.org/wiki/Download
After you download the wanted version read and follow the INSTALL instructions in order to compile it on your target OS. Afterwards make sure you add the correct path to the 'site-packages' folder the installation chose into your PYTHONPATH environment variable.
To test start your python interpreter and run 'import rpm'
HTH,
Ran