We have an application that uses pyenv/virtualenv to manage python dependencies. We want to ensure that everyone who works on the application will have the same python version. Coming from ruby, the analog is Gemfile. To a certain degree, .ruby-version.
What's the equivalent in python? Is it .python-version? I've seen quite a few .gitignore that have that in it and usually under a comment ".pyenv". What's the reason for that? And what's the alternative?
Recent versions of setuptools (24.2.0+) allow you to control Python version at the distribution level.
For example, suppose you wanted to allow installation only on a (compatible) version of Python 3.6, you could specify:
# in setup.py
from setuptools import setup
setup(
...
python_requires='~=3.6',
...
)
The distribution built by this setup would have associated metadata which would prevent installation on incompatible Python version. Your clients need a current version of pip for this feature to work properly, older pip (<9.0.0) will not check this metadata.
If you must extend the requirement to people using older version of pip, you may put an explicit check on sys.version somewhere in the module level of the setup.py file. However, note that with this workaround, the package will still be downloaded by pip - it will fail later, on a pip install attempt with incorrect interpreter version.
Related
What is the purpose of the namespace_packages argument in setup.py when working with PEP420 namespace packages (the ones without __init__.py)?
I played with it and saw no difference whether I declared the namespace packages or not.
"setup.py install" and "pip install ." worked in any case.
I am building an automatic setup.py code generator and would be happy not to handle this if this is not necessary.
As long as you:
aim for Python 3.3 and newer or Python 2.7 with importlib2 dependency installed (a backport of importlib for Python 2),
use a recent version of setuptools for packaging (I think it should be 28.8 or newer)
and use a recent pip version for installing (9.0 and newer will be fine, 8.1.2 will probably also work, but you should test that yourself),
you are on the safe side and can safely omit the namespace_packages keyword arg in your setup scripts.
There is a PyPA's official repository named sample-namespace-packages on GitHub that contains a suite of tests for different possible scenarios of distributions installed that contain namespace packages of each kind. As you can see, the sample packages using the implicit namespace packages don't use namespace_packages arg in their setup scripts (here is one of the scripts) and all of the tests of types pep420 and cross_pep420_pkgutil pass on Python 3; here is the complete results table.
Namespace packages are separate packages that are installed under one top-level name.
Usually two different packages (for example SQLObject and Cheetah3) install two (or more) different top-level packages (sqlobject and Cheetah in my examples).
But what if I have a library that I want to split into parts and allow to install these parts without the rest of the library? I use namespace packages. Example: these two packages are 2 parts of one library: m_lib and m_lib.defenc. One installs m_lib/defenc.py which can be used separately, the other installs the rest of the m_lib library. To install the entire library at once I also provide m_lib.full.
PS. All mentioned packages are mine. Source code is provided at Github or my personal git hosting.
I am creating a requirements.txt file for my Python project. When someone installs my project via pip; I want pip to also download the Tkinter and time Python modules, both of which are modules in the Python Standard Library.
I used the command pip freeze to list all of the currently installed Python packages, but when I scrolled down to the T section, I could not find Tkinter or time in the list. I then tried to install Tkinter via pip using the command pip install Tkinter but got the following error:
Could not find a version that satisfies the requirement Tkinter (from versions: )
No matching distribution found for Tkinter
I also did the same with time and got the same error. (I am running Python 3.6 and Windows 10.)
Is it possible to get pip to install those modules while pip is installing my package, or do I just have to trust that because the user has Python installed they will also have Tkinter and time installed?
All help is welcome and appreciated!
Python's Standard Library is called the Standard Library because it is a standard of Python. In other words, if there is no Standard Library installed, the python environment is not python at all.
The Standard Library is tested and released together with each Python release as part of this release (not as an addition or extension).
So, YES, you can expect these libraries to exist if the user has Python installed. Just by definition.
Regarding the upgrades of the built-in libraries: NO, you cannot do this. Because they are part of the python setup, not of the application environment. Python is very tightly bound to the specific code in those libraries. All python apps & libs expect the same behavior of those libraries, even if they are buggy.
In addition to that, you cannot install a module/package with the same name as one of the python's builtins, because it will create the ambiguity on import, and can confuse/break all other libraries which depend on it (or worse, the system applications if you install it into the system python).
However, in some cases you can find the backports of some of the libraries. Usually, they are backported from py3 to py2. Of course, their name is changed.
As an example, you can look into concurrent.features library, which is a handy builtin in py3.2+, but was absent in py2.7.
UPD: Though, as #JulienPalard hints in the comments, some OS distributions can split this standard library to simplify the binary dependencies: e.g., on Debian, Tkinter will be installable separately as python3-tk.
This makes sense, indeed, from the point of view of the binary OS packaging: it is not worth installing the UI parts of the python library if you have no UI at all and want to save the disk space.
However, you are still unable to install it via pip. Because this package is not packaged and available separately on PyPI. This standard library separation is made by the selected OS distributions and is resolved with the means of that OS distribution only.
pip installs packages from pypi, which does not expose the standard library, which is bundled into Python.
So in theory you should trust your users environment, if they have Python, they should have the whole stadard library.
But some distributions are splitting Python in a few packages for reasons (A minimal Debian already depends on Python, but they don't want Python to pull tk to pull libx11 to pull the life, the universe, and everything).
So in practice some package will be there, and you can trust the distribs for this, like time will always be here. And some package may not be here, like tkinter, in which case you may want to surround the import tkinter by a try to error a nice "Looks like your distribution does not provide tk by default, please install it.".
Be reassured, you won't have to surround every imports by try statements just in case some distribution splitted the stdlib, just tkinter.
I am currently developing a command line application in python, which I will be uploading to pypi for end users to pip install and use. I am taking advantage of the extras functionality in setup.py to support 2 versions of my application, one is a basic functionality version with minimal dependencies and the other is more feature rich but has a large amount of dependencies (numpy, pandas, networkx, matplotlib, etc)
So briefly:
pip install app # simple, no deps
pip install app[all] # all the deps
Now the problem is that one of my dependencies in the feature rich version has what has been described as "flakey" pypi support. Basically, it cannot be installed unless one of its dependencies is already pre-installed before the whole installation process occurs. Luckily (or not), my application (which pulls in this flakey module) also has the flakey modules needed module. Lets refer to the module that flakey module needs pre-installed fixer-module
So:
pip install app[all] # triggers the installation of flakey module
Installation will fail here if fixer-module is not installed, even though it will be installed before flakey module is. One must basically do this:
pip install fixer-module
pip install app[all]
Now what I would like to do is include some kind of checking code that accomplishes the following:
Runs only when the app[all] distribution is being installed
Does a try import fixer-module, except ImportError check and prints a message explaining the situation.
Stops and cleans up the installation process before it fails
I have been researching this for quite some time. I found some examples of checking the input args to setup.py and whatnot but none of them seem to cover how to handle stuff during the end user install process. Any pointers are much appreciated!
Currently, we are setting\installing up some packages on system by mentioning their version and dependencies in setup.py under install_requires attribute. Our system requires Python 2.7. Sometimes, users are having multiple versions of Python on their systems, say 2.6.x and 2.7, some packages it says are available already but actually on the system available under 2.6 site packages list. Also some users have 2.6 only, how to enforce from setup.py or is there any other way to say to have only Python 2.7 and all packages which we want setup.py to update are for only 2.7. We require minimum 2.7 on the machine to run our code.
Thanks!
Santhosh
The current best practice (as of this writing in March 2018) is to add a python_requires argument directly to the setup() call in setup.py:
from setuptools import setup
[...]
setup(name="my_package_name",
python_requires='>3.5.2',
[...]
Note that this requires setuptools>=24.2.0 and pip>=9.0.0; see the documentation for more information.
As the setup.py file is installed via pip (and pip itself is run by the python interpreter) it is not possible to specify which Python version to use in the setup.py file.
Instead have a look at this answer to setup.py: restrict the allowable version of the python interpreter which has a basic workaround to stop the install.
In your case the code would be:
import sys
if sys.version_info < (2,7):
sys.exit('Sorry, Python < 2.7 is not supported')
I'm currently doing some embedded systems programming. This was set up by somebody else a few years ago. So now I'm looking to upgrade to Python 2.7.2 to make things simpler because I have already run into two cases where what I coded wasn't supported.
What is currently running:
: uname -a
Linux host1 2.6.18-6-486 #1 Sun Feb 10 22:06:33 UTC 2008 i586 GNU/Linux
: python -v
Python 2.4.4
: pyversions -i
python2.4
So right now only 2.4 is installed.
I untarred python2.7.2 and when I go to that directory and run python27 setup.py install --home=/home/jhemilian and it seems like python2.4 doesn't seem to know the with...as statement syntax:
host1:/home/jhemilian/src/Python-2.7.2: python setup.py install --home=/home/jhe
milian
File "setup.py", line 361
with open(tmpfile) as fp:
^
SyntaxError: invalid syntax
Before I go figuring this out I first have a question: python itself is being used to install Python? What if I didn't have the first version of Python installed? I know it's shipped with most Linux but hypothetically -- how does such a seeming catch-22 like that work?
What I am looking to do is install python2.7 in a benign location, keeping the python command still as using Python 2.4 just in case the "legacy" software i'm running is dependent on it, and running python2.7 myscript.py et cetera when I want to run one of my newer scripts. Feel free to comment if there is a cleaner or more practical (or even safer!) way to do this.
I don't think it would make much sense to go replacing all the with statements with compatible try blocks. I've looked though the READMEs and online documentation but I can't seem to find a way to install Python without already having Python. Note that I DO NOT have internet connection, although if desirable or necessary I could. It would be great if somebody could point me in the right direction. Thanks!!
It's all right in the README...
You don't need to use python to install, in fact, you shouldn't...just:
./configure
make
make install
If you want to install in a specific dir, just follow what the README says:
Installing
To install the Python binary, library modules, shared library modules
(see below), include files, configuration files, and the manual page,
just type
make install
This will install all platform-independent files in subdirectories of
the directory given with the --prefix option to configure or to the
prefix' Make variable (default /usr/local). All binary and other
platform-specific files will be installed in subdirectories if the
directory given by --exec-prefix or theexec_prefix' Make variable
(defaults to the --prefix directory) is given.
If DESTDIR is set, it will be taken as the root directory of the
installation, and files will be installed into $(DESTDIR)$(prefix),
$(DESTDIR)$(exec_prefix), etc.
All subdirectories created will have Python's version number in their
name, e.g. the library modules are installed in
"/usr/local/lib/python/" by default, where is the
. release number (e.g. "2.1"). The Python binary is
installed as "python" and a hard link named "python" is
created. The only file not installed with a version number in its
name is the manual page, installed as "/usr/local/man/man1/python.1"
by default.
If you want to install multiple versions of Python see the section
below entitled "Installing multiple versions".
The only thing you may have to install manually is the Python mode for
Emacs found in Misc/python-mode.el. (But then again, more recent
versions of Emacs may already have it.) Follow the instructions that
came with Emacs for installation of site-specific files.
EDIT: virtualenv is apparently for already-installed Python versions. Disregard this recommendation.
I think what you want is virtualenv.
I haven't used it myself, but I understand this is what it's meant for.
From the website:
virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.
EDIT: Upon review, I think you want Alberto's answer, so I voted him up for visibility.