I'm working with a software that has an embedded python environment which does not have all of the standard python modules available.
For example there's no pip or pkg_resources. The only way to add 3rd-party modules/packages is by copying them into a path which is referenced in sys.path.
At the moment I am writing the first version of my own package and more will come in the future.
Now to my question. Is it considered a bad approach to have the version number inside the package name or completely as subpackage?
For example mypackage_0_1/ or mypackage/0_1/
If so, what would be a good alternative to solve this problem?
In most of the cases, the version number in a package, regardless of the language, is used to communicate compatibility. For example, Qt: there are Qt4 and Qt5 series. What the numbers mean is that we have different APIs and switching from Qt4 to Qt5 will mostly break your application. Likewise, Python follows this pattern: you have Python 2.x and 3.x series. This means that if you write your application for Python 3.6, it is expected that it will work with Python 3.7. In other words, Python 3.x releases are forward compatible (ie, software written for a older version of Python 3.x is expected to run OK with newer versions). However, writing your application for Python 2.7 and running it in Python 3.7, for example, will mostly bring unexpected results. Hence, Python 3.x breaks the compatibility with Python 2.x and they are neither forward of backward compatible.
So, my advice is to follow this rule: use a version number in the package name only and only if you are communicating that you are creating a series of forward compatible versions of your package.
Related
How should the shebang for a Python script look like?
Some people support #!/usr/bin/env python because it can find the Python interpreter intelligently. Others support #!/usr/bin/python, because now in most GNU/Linux distributions python is the default program.
What are the benefits of the two variants?
The Debian Python Policy states:
The preferred specification for the Python interpreter is /usr/bin/python or /usr/bin/pythonX.Y. This ensures that a Debian installation of python is used and all dependencies on additional python modules are met.
Maintainers should not override the Debian Python interpreter using /usr/bin/env python or /usr/bin/env pythonX.Y. This is not advisable as it bypasses Debian's dependency checking and makes the package vulnerable to incomplete local installations of python.
Note that Debian/Ubuntu use the alternatives system to manage which version /usr/bin/python actually points to. This has been working very nicely across a lot of python versions at least for me (and I've been using python from 2.3 to 2.7 now), with excellent transitions across updates.
Note that I've never used pip. I want automatic security upgrades, so I install all my python needs via aptitude. Using the official Debian/Ubuntu packages keep my system much cleaner than me messing around with the python installation myself.
Let me emphasize one thing. The above recommendation refers to system installation of python applications. It makes perfectly sense to have these use the system managed version of python. If you are actually playing around with your own, customized installation of python that is not managed by the operating system, using the env variant probably is the correct way of saying "use the user-preferred python", instead of hard-coding either the system python installation (which would be /usr/bin/python) or any user-custom path.
Using env python will cause your programs to behave differently if you call them from e.g. a python virtualenv.
This can be desired (e.g. you are writing a script to work only in your virtualenv). And it can be problematic (you write a tool for you, and expect it to work the same even within a virtualenv - it may suddenly fail because it is missing packages then).
My humble opinion is that you should use the env-variant. It's a POSIX component thus found in pretty much every system, while directly specifying /usr/bin/python breaks in many occasions, i.e. virtualenv setups.
I use #!/usr/bin/env python as the default install location on OS-X is NOT /usr/bin. This also applies to users who like to customize their environment -- /usr/local/bin is another common place where you might find a python distribution.
That said, it really doesn't matter too much. You can always test the script with whatever python version you want: /usr/bin/strange/path/python myscript.py. Also, when you install a script via setuptools, the shebang seems to get replaced by the sys.executable which installed that script -- I don't know about pip, but I would assume it behaves similarly.
As you note, they probably both work on linux. However, if someone has installed a newer version of python for their own use, or some requirement makes people keep a particular version in /usr/bin, the env allows the caller to set up their environment so that a different version will be called through env.
Imagine someone trying to see if python 3 works with the scripts. They'll add the python3 interpreter first in their path, but want to keep the default on the system running on 2.x. With a hardcoded path that's not possible.
Is there an easy way to tell if a Python package will work if not matched to my Python release? For example, I cannot get PyGreSQL for Python 2.7 on Win (only 2.6) and I can't get ZODB for Win any more recent than Python 2.4. Does the version mismatch guarantee incompatibility, or do I just need to rename the package directory?
There's no way to tell for sure if the package will work on another Python version, but using older packages on newer Python 2.x's is much much more likely to work than the other way around.
One thing to watch out for: if you're installing a Windows kit, it may only install in the version it was meant for, due to registry look-ups and the like.
I have the default Python 2.6.1 installed as /usr/bin/python and Python 3.1.2 installed in /usr/local/bin/python3.1. Considering that I use only 3.x syntax, is it safe to replace the default interpreter (2.6) with the 3.1 one (python-config included) using symlinks (and removing old Python binary)? Or is the system relying on the 2.x version for some purpose I don't know?
If you're only using Python 3, start your scripts with:
#! /usr/bin/env python3.1
And you'll be using the right version, without doinking the system about.
edit: BTW this idea is suggested by the Python docs. Each script will be running the version of Python they depend on. Since Python 3 is not backward compatible, it seems dangerous to be replacing the Python executable with one that will break scripts other utilities may rely on.
You can not safely replace the system supplied python. I cannot find a Mac-specific reference for you... but some recent Python versions are not backwards compatible... Many scripts made dependent on an older version of Python will not run on an upgraded python. OS X Comes with Python pre-installed because it has dependencies on it.
Try using VirtualEnv instead.
Update: Just came across python-select from macports which may solve your problem.
Don't replace / remove any binaries unless you are in dire need for storage. In that case too, the mileage is very little in removing them.
You can simply make 3.1 as default with :
defaults write com.apple.versioner.python Version 3.1
There are other ways to ensure that you use 3.1 by default, I have not used them though.
export VERSIONER_PYTHON_VERSION=3.1
On Tiger, I used a custom python installation to evaluate newer versions and I did not have any problems with that*. Now Snow Leopard is a little more up-to-date and by default ships with
$ ls /System/Library/Frameworks/Python.framework/Versions/
2.3 2.5 2.6 #Current
What could be considered best practice? Using the python shipped with Mac OS X or a custom compiled version in, say $HOME.
Are there any advantages/disadvantages using the one option over the other?
My setup was fairly simple so far and looked like this: Custom compiled Python in $HOME and a $PATH that would look into $HOME/bin first, and subsequently would use my private Python version. Also $PYTHONPATH pointed to this local installation. This way, I did not need to sudo–install packages - virtualenv took care of the rest. Note: I liked this setup, so I'm just curious and thought I inquire the hive mind.
I guess it depends on your needs. Personally, I use the latest version for any series (2.5, 2.6, etc.) from MacPorts.
I compile it myself because this gives me the newest 64-bit version. The official OS X builds seem to be 32-bit only. I dropped MacPorts several months ago because its dependency system and its often outdated packages were too annoying.
The problem with using the Python version that's shipped with your OS is that it may contain bugs, or be limited in other ways. If you install Python from Fink or MacPorts, you have the liberty of updating it.
Another, important advantage of managing your own version of Python with a package manager (Fink or MacPorts) is that they help a lot with the compilation of module dependencies (for instance when you use a module that depends on compiled C code). Thus, installing Python modules is certainly easier if you do not use the Python shipped with OS X. This is an important point to consider, before making your choice.
What you do on your on machine is totally up to you. If you're going to deploy your code to other people's, I'd say it's clearly preferable to use the shipped version unless you really need the newer version.
I'm digging up this old topic.
There are no real answers to create your own python distro/framework from source and bundle and package it in a correct way. I tried to compile it from source, linking it against my C programmed application which uses python 3 and works on my computer. But when I move it around the filesystem (e.g to /tmp) it has hardcoded paths in the python compile. And I have no clue what script/wrapper to make.
My point to ship our own python distro is to make sure there is no oddness in interpeter change involved when shipping the application and rely on the OS python install.
It is also nowhere documented on the python docs.
I already found this post 4206511
Various software installations on my laptop each require their own particular version of Python. ViewVC requires Python 2.5 and Blender requires Python 2.6. Mercurial (thankfully) comes with its Python interpreter packaged in a DLL in the Mercurial installation itself.
How do I get by without having to install the entire Python environment each time? Is there some minimal installer which will install the bare minimum without affecting other programs? Can I modify the Blender and ViewVC installations so that they too use their own Python-in-a-DLL?
It's hard to know which "bare minimum" the Blender scripts you'll want to use in the future may be counting on (short of the full Python standard library, which isn't all that large in term of disk space after all). Why not install both Python 2.5 and 2.6? They can coexist nicely (if your scriptable apps use hashbangs like #!/usr/bin/env python instead of specifically mentioning python2.5 or python2.6, you may need to trick out their PATHs just a little bit).
You should be able to get away with installing the Python binaries in the same tree as the specific application I believe (Totally untested hunch though).