I am building python2.6 from source on Debian Lenny.
( ./configure make make altinstall )
I don't want it to conflict with anything existing, but I want it to be in the default search path for bash.
Suggestions?
(ps, I'm using a vm, so I can trash it and rebuild.)
That's the purpose of /usr/local according to the FHS.
The /usr/local hierarchy is for use by the system administrator when installing software locally.
I think configure typically defaults to /usr/local unless told otherwise, but to be sure you could run ./configure --prefix=/usr/local ....
I strongly recommend you do one of these two options.
Build a .deb package, and then install the .deb package; the installed then go in the usual places (/usr/bin/python26 for the main interpreter).
Build from source, and install from source into /usr/local/bin.
I think it is a very bad idea to start putting files in the usual places, but not known or understood by the package manager. If you built it by hand and installed it by hand, it should be confined in the /usr/local tree.
I would recommend to fetch the source package from testing or unstable and to rebuild it locally so that you get a .deb instead. Doesn't backports.org have it?
Edit: Debian has python2.6 only in experimental, see here. You could also take the source package from Ubuntu.
Your safest bet is to put Python 2.6 in /opt (./configure --prefix=/opt), and modify /etc/profile so that /opt/bin is searched first.
Related
I am building Python 3.10 from source on Ubuntu 18.04, following instructions from several web links, primarily the Python website (https://devguide.python.org/setup) and RealPython (https://realpython.com/installing-python/#how-to-build-python-from-source-code). I extracted Python-3.10.0.tgz into /opt/Python3.10. I have three questions.
First, the Python website says to use ./configure --with-pydebug and RealPython says to use ./configure --enable-optimizations --with-ensurepip=install. Another source says to include --enable-shared and --enable-unicode=ucs4. Which of these is best? Should I use all of those flags?
Second, I currently have Python 3.6 and Python 3.8 installed. They are installed in several directories under /usr. Following the directions I have seen on the web I am building in /opt/Python3.10. I assume that make altinstall (the final build step) will take care of installing the build in the usual folders under /usr, but that's not clear. Should I use ./configure --prefix=directory although none of the web sources mention doing that?
Finally, how much does --enable-optimizations slow down the install process?
This is my first time building Python from source, and it will help to clear these things up. Thanks for any help.
Welcome to the world of Python build configuration! I'll go through the command line options to ./configure one by one.
--with-pydebug is for core Python developers, not developers (like you and me) just using Python. It creates debugging symbols and slows down execution. You don't need it.
--enable-optimizations is good for performance in the long run, at the expense of lengthening the compiling process, possibly by 3-fold (or more), depending on your system. However, it results in faster execution, so I would use it in your situation.
--with-ensurepip=install is good. You want the most up-to-date version of pip.
--enable-shared is maybe not a good idea in your case, so I'd recommend not using it here. Read Difference between static and shared libraries? to understand the difference. Basically, since you'll possibly be installing to a non-system path (/opt/local, see below) that almost certainly isn't on your system's search path for shared libraries, you'll very likely run into problems down the road. A static build has all the pieces in one place, so you can install and run it from wherever. This is at the expense of size - the python binary will be rather large - but is great for non-sys admins. Even if you end up installing to /usr/local, I would argue that static is better/easier than shared.
--enable-unicode=ucs4 is optional, and may not be compatible with your system. You don't need it. ./configure is smart enough to figure out what Unicode settings are best. This option is left over from build instructions that are quite a few versions out of date.
--prefix I would suggest you use --prefix=/opt/local if that directory already exists and is in your $PATH, or if you know how to edit your $PATH in ~/.bashrc. Otherwise, use /usr/local or $HOME. /usr/local is the designated system-wide location for local software installs (i.e., stuff that doesn't come with Ubuntu), and is likely already on your $PATH. $HOME is always an option that doesn't require the use of sudo, which is great from a security perspective. You'll need to add /home/your_username/bin to your $PATH if it isn't already present.
First, my reasons to do this - I know it's a bad idea but I am out of ideas.
I want to install a package which requires a ld version, which is higher than the one in the repo of my Centos 6.5. So I should either go for setting up everything in a Docker and running it in production - something I lack experience with and I don't feel comfortable doing for a serious project. Or upgrade ld manually building from external source. Which I read, could result in devastation of my Centos. So the last option I am left with is install the packed on other machine and manually copy it to site-packages.
I have successfully installed the package on my home laptop under Debian.
I encountered everywhere advice to copy the whole site-packages directory. Something which I don't want to do as I have different packages on both machines and I want to avoid messing up with other stuff.
I copied the .so build and .egginfo of the package. Then, on the target machine, pip freeze indeed showed me the transferred package. However, Python can't find it when I try to import and use it.
Am I missing something else?
Not any of that.
Don't mess with system Python's site-packages dir, this belongs to the system Python env only. You should only add/remove code in there by using the package manager of your OS (that's yum for CentOS). This is especially true in Linux where many OS services can rely on system Python.
So what to do instead? Use a virtualenv and/or pipx to isolate any other dependencies of the package you want to install from the system versions.
I downloaded and built a new installation of Python on my Ubuntu 12.04 system without realizing that Ubuntu already comes with it installed.
Whatever I did messed things up as one of the modules I need is no longer working. Is there a way to revert back to the original install?
Thx!
Just never build anything like this using ./configure && make && make install on systems with package manager. This may (and will) cause unforeseen consequences.
In order to uninstall what you have installed without package manager cd to folder with makefile and perform make uninstall.
If you want to install other version of python alongside with your existing version, find appropriate package and use update-alternatives to choose default one.
I have three different Python 2.7s at:
/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
I use a number of packages that come from different sources. I am currently installing packages from port (MacPorts), easy_install, pip (installed by easy_install), and Mercurial. There are also some that I have to install from image or build from source. I have more control over those.
The problem is that easy_install and pip seem to be installing to one location (/Library/Frameworks/...) and MacPorts installs to another (/opt/local/Library/Frameworks/...).
What's my best action now? Delete /Library/Frameworks/.../python2.7 and move easy_install and pip to the MacPorts one at /opt/local/...? Link the two directories? Move the MacPorts installation to /Library/Frameworks/...?
How can I consolidate these Pythons? I have tried putting both site-packages locations in my path, but only certain packages are available only for one Python and not the other and others vice versa, and I need them all available at once.
It seems that you have control over the stuff you're building yourself. This is how I consolidate macports with pip:
I like using Macports for all my stuff, so I just make sure that pip and easy_install build into macports' installation of python (the one in /opt/local/...).
You can tell where pip and easy_install will install things by using:
readlink `which pip`
(those are backticks)
If you want pip to install to the macports direcectories, use macports to install pip:
sudo port install py-pip
Then, be sure that which pip points to something like:
askewchan#rock:~$ which pip
/opt/local/bin/pip
askewchan#rock:~$ readlink `which pip`
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pip-2.7
From the comments below (thanks #Jonathan and #Ned) you can do the same with easy_install but its port is called py-distribute:
sudo port install py-distribute
But as far as I know, you never need to use easy_install because anything that can be easy_installed can be piped better.
Note the port descriptions:
askewchan#rock:Tracking {master *}$ port search *easy*install*
py-pip #1.2.1 (python, www)
An easy_install replacement
askewchan#rock:Tracking {master *}$ port search py*distribute
py-distribute #0.6.35 (python, devel)
Replacement for setuptools
I suggest deciding on one and only one Python for your development work ( personally, I use distribution from Python.org )
You can't get rid of /Library/Frameworks - that's the default OSX one, and you could break things
of the two remaining Pythons, I'm assuming one is Macports and the other is Python.org -- you need to choose which one you want to be your development env and to stick with that.
I would strongly recommend against using pip or easy_install from one Python to install modules for another. The reason is that there can be differences in the compile options. It can be hard enough as-is to get certain packages to compile on OSX properly -- if you start compiling against different binaries ( which might support different architectures ) you're just going to increase your headaches.
I personally chose the following path:
I use the Python.org package for all development.
On a terminal login, I run shell scripts to prioritize my Python choice
All of my projects have their own virtualenv , and I disable system packages
When starting to work on any project, I tend to have an environment setup script. I just type in go_myproject.source ; that cds me to the right directory and runs the source /path/to/virtualenv/bin/activate to get me set up for that project.
There's a tiny bit of overhead on getting things setup, but I have been in complete heaven ever since. Managing projects and not needing to worry about dependencies/upgrades for one thing killing something else is... blissful.
While not a general solution, I install Mercurial and other Python-based applications using virtualenv. In particular, pip and easy_install will install to the respective virtual environment only and not clutter any system folder. The downside is, of course, that I will have duplicates of some packages; the advantage is that I have a clean, self-contained environment with a known version of Python (which for things such as Mercurial and other mission-critical applications is more important for me).
Another downside is that I need to link individual applications to my personal bin directory or add the bin directories of the virtual environments to my path. (Personally, I manage this with some simple scripts that do the symlinking for me.)
I sugest to move all python installations to the one place and create symlinks.
After that configure python environment to avoid problems with imports and "visibility" of the modules.
Try to use commands:
# easy_install
env PYTHONPATH=/custom/path easy_install –install-dir /custom/path
#pip
pip install --install-option="--prefix=$PREFIX_PATH" package_name
Can someone please explain to me what is going on with python in ubuntu 9.04?
I'm trying to spin up virtualenv, and the --no-site-packages flag seems to do nothing with ubuntu. I installed virtualenv 1.3.3 with easy_install (which I've upgraded to setuptools 0.6c9) and everything seems to be installed to /usr/local/lib/python2.6/dist-packages
I assume that when installing a package using apt-get, it's placed in /usr/lib/python2.6/dist-packages/ ?
The issue is, there is a /usr/local/lib/python2.6/site-packages as well that just sits there being empty. It would seem (by looking at the path in a virtualenv) that this is the folder virtualenv uses as backup. Thus even thought I omit --no-site-packages, I cant access my local systems packages from any of my virtualenv's.
So my questions are:
How do I get virtualenv to point to one of the dist-packages?
Which dist-packages should I point it to? /usr/lib/python2.6/dist-packages or /usr/local/lib/python2.6/dist-packages/
What is the point of /usr/lib/python2.6/site-packages? There's nothing in there!
Is it first come first serve on the path? If I have a newer version of package XYZ installed in /usr/local/lib/python2.6/dist-packages/ and and older one (from ubuntu repos/apt-get) in /usr/lib/python2.6/dist-packages, which one gets imported when I import xyz? I'm assuming this is based on the path list, yes?
Why the hell is this so confusing? Is there something I'm missing here?
Where is it defined that easy_install should install to /usr/local/lib/python2.6/dist-packages?
Will this affect pip as well?
Thanks to anyone who can clear this up!
I believe Mike Orr's answer from the virtual-env mailing list seems to be the best. Note the OP published this question in both places.
Original content of mail:
Years ago Debian created /usr/local/lib/pythonVERSION/site-packages,
and compiled the Python binary to include it in the default search
path. Ubuntu followed Debian's lead as it normally does. The Python
developers did not like this because you'd get interference with a
locally-installed /usr/local/bin/python using the same site-packages
directory. Ubuntu finally decided to abandon site-packages and use
dist-packages instead, a name which they invented so it wouldn't
interfere with anything. The loing story is out there somewhere if
you google it, somewhere in the Python bug tracker or distutils SIG or
such.
The system works, at least if you use the Ubuntu virtualenv package.
Some people have had problems using a locally-installed virtualenv on
Ubuntu because the magic sys.path entries weren't being added or
something. I'm not sure about --no-site-packages because I never use
that option: I run PIL and mysqldb from the Ubuntu packages because it
can be hard to compile their C dependencies sometimes. (Need the
right header files, Python is ignoring the header files, etc.)
So Ubuntu Python packages go into
/usr/lib/pythonVERSION/dist-packages. Or that python-support
directory for some reason. Locally-installed Python packages go into
/usr/local/lib/pythonVERSION/dist-packages by default. Whenever I
install an Ubuntu 9.04 system I run:
$ sudo apt-get install python-setuptools (6.0c9)
$ sudo apt-get install python-virtualenv (1.3.3)
$ sudo easy_install pip
$ sudo pip install virtualenvwrapper
The virtualenvs work fine this way, although I haven't tried --no-site-packages.
I'm trying to spin up virtualenv, and the --no-site-packages flag
seems to do nothing with ubuntu. I installed virtualenv 1.3.3 with
easy_install (which I've upgraded to setuptools 0.6c9)
These versions are both in Ubuntu 9.04, so you're making it harder on
yourself by installing them locally.
and everything
seems to be installed to /usr/local/lib/python2.6/dist-packages
Yes
I assume that when installing a package using apt-get, it's placed in /
usr/lib/python2.6/dist-packages/ ?
Yes
Is it first come first serve on the path? If I have a newer
version of package XYZ installed in /usr/local/lib/python2.6/dist-
packages/ and and older one (from ubuntu repos/apt-get) in /usr/lib/
python2.6/dist-packages, which one gets imported when I import xyz?
I'm assuming this is based on the path list, yes?
sys.path is scanned in order. The only funny thing is that .pth eggs
get put earlier or later in the path than some people expect. But if
you're using pip for everything it can do (i.e. except to install pip
itself, precompiled eggs, and a snapshot of a local directory that's a
copy rather than an egg link), you won't have many .pth eggs anyway.
Why the hell is this so confusing? Is there something I'm
missing here?
It's not well documented. I figured it out by scanning the web.
Will this affect pip as well?
Yes, pip will automatically install to
/usr/local/lib/pythonVERSION/site-packages. Use "pip install -E
$VIRTUAL_ENV packagename" to install into a virtualenv.
I'd be tempted to hack it by making site-packages a link to dist-packages, but I guess this might affect other cases where you want to install some extension other than from the ubuntu dist. I can't think of another answer to 1 except tweaking virtualenv's sources (with both ubuntu and virtualenv being so popular I wouldn't be surprised to find tweaked versions already exist).
Re 2, if you're using /usr/local/bin/python you should use the /usr/local version of the lib (including site-packages) and conversely if you're using /usr/bin/python.
Re 3, there will be something there if you ever install an extension for /usr/bin/python from sources (not via easy_install or from ubuntu's distro).
Re 4, yes, earlier entries on the path take precedence.
Re 5, easy_install is easy only in its name -- it does so much dark magic that it's been carefully kept out of the standard python library despite its convenience because the consensus among us python committers is that deep dark magic for convenience is "easy" only on the surface.
Re 6, I think that's an ubuntu modification to easy_install -- if that's right then it's defined wherever Canonical or other ubuntu maintainers make their collective decisions.
Re 7, sorry, no idea -- I have no reasonably recent ubuntu at hand to check.
You really should not touch Ubuntu's Python installation unless you are building system admin tools, or building something that could be considered to be a new system service.
If you are using Ubuntu to develop or deploy Python applications, always build your own Python from source, tar it up, and use that for deployment. That way you will have all the directories in the right place and virtualenv will work normally. If you will deploy several Python apps on the server, then make your Python live in some place like /home/python or /opt/python or somewhere outside of your home directory. Make sure that you have write permissions for the developers group (users?) so that people can easily add packages.
This also allows you to have two tiers of packages. The ones that are your in-house standard tools can be installed in your Python distro and be part of the tarball that you deploy, and only the app-specific packages would be in a virtualenv.
Do not upgrade or modify the Ubuntu system installed Python.
Well I have a Ubuntu 9.04 and quickly tried setting up a couple sandboxes with site-packages and one without. And things are working fine.
The only difference in the approach I took is I used Ubuntu's python-virtualenv package (1.3.3). And presume that it is tweaked by Ubuntu team to suit Ubuntu setups.
To sum up disable easy_installed virtualenv for a while, use packaged python-virtualenv and see if that meets your expectations.
In fact we use similar setup for production without any problem. Rest is already answered by Alex.
Another way to fix it:
https://stackoverflow.com/a/17265840/202168
Have to remember to do that in each virtualenv where you need it, but doesn't rely on hacks or a special version of virtualenv