Problem
I can't seem to get my packages to install to the correct site-packages directory.
Background
I wanted to upgrade to python 3.3 but I found that I still need python2.6 in order to use yum (it isn't configured for python3). And so I have both on my system.
Separate pip and easy_install for each version???
The issue is that when I download a package, and run "python setup.py install" I find that it is installed in python2.6. I have tried using pip and easy_install but they do the same. I have read that a solution is to have different versions of pip (pip_2.6 and pip_3.3). But I can't find a way to download the separate versions. One work around is to give the "-d PathToSitePackages" argument in pip but this is inelegant.
other programs not using the newer version
Another issue is that I find that when I use systems such as sphinx, they use the old version of python. I can't seem to figure out how to get them to use the newer version of python.
First, Python is explicitly designed to make it easy to have Python X.Y and Python V.W at the same time if X != V. PEP 394 explains the details, but basically, python3 will always mean a 3.x version, and if python also means a 3.x version, python2 will mean a 2.x version. The same should be true for all well-behaved tightly-integrated scripts like pip or ipython.
This is a good thing, because many POSIX platforms (most linux distros, OS X, etc.) include code that depends on some particular version of Python, so you have to be able to have that version, but often you want a newer version for your own code.
On most platforms, even if X == V, things are still easy as long as Y != W, because a default installation will also give you pythonX.Y, and set things up so well-behaved scripts do the same. This part isn't mandated by the PEP, so some linux distros don't do it, or do it in a different way. But fortunately, you just want 2.6 and 3.3, so you don't have to worry about this part.
So, if you just install pip in any of the normal ways using Python X.Y, , you will get some new packages in Python X.Y's site-packages, probably a script named pip-X.Y (note that it's a hyphen, not an underscore), and usually a symlink pip and/or pip3 to that script. So, you don't have to do anything to get this.
In particular, testing this sequence (the way the documentation recommends for system-wide installation):
$ curl -O https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
$ sudo python3.3 ez_setup.py
$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
$ sudo python3.3 get-pip.py
… I ended up with exactly that.
If you didn't, you'll have to give us details on your platform, how you installed each Python (presumably 2.6 came as part of the system installation, but not 3.3), and how you installed each pip.
If you instead did sudo yum install python-pip and sudo yum install python33-pip (or python3-pip), exactly what you get is up to the distro, so it's possible that you'll end up with just, say, pip (for 2.6) and pip-3.3 (for 3.3), rather than pip-2.6 and pip-3.3 and symlinks pip and pip3. Looking at the RPM contents for various distros' packages, it looks like the standard name for Red-Hat-like systems is python3-pip, with a symlink to pip-python3, and sometimes other names besides.
Anyway, unless you mix and match different methods, you're almost certain to get some way to distinguish the two pip versions.
If worst comes to worst, and you've just got a script named pip, and whichever one you installed last overwrote the one you installed first… You can always install the first one, cp pip pip-2.6, install the second, and mv pip pip-3.3; ln -s pip-2.6 pip; ln -s pip-3.3 pip3.
Or, if you can't even do that, the script is trivial, something like this:
#!/usr/local/bin/python3.3
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.2.1','console_scripts','pip-3.3'
__requires__ = 'pip==1.2.1'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('pip==1.2.1', 'console_scripts', 'pip-3.3')()
)
And you can just make a copy and s/3.3/2.6/g the copy (you may also have to replace the 1.2.1 if you somehow installed different versions of pip into the different site-packages) and it will work.
But it really, really shouldn't come to this, however. Both setuptools and pip respect PEP 394, and any package manager that doesn't has to have some rules or its python 3 packages are useless. So, if you think these tricks are necessary, you probably did something wrong earlier.
Related
I have installed some modules using pip and whenever I try to import them in Python I am told that no module exists. I think there is something wrong with my paths. This is some terminal output, does anyone know how I can fix this?
Nicks-MacBook-Pro:~ nickporter$ which python
/usr/bin/python
Nicks-MacBook-Pro:~ nickporter$ which pip
/usr/local/bin/pip
Nicks-MacBook-Pro:~ nickporter$
You can use pip freeze to find the packages installed.
I do not know whether you use a virtualenv. If you use it, you have to source it, to activate it.
It looks like you probably have multiple installations of Python or pip (or both). First, determine if you have multiple Python binaries installed and decide on the one you want to use by default.
You clearly have /usr/bin/python...check if there is another one in /usr/local/bin/python. If you do have one in /usr/local/bin, I'm guessing you installed it with Homebrew. Pick the Python you want to use (if there are multiple) by playing with your PATH. If you want to use /usr/local/bin/python (assuming it exists) ensure /usr/local/bin comes before /usr/bin in your PATH.
Once you do that, I would remove your current pip. After you remove it, look for it again with which pip, just to make sure you don't have any old one laying around. Once you've totally removed it, reinstall.
If you're installing with Homebrew, I'd stick to using the Homebrew python and pip. If you're not using Homebrew I'd install pip using: https://bootstrap.pypa.io/get-pip.py
curl -q https://bootstrap.pypa.io/get-pip.py | sudo python
You have homebrew version of python shadowed by the system python, but because system python doesn't have pip, a pip you see is actually a brew one.
I found, that the reason for this is that brew by default names python2.7 as python2 in /usr/local/bin (probably for preventing possible compatibility issues with some system services (?)) , i.e. you may try to run python2 or try to ln /usr/local/bin/python2 /usr/local/bin/python and restart the term session.
Make sure that /usr/local/bin goes before /usr/bin and /bin in your $PATH.
BTW. brew info {package} often contains recommendations for proper configuration of packages.
I need to work with Both Python 2.7.12 and python 3.5.2 simultaneously on my Ubuntu 16.04.1 LTS. Python 3 came pre-installed so I've no idea where it sits, in terms of path to the directory, while python 2 sits in /usr/local/lib/python2.7/.
I found lots of questions on SO and on askubuntu about how to install but nothing about how to use them separately, installing different libraries, and what should I avoid or be careful of, if I maintain this dual python thing for the long term? For example, I usually run pip install to install a library and I can check that its installed in my python2 directory but how do I install the same package for my python3 without conflicts? Something like: python3 pip install <package> ?? Where is the default python3 installed? And how do I call python3 for paths where python is not part of the command for example: pip freeze, sudo-apt get, etc.?
PS: I've not officially worked with Virtualenv but I've been informed that is usually good for isolating projects within a python language version, rather than isolating two different language versions from each other.
Please let me know.
Thanks
This is absolutely no problem, as Python does that for you. You don't need a virtualenv at all.
If you use Ubuntu packages, make sure you use the python3- versions for Python 3, and the normal python- versions for Python 2.
For example, python3-numpy and python-numpy.
If you use pip to install extra packages, you an either use the pip script with the version number appended: pip2.7 or pip3.5, or, my preferred method, call pip as a module for the respective Python executable:
python2.7 -m pip install <whatever>
and
python3.5 -m pip install <whatever>
Other than that, there should not be any issue: Python stores the packages in completely separate directories, and each Python executable only uses its respective directive.
Do not fiddle around with PYTHONPATH, unless you really know what you're doing. This has the danger of setting your PYTHONPATH to a directory with Python 2.7 modules and then using Python 3.5 to run things.
If you start from scratch, you may need to install pip first.
For the system Python(s), use the relevant package:
sudo apt install python-pip
sudo apt install python3-pip
For your locally installed Python(s), use the built-in bootstrapper module:
pythonx.y -m ensurepip
Note on the OS-installed Python executables:
Python 3.5 lives at /usr/bin/python3.5, Python 2.7 (the OS one) at /usr/bin/python2.7.
You could even use the OS 2.7 one next to your locally installed /usr/local/bin/python2.7 (and confuse yourself when a package can't be found because you used the wrong one).
Or install Python 3.6 next to Python 3.5 (provided you've used make altinstall, so python3 doesn't get overwritten).
This is also why you don't really want to run pip (or even pip2.7) as is: pip2.7 may get you the system one, instead of the one in /usr/local/bin/pip2.7, depending on your PATH.
(The same goes for the python2.7 executable, so if you need to specify the full path /usr/local/bin/python2.7 to run that one (or have an alias), the same holds for pip2.7. If, on the other hand, /usr/local/bin is first on your PATH, you should in principle never run into the same pip and python executables.)
I have the requests module installed on my system.
pip install requests
Now I am trying to import requests in the rpel
import requests
It fails with the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
The most common reason for this is that you have two versions of Python 2.x, and the pip that comes first in your PATH doesn't go with the python that comes first in your PATH.
There are two ways that can happen.
First, you may have, e.g., /usr/local/bin before /usr/bin on your PATH, but your /usr/local copy of Python doesn't have pip. So, when you run pip install requests, that's /usr/bin/pip, which installs into /usr/lib/python2.7/site-packages. But when you run python, that's /usr/local/bin/python, which looks in /usr/local/lib/python2.7/site-packages.
Second, even though your two Python 2.x's are in different locations, they may want to install pip (and other scripts and executables) to the same place. In particular, /usr/bin is usually reserved for stuff that comes with the OS or its package manager, so if you use /usr/bin/python ez_setup.py or /usr/bin/easy_install pip or many other common ways to install pip, it may end up in /usr/local/bin. In which case it will overwrite any earlier /usr/local/bin/pip that went with your /usr/local/bin/python. At any rate, the result is basically the same: pip now means /usr/local/bin/pip, but it still goes with your /usr Python, not your /usr/local Python, and installs into /usr/lib/python2.7/site-packages, which /usr/local/bin/python can't see.
If your two versions are, e.g., 2.7 and 3.4, there's no problem; per PEP 394, either the 3.x versions of everything have to be run with python3 and pip3 and so on, or the 2.x versions have to be run with python2 and pip2 and so on.
If your two versions are, e.g., 2.6 and 2.7, there is a problem, but you can easily work around it, because you should always have pip2.6 and python2.6 vs. pip2.7 and python2.7. You can confuse yourself with python and pip, but you don't have to.
If your two versions are both 2.7, however, there's no way to disambiguate (except by using complete absolute paths all the time, which no one wants to do).
So, why would anyone ever install two copies of Python 2 without knowing what they're doing?
The most common reason is that they're on a Mac, which comes with Python 2.7, but they read a blog post that told them to install another Python and didn't explain how to know what they're doing. Apple's pre-installed Python is in /usr/bin but installs scripts and binaries to /usr/local/bin. The most popular alternative Python versions are the python.org installer and Homebrew, both of which install to /usr/local/bin by default. The fact that Mac users tend to be less Unix-savvy than Linux or FreeBSD users probably doesn't help, but even without that, this is a perfect way to end up with thousands of people who have a pip and a python that doesn't match, and no idea why.
There used to be good reasons for almost all Mac Python users to installing a second Python. Until OS X 10.6, Apple's pre-installed Python versions tended to be badly out of date, and sometimes broken. If Apple's only giving you 2.4, it makes sense to install 2.6. And doing so is no problem, because python2.4 and python2.6 are easy to disambiguate. But Apple has been installing a working 2.7 for years now. There are sometimes good reasons why you need a different one (you need a bug fix in 2.7.7 but Apple gave you 2.7.5, you need a 32-bit build, you need an extra-batteries version like Enthought, you need to build py2app bundles out of it, …), but these reasons do not apply to most people anymore.
In fact, many people on StackOverflow seem to have three versions of Python 2.7. I'm not sure why this is so common, but they'll use Homebrew to install Python 2.7, and then use an installer from Python.org or Enthought, and now they've got three Python 2.7 versions all fighting over ownership of /usr/local/bin.
So, how can you fix this?
If you can use Python 3.x, install that and just use pip3 and python3 (and ipython3 and so on), and paths aren't an issue anymore.
If you don't need a second Python 2.7, get rid of the non-Apple one and just use Apple's.
Otherwise, do not ever use Apple's Python, do not install things for it, do not touch it; just leave it alone for Apple's own tools. If you use Homebrew, its Python should be higher on the PATH (make sure you've got /usr/local/bin before /usr/bin), and it should let you pip install foo without sudo, while Apple's won't, which makes it hard to accidentally screw up and install to the wrong one.
I've also seen at least one Windows user who had both C:\Python27 and D:\Python27, both on the PATH, with the C: one first, but pip only installed for D:. This seems to be far less common than the Mac confusion (probably because Windows doesn't come with Python, and there are no package managers, so the only way you're going to get any Python is by running an installer). And the solution is even simpler here: Windows doesn't need Python, so you can delete whichever one you want.
Finally, on non-Mac *nix systems, especially RHEL/CentOS Linux, you may have a Python 2.6 or 2.4 that's needed by the OS plus a Python 2.7 that you installed because you needed it, or a 2.7 that's needed by the OS and a 2.5 installed as a dependency for some "compatibility" package, or similar. Either way, you can easily accidentally install the pip for the one you don't actually use (especially if you install it with the pip bootstrap instead of your package manager).
The solution here is pretty simple: uninstall that pip, and use yum or apt or whatever to install the python-pip that goes with the Python 2.7 you want to use. And get in the habit of using python2.7 and pip2.7—or just add aliases to your profile so that python or py or whatever you prefer runs python2.7.
For devs with similar problem: Intall python3 version directly from the pack file on their website.
DO NOT DELETE OR EDIT ANYTHING ON usr/local/bin !
rm -rf /Library/Frameworks/Python.framework/Versions/2.7
rm -rf "/Applications/Python 2.7"
Install python3 directly from package on official website.
Reopen VSCode, (if you don't use it, you must) re-install modules in "Not Found" state, run command:
pip3 install requests
and other "not found" modules by command pip3 install xxxxxx
Add "python.pythonPath": "/usr/local/bin/python3" to your settings.json file.
pip install request
-bash: pip: command not found
or
no module name requests
how to fix errors
first:
Download the following and install with python http://get-pip.py
https://bootstrap.pypa.io/get-pip.py
[go to the link and download the python script that I provided and then run it or right click and save as]
then after running script "python space (drag and drop the script in terminal)" run "pip install requests" if you get the error below;
-bash: pip: command not found
See the path mentioned in the warning.
then follow this steps acordilly
see the path in yellow?yeah
/Users/macbookair/Library/... etc
type cd in terminal and then drag and drop the bin folder
1.cd /Users/macbookair/Library/Python/2.7/bin {cd /Users/name/path/Python/version/bin )
./pip install requests
after theses steps
Now run your python script again it should work
up vote me if it helped you.
Are you using Linux and have both python2 and python3 installed?
If so, you installation by:
pip install requests
would install the module to python2
So if you run import in python3, it may cause the problem.
Try to call:
pip3 install requests
to install the module in python3 environment.
This is a beginner python installation question. This the first time I have tried to install and call a package. I've got pip installed, and I tried to install two modules - numpy and pandas.
In terminal, I ran the following commands:
sudo pip install numpy
sudo pip install pandas
Both commands returned with a success message. Here is the pandas success message (it's the second package I installed and was still in my terminal history):
Successfully installed pandas
Cleaning up...
pip returned a similar message after numpy was installed.
Now, when I launch python and try to call it with:
import pandas
I get this error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pandas
Same when I try numpy.
Can anyone tell me what I'm doing incorrectly?
argh. you've got two pythons in your path that are the same version? don't do that.
pip, easy-install, etc are associated with a particular python install and will use that python by default. so if you have a system-provided python and a system-provided easy_install (or installed easy_install yourself using the system python) then easy_install will, by default, install packages for the system python.
the best way to avoid this mess, imho, is to use use system python for that version (2.7 probably) and, for other versions, use make alt-install when installing, which will give you executables like python3.1 and the like. if you really need to replace the version provided by the system, uninstall it.
once you've done that. each python will have a distinct name (ending in the version) and python will remain the system one.
next, when you install easy_install, you'll notice that there are version-specific versions (easy_install-2.7 for example). use those. if one is missing, then install distutils with the appropriate python (eg use python3.1 and you'll get an easy_install-3.1). unfortunately, each time you do this (iirc) you overwrite the un-versioned easy_install, so never use that - always use the versioned one.
alternatively, you could not install easy_install or pip for anything other than the system version, then always use virtualenv. virtualenv will let you specify a python version (so you can use the system virtualenv for all pythons installed) and then installs easy_install/pip for the python you use. so once you're inside the virtual environment, everything just works.
and i just realised i haven't much experience with pip, so i can't actually help with that (except to note that virtualenv does provide it) (about which is preferable: it used to be that pip was better maintained; i think these days the latest distutils/easy_install is as good as pip, but pip has a few more features that i have never used).
disclaimer: the above is from experience gained developing lepl, which runs on 2.6 to 3.2, so i need to test it on all those. as far as i know, what i describe above works for me, but i have no deep knowledge of python/easy_install/pip so i may have some mistakes in rationalising/describing things (in other words, i'm writing all this in case it helps, but i'm a bit worried i have an error - please, someone correct me if so).
With this, I solve the problem (may help you):
$ sudo apt-get install python-pandas
$ sudo apt-get install python-numpy
In ubuntu 9.10, in usr/lib/ there are the directories python2.4, python2.5, python2.6 and python3.0
Only python 2.6 is actually working.
python2.4 has only a lib-dynload directory,
python2.5 has only lib-dynload and site-packages,
python3.0 has only a dist-packages directory.
Now i'm wondering what is the idea behind this?
Because when i install python2.5 with ./configure, make, make install | altinstall
this goes into usr/local/lib and not usr/lib/ so why were these directories added tu ubuntu, how am i supposed to install python to use them?
j3ll3, in Ubuntu (or any DPKG-based Linux OS) you can ask the question "What package provides XYZ" by typing
dpkg -S /path/to/XYZ
So, for example, in Ubuntu 9.10,
dpkg -S /usr/lib/python2.5/lib-dynload/gdbm.so
returns
python-gdbm: /usr/lib/python2.5/lib-dynload/gdbm.so
You can find out more about the python-gdbm package by typing
apt-cache show python-gdbm
which says that python-gdbm provides "GNU dbm database support for Python".
Perhaps more interestingly, if you type
dpkg --listfiles python-gdbm
you get to see a listing of all the files that python-gdbm installs:
...
/usr/lib/python2.4
/usr/lib/python2.4/lib-dynload
/usr/lib/python2.4/lib-dynload/gdbm.so
/usr/lib/python2.5
/usr/lib/python2.5/lib-dynload
/usr/lib/python2.5/lib-dynload/gdbm.so
/usr/lib/python2.6
/usr/lib/python2.6/lib-dynload
/usr/lib/python2.6/lib-dynload/gdbm.so
...
So it looks like this single package installs 3 .so libraries, one for each version of python.
Python2.6 is the default version of python in Ubuntu 9.10, but it is also possible to install
python2.4, 2.5 and/or 3.0. Unless you do so, only /usr/lib/python2.6/lib-dynload/gdbm.so is used, the others are just wasting space.
Since the unneeded files in python2.4, 2.5, 3.0 are not very large, the package maintainer probably felt it was easier to ship one package rather than one for each version of python.
However, unless you know how to fix future apt-get errors, I'd recommend not manually deleting any files that were installed by packages in Ubuntu.
Sounds like they're an accident from some package(s) you have installed.
The Python version in use determines the locations searched to find installed Python packages/modules, and the "system version" of Python in Ubuntu 9.10 is 2.6, so that's what practically everything should be using. If you were to install the python2.5 package (or it gets installed as a dependency of something else), then it would use /usr/lib/python2.5/*. Try running which python and python --version; also which python2.6 and which python2.5.
From what I understand, though I'm not sure exactly why at all, Debian (from which Ubuntu is derived) uses a dist-packages naming scheme instead of site-packages.
Terminology: Python has packages and Debian (and so Ubuntu) has packages. They aren't the same kind of package, though individual Debian packages will install specific Python packages.
The short answer to your question: when you install packages from source, you should use the packages' setup.py installer to install them automatically and correctly. This installer already knows where to properly install the modules so Python can find them. To use, simply call with the exact Python interpreter you want to use the package with.
A crash course in setup.py. First, run it with the exact Python executable that you want the package to be available to. If you want to use the package with /usr/bin/python2.5, you should use /usr/bin/python2.5 to run setup.py. Second, go to the directory where that package's setup.py is installed. Third, you must install as root, so it's easiest to do the whole tihng as root. Fourth, if you want to install to multiple Python interpreters, you should run setup.py with each, but you should clean it in between. So here's what I would do:
% cd /root/directory/of/untarred/source/package
% sudo su
# /path/to/first/python setup.py build install
# rm -rf build
# /path/to/second/python setup.py build install
# rm -rf build
# exit
%
If you're installing modules by hand... you shouldn't, you should use its setup.py. (If you wrote a new module, you should write a setup.py for it.) If you must install by hand, you'll need to figure out which is the proper directory to install into for each Python, either by exploration and experimentation, or by calling into the same libraries that the installer calls to determine the proper directory. Installers using distutils call distutils.sysconfig.get_python_lib(); installers using setup_tools look in setup_tools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"].
Regarding dist-packages: I had a conversation with the maintainer of the Python package for Debian earlier this year. He'd implemented this dist-packages in the beta packages picked up by Ubuntu 9.04, but the code had a bug wrt PYTHONUSERBASE which I tripped over. We wound up talking a little. IIRC the reason for dist-packages had something to do with forcing the user to install packages in a different directory from apt-get. I clearly don't really understand the motivation, though, because in practice both the user and apt-get still install into the same directory.
lib-dynload isn't a Debian thing; that's a directory Python itself installs. I believe it was a directory just for shared libraries implementing modules. I'm not sure Python still uses it.
Finally, I don't know what you mean by "only python2.6 is actually working". What about these differently-named directories is "not working"?
I'm not sure what you mean by "Only python 2.6 is actually working." Suppose you run the "terminal emulator" and get a command-line prompt. Is this what you mean:
% python -V
Python 2.6
In other words, when you run Python, you get version 2.6? Well, have you tried this:
% python2.4
If Python 2.4 is correctly installed on your system, it will run. Likewise python2.5 will run Python 2.5.
If these don't run, and that is what you meant by "Only python 2.6 is actually working.", then one thing to try is to make sure that you actually have Ubuntu packages installed for Python 2.4 and Python 2.5.
% sudo apt-get install python2.4 python2.5
If you didn't have them installed before, this should add them. My thought is that you might have various libraries to support the older versions of Python, but you just don't have the actual Ubuntu packages for those older versions.