I've seen quite a few posts on this topic, but there seems to be little agreement or definitive answer. I understand that OSX (in my case 10.10.5) comes with Python2.7 installed, as detailed in the official python docs.
After following those instructions, I now have python 2.7 and python 3.4 installed (accessible by $ python and $ python3, respectively) and perhaps more versions. I find python versions in a few places:
/Applications/Python 3.4
/usr/local/bin (contains python, python2, python2.7, python3, python3.4
/usr//bin (contains python2.6, python2.7)
/System/Library/Frameworks/Python.framework/Versions (contains 2.3, 2.5, 2.6, 2.7, Current)
Moreover, I'm still having trouble when I install python packages...and I think it's because I don't completely understand where certain packages are installed, their PATH variables and how they are aliased.
Problem:
For example, after successfully installing nose with pip install nose, I cannot use $ nosetests because the command is not found.
Suggested solutions:
Inspect the nose directories and look for nosetests to alias it (but despite there being many other files within /nose, regardless of the version, there isn't anything about nosetests).
Another suggestion is to run python setup.py install within /nose but that simply gives this error:
/usr/local/Cellar/python/2.7.10_1/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'setup.py': [Errno 2] No such file or directory
Some posts suggested simply using sudo as a fix to the problem...but I have been warned against using sudo without understanding why I need it, and the nose docs don't mention needing sudo, so I don't want to simply put a band-aid over a deeper issue that may arise later.
My original question was basically two questions: one about conflicting versions of python and the other about installing the nose package specifically.
For the first question, I found jonrsharpe's initial comment to my question very helpful. He links to a tutorial that explains virtualenv and how to set it up. However, I was still uncertain how to install packages only within that virtualenv, because even when I'm in the virtualenv directory and I pip install, the package is still listed under pip freeze when outside of that virtualenv directory.
Solution:
This tutorial was helpful in further explaining virtualenv, what it is and how to use it. In short, virtualenv creates a copy of python in the designated virtualenv directory that has its OWN pip. That local pip must be used to install packages locally for that virtualenv, referencing it as [virtualenv_dir_you_created]/bin/pip install nose
And, by using virtualenv, problems with permission for writing to packages globally disappear.
Related
I have a question regarding installing a python package like pathlib2 for the python interpreter that comes with a software called Nuke for example.
In this case the python executeable is located in the following directory
C:\Program Files\Nuke11.1v6\python.exe
Because it is shipped by a piece of software, it looks a little different to your default python installation, at least for my knowledge. For example, I believe the modules are located inside pythonextensions\site-packages and I cannot find the typical "Scripts" folder in which pip.py should be.
The closest thing I found is C:\Program Files\Nuke11.1v6\lib\ensurepip\_bundled with pip-9.0.1-py2.py3-none-any.whl and setuptools....... but so far nothing else
What I tried:
I cd'ed into the Nuke folder and ran python.exe -m pip install pathlib2 which returned with the error No module named pip, as predicted.
I am sure it is bad practice to change the python interpreter of a software but I have no other option. Therefore I would like to modify the python version as less as possible (like using the pip it comes with if possible, instead of installing a new pip).
Second question: does a pip install always install with the compiler for the current python version in mind? For example, this article explained that the module numpy uses c compiled code, which has to be compiled against the same compiler of the python installation.
Thanks for the help.
You have ensurepip, good. First try
"C:\Program Files\Nuke11.1v6\python.exe" -m ensurepip
to install pip. Now do
"C:\Program Files\Nuke11.1v6\python.exe" -m pip install pathlib2
You better remove the second question and ask it in a separate topic.
I recently installed pipenv using the following command: pip3 install --user pipenv. (It's also worth mentioning that I'm following Python's official guide here.) Most online resources seem to indicate that the default installation directory for user-scoped packages is at ~/.local/bin. However, it would seem that my installation of pipenv resides in ~/Library/Python/3.6/bin. I'm concerned that keeping the installation in a version specific directory (i.e. Python 3.6) could lead to problems down the road. What happened? Should I be worried?
This is entirely correct behaviour, and not something that you need to worry about.
Python packages with native compiled extensions are tied to the specific Python version into which it is installed and should not be shared. Because you can't detect a-priori what package will contain native extensions, all Python packages are installed in a version-specific location.
The --user switch installs in the User Scheme location:
With Python 2.6 came the "user scheme" for installation, which means that all Python distributions support an alternative install location that is specific to a user. The default location for each OS is explained in the python documentation for the site.USER_BASE variable. This mode of installation can be turned on by specifying the --user option to pip install.
You can always list your USER_BASE location by running:
python3 -m site
(using the same Python binary as tied to your pip command).
The Python module search path automatically includes the user location, and because that location is Python version (major.minor) specific, won't interfere with other Python versions.
~/Library/Python/3.6/ is the Mac OS X specific path used when you have a framework build. You can override the path by setting the PYTHONUSERBASE environment variable.
I went ahead and downloaded the latest version of python3 onto my mac from python.org/downloads/, however, I was having trouble using pip in my terminal, so I used Homebrew to install python via the command line. It looks like Homebrew installed a 2.x version. I also know that my mac already has some python2.x version built in. I would like to remove the 2.x versions and only use the python3.6.1 that I have. How can I accomplish this?
There are several aspects to your question: invoking different major Python versions, removing unneeded Homebrew-installed packages, and identifying which installation is in use.
Major Python versions, 2 and 3, use different command names to invoke them. The first, python, uses the system default according to your configured PATH environment variable and should default to Python 2, but does not on all systems. Reference: https://www.python.org/dev/peps/pep-0394/
Instead, explicitly invoke the version you want, e.g. by running python3. Note that virtual environments (created by running python3 -m venv <name> or virtualenv) will create an isolated environment using the Python version of your choice where python will always use that version, and pip will install packages relative to, instead of system-wide, which may avoid the requirement of superuser (sudo) privileges.
Note that Homebrew might not override the default versions provided by the system, requiring you to make changes to your PATH to force Homebrew's versions to be preferred. After installing a package Homebrew will inform you of this type of information, and you can get the messages later by running: brew info <package>
Uninstalling a brew-installed package requires running: brew uninstall <package>
For more information, run: brew help
As mentioned above, identifying just which Python you're using starts with the PATH. To quickly identify the executable you invoke when typing python, python2, or python3 into your shell is by running: which python — this can help identify if you're using the Homebrew installed version or the system version, or if you are using a version from an active virtual environment. You can also use which to identify the pip or pip3 command location.
This gets you the first part of the equation. The second then boils down to: where does Python think things are? Invoking it then running the following will tell you exactly where it thinks things (imports) are:
import sys, print
pprint(sys.path)
This can help track down issues related to why you can't import that package you just installed with pip.
I can highly recommend using a virtual environment to isolate your projects from system level packages and differences. Within one, python and pip will basically always behave as expected, as you determined when creating the environment.
I have the requests module installed on my system.
pip install requests
Now I am trying to import requests in the rpel
import requests
It fails with the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
The most common reason for this is that you have two versions of Python 2.x, and the pip that comes first in your PATH doesn't go with the python that comes first in your PATH.
There are two ways that can happen.
First, you may have, e.g., /usr/local/bin before /usr/bin on your PATH, but your /usr/local copy of Python doesn't have pip. So, when you run pip install requests, that's /usr/bin/pip, which installs into /usr/lib/python2.7/site-packages. But when you run python, that's /usr/local/bin/python, which looks in /usr/local/lib/python2.7/site-packages.
Second, even though your two Python 2.x's are in different locations, they may want to install pip (and other scripts and executables) to the same place. In particular, /usr/bin is usually reserved for stuff that comes with the OS or its package manager, so if you use /usr/bin/python ez_setup.py or /usr/bin/easy_install pip or many other common ways to install pip, it may end up in /usr/local/bin. In which case it will overwrite any earlier /usr/local/bin/pip that went with your /usr/local/bin/python. At any rate, the result is basically the same: pip now means /usr/local/bin/pip, but it still goes with your /usr Python, not your /usr/local Python, and installs into /usr/lib/python2.7/site-packages, which /usr/local/bin/python can't see.
If your two versions are, e.g., 2.7 and 3.4, there's no problem; per PEP 394, either the 3.x versions of everything have to be run with python3 and pip3 and so on, or the 2.x versions have to be run with python2 and pip2 and so on.
If your two versions are, e.g., 2.6 and 2.7, there is a problem, but you can easily work around it, because you should always have pip2.6 and python2.6 vs. pip2.7 and python2.7. You can confuse yourself with python and pip, but you don't have to.
If your two versions are both 2.7, however, there's no way to disambiguate (except by using complete absolute paths all the time, which no one wants to do).
So, why would anyone ever install two copies of Python 2 without knowing what they're doing?
The most common reason is that they're on a Mac, which comes with Python 2.7, but they read a blog post that told them to install another Python and didn't explain how to know what they're doing. Apple's pre-installed Python is in /usr/bin but installs scripts and binaries to /usr/local/bin. The most popular alternative Python versions are the python.org installer and Homebrew, both of which install to /usr/local/bin by default. The fact that Mac users tend to be less Unix-savvy than Linux or FreeBSD users probably doesn't help, but even without that, this is a perfect way to end up with thousands of people who have a pip and a python that doesn't match, and no idea why.
There used to be good reasons for almost all Mac Python users to installing a second Python. Until OS X 10.6, Apple's pre-installed Python versions tended to be badly out of date, and sometimes broken. If Apple's only giving you 2.4, it makes sense to install 2.6. And doing so is no problem, because python2.4 and python2.6 are easy to disambiguate. But Apple has been installing a working 2.7 for years now. There are sometimes good reasons why you need a different one (you need a bug fix in 2.7.7 but Apple gave you 2.7.5, you need a 32-bit build, you need an extra-batteries version like Enthought, you need to build py2app bundles out of it, …), but these reasons do not apply to most people anymore.
In fact, many people on StackOverflow seem to have three versions of Python 2.7. I'm not sure why this is so common, but they'll use Homebrew to install Python 2.7, and then use an installer from Python.org or Enthought, and now they've got three Python 2.7 versions all fighting over ownership of /usr/local/bin.
So, how can you fix this?
If you can use Python 3.x, install that and just use pip3 and python3 (and ipython3 and so on), and paths aren't an issue anymore.
If you don't need a second Python 2.7, get rid of the non-Apple one and just use Apple's.
Otherwise, do not ever use Apple's Python, do not install things for it, do not touch it; just leave it alone for Apple's own tools. If you use Homebrew, its Python should be higher on the PATH (make sure you've got /usr/local/bin before /usr/bin), and it should let you pip install foo without sudo, while Apple's won't, which makes it hard to accidentally screw up and install to the wrong one.
I've also seen at least one Windows user who had both C:\Python27 and D:\Python27, both on the PATH, with the C: one first, but pip only installed for D:. This seems to be far less common than the Mac confusion (probably because Windows doesn't come with Python, and there are no package managers, so the only way you're going to get any Python is by running an installer). And the solution is even simpler here: Windows doesn't need Python, so you can delete whichever one you want.
Finally, on non-Mac *nix systems, especially RHEL/CentOS Linux, you may have a Python 2.6 or 2.4 that's needed by the OS plus a Python 2.7 that you installed because you needed it, or a 2.7 that's needed by the OS and a 2.5 installed as a dependency for some "compatibility" package, or similar. Either way, you can easily accidentally install the pip for the one you don't actually use (especially if you install it with the pip bootstrap instead of your package manager).
The solution here is pretty simple: uninstall that pip, and use yum or apt or whatever to install the python-pip that goes with the Python 2.7 you want to use. And get in the habit of using python2.7 and pip2.7—or just add aliases to your profile so that python or py or whatever you prefer runs python2.7.
For devs with similar problem: Intall python3 version directly from the pack file on their website.
DO NOT DELETE OR EDIT ANYTHING ON usr/local/bin !
rm -rf /Library/Frameworks/Python.framework/Versions/2.7
rm -rf "/Applications/Python 2.7"
Install python3 directly from package on official website.
Reopen VSCode, (if you don't use it, you must) re-install modules in "Not Found" state, run command:
pip3 install requests
and other "not found" modules by command pip3 install xxxxxx
Add "python.pythonPath": "/usr/local/bin/python3" to your settings.json file.
pip install request
-bash: pip: command not found
or
no module name requests
how to fix errors
first:
Download the following and install with python http://get-pip.py
https://bootstrap.pypa.io/get-pip.py
[go to the link and download the python script that I provided and then run it or right click and save as]
then after running script "python space (drag and drop the script in terminal)" run "pip install requests" if you get the error below;
-bash: pip: command not found
See the path mentioned in the warning.
then follow this steps acordilly
see the path in yellow?yeah
/Users/macbookair/Library/... etc
type cd in terminal and then drag and drop the bin folder
1.cd /Users/macbookair/Library/Python/2.7/bin {cd /Users/name/path/Python/version/bin )
./pip install requests
after theses steps
Now run your python script again it should work
up vote me if it helped you.
Are you using Linux and have both python2 and python3 installed?
If so, you installation by:
pip install requests
would install the module to python2
So if you run import in python3, it may cause the problem.
Try to call:
pip3 install requests
to install the module in python3 environment.
In ubuntu 9.10, in usr/lib/ there are the directories python2.4, python2.5, python2.6 and python3.0
Only python 2.6 is actually working.
python2.4 has only a lib-dynload directory,
python2.5 has only lib-dynload and site-packages,
python3.0 has only a dist-packages directory.
Now i'm wondering what is the idea behind this?
Because when i install python2.5 with ./configure, make, make install | altinstall
this goes into usr/local/lib and not usr/lib/ so why were these directories added tu ubuntu, how am i supposed to install python to use them?
j3ll3, in Ubuntu (or any DPKG-based Linux OS) you can ask the question "What package provides XYZ" by typing
dpkg -S /path/to/XYZ
So, for example, in Ubuntu 9.10,
dpkg -S /usr/lib/python2.5/lib-dynload/gdbm.so
returns
python-gdbm: /usr/lib/python2.5/lib-dynload/gdbm.so
You can find out more about the python-gdbm package by typing
apt-cache show python-gdbm
which says that python-gdbm provides "GNU dbm database support for Python".
Perhaps more interestingly, if you type
dpkg --listfiles python-gdbm
you get to see a listing of all the files that python-gdbm installs:
...
/usr/lib/python2.4
/usr/lib/python2.4/lib-dynload
/usr/lib/python2.4/lib-dynload/gdbm.so
/usr/lib/python2.5
/usr/lib/python2.5/lib-dynload
/usr/lib/python2.5/lib-dynload/gdbm.so
/usr/lib/python2.6
/usr/lib/python2.6/lib-dynload
/usr/lib/python2.6/lib-dynload/gdbm.so
...
So it looks like this single package installs 3 .so libraries, one for each version of python.
Python2.6 is the default version of python in Ubuntu 9.10, but it is also possible to install
python2.4, 2.5 and/or 3.0. Unless you do so, only /usr/lib/python2.6/lib-dynload/gdbm.so is used, the others are just wasting space.
Since the unneeded files in python2.4, 2.5, 3.0 are not very large, the package maintainer probably felt it was easier to ship one package rather than one for each version of python.
However, unless you know how to fix future apt-get errors, I'd recommend not manually deleting any files that were installed by packages in Ubuntu.
Sounds like they're an accident from some package(s) you have installed.
The Python version in use determines the locations searched to find installed Python packages/modules, and the "system version" of Python in Ubuntu 9.10 is 2.6, so that's what practically everything should be using. If you were to install the python2.5 package (or it gets installed as a dependency of something else), then it would use /usr/lib/python2.5/*. Try running which python and python --version; also which python2.6 and which python2.5.
From what I understand, though I'm not sure exactly why at all, Debian (from which Ubuntu is derived) uses a dist-packages naming scheme instead of site-packages.
Terminology: Python has packages and Debian (and so Ubuntu) has packages. They aren't the same kind of package, though individual Debian packages will install specific Python packages.
The short answer to your question: when you install packages from source, you should use the packages' setup.py installer to install them automatically and correctly. This installer already knows where to properly install the modules so Python can find them. To use, simply call with the exact Python interpreter you want to use the package with.
A crash course in setup.py. First, run it with the exact Python executable that you want the package to be available to. If you want to use the package with /usr/bin/python2.5, you should use /usr/bin/python2.5 to run setup.py. Second, go to the directory where that package's setup.py is installed. Third, you must install as root, so it's easiest to do the whole tihng as root. Fourth, if you want to install to multiple Python interpreters, you should run setup.py with each, but you should clean it in between. So here's what I would do:
% cd /root/directory/of/untarred/source/package
% sudo su
# /path/to/first/python setup.py build install
# rm -rf build
# /path/to/second/python setup.py build install
# rm -rf build
# exit
%
If you're installing modules by hand... you shouldn't, you should use its setup.py. (If you wrote a new module, you should write a setup.py for it.) If you must install by hand, you'll need to figure out which is the proper directory to install into for each Python, either by exploration and experimentation, or by calling into the same libraries that the installer calls to determine the proper directory. Installers using distutils call distutils.sysconfig.get_python_lib(); installers using setup_tools look in setup_tools.command.easy_install.easy_install.INSTALL_SCHEMES[os.name]["install_dir"].
Regarding dist-packages: I had a conversation with the maintainer of the Python package for Debian earlier this year. He'd implemented this dist-packages in the beta packages picked up by Ubuntu 9.04, but the code had a bug wrt PYTHONUSERBASE which I tripped over. We wound up talking a little. IIRC the reason for dist-packages had something to do with forcing the user to install packages in a different directory from apt-get. I clearly don't really understand the motivation, though, because in practice both the user and apt-get still install into the same directory.
lib-dynload isn't a Debian thing; that's a directory Python itself installs. I believe it was a directory just for shared libraries implementing modules. I'm not sure Python still uses it.
Finally, I don't know what you mean by "only python2.6 is actually working". What about these differently-named directories is "not working"?
I'm not sure what you mean by "Only python 2.6 is actually working." Suppose you run the "terminal emulator" and get a command-line prompt. Is this what you mean:
% python -V
Python 2.6
In other words, when you run Python, you get version 2.6? Well, have you tried this:
% python2.4
If Python 2.4 is correctly installed on your system, it will run. Likewise python2.5 will run Python 2.5.
If these don't run, and that is what you meant by "Only python 2.6 is actually working.", then one thing to try is to make sure that you actually have Ubuntu packages installed for Python 2.4 and Python 2.5.
% sudo apt-get install python2.4 python2.5
If you didn't have them installed before, this should add them. My thought is that you might have various libraries to support the older versions of Python, but you just don't have the actual Ubuntu packages for those older versions.