I went ahead and downloaded the latest version of python3 onto my mac from python.org/downloads/, however, I was having trouble using pip in my terminal, so I used Homebrew to install python via the command line. It looks like Homebrew installed a 2.x version. I also know that my mac already has some python2.x version built in. I would like to remove the 2.x versions and only use the python3.6.1 that I have. How can I accomplish this?
There are several aspects to your question: invoking different major Python versions, removing unneeded Homebrew-installed packages, and identifying which installation is in use.
Major Python versions, 2 and 3, use different command names to invoke them. The first, python, uses the system default according to your configured PATH environment variable and should default to Python 2, but does not on all systems. Reference: https://www.python.org/dev/peps/pep-0394/
Instead, explicitly invoke the version you want, e.g. by running python3. Note that virtual environments (created by running python3 -m venv <name> or virtualenv) will create an isolated environment using the Python version of your choice where python will always use that version, and pip will install packages relative to, instead of system-wide, which may avoid the requirement of superuser (sudo) privileges.
Note that Homebrew might not override the default versions provided by the system, requiring you to make changes to your PATH to force Homebrew's versions to be preferred. After installing a package Homebrew will inform you of this type of information, and you can get the messages later by running: brew info <package>
Uninstalling a brew-installed package requires running: brew uninstall <package>
For more information, run: brew help
As mentioned above, identifying just which Python you're using starts with the PATH. To quickly identify the executable you invoke when typing python, python2, or python3 into your shell is by running: which python — this can help identify if you're using the Homebrew installed version or the system version, or if you are using a version from an active virtual environment. You can also use which to identify the pip or pip3 command location.
This gets you the first part of the equation. The second then boils down to: where does Python think things are? Invoking it then running the following will tell you exactly where it thinks things (imports) are:
import sys, print
pprint(sys.path)
This can help track down issues related to why you can't import that package you just installed with pip.
I can highly recommend using a virtual environment to isolate your projects from system level packages and differences. Within one, python and pip will basically always behave as expected, as you determined when creating the environment.
Related
I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.
If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page
One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.
I just realized that pip was somehow uninstalled and all my packages are missing. My Apps have stopped running on local environment.
I was attempting to upgrade pandas using pip3 and conda. I had the environment up and running fine until then.
Is there a way to recover installed packages or restore the environment?
When I run pip3 list, I get:
Package Version
---------- -------
pip 10.0.1
setuptools 39.0.1
Probably there isn't any easy way to restore the packages. You could inspect your console output because pip shows which packages are uninstalled and just install them again.
Good practice for next time is to store packages needed for each script in requirements.txt file and also separating environments so that each script has its own virtual environment with packages in required version. You can read more about venv here.
It seems like, your Python (manually or by your IDE) has been updated. One who encounters this problem maybe give a chance to change the environment paths order. Search for the "environment variables" on Windows. And check if you have already installed two different Python version. If you have so; you may change your older Python version's order to an upper position of newly installed version. This may help, but as mentioned at the first answer, using virtual environment for further projects is the best-practice.
I have python2.7 and python3.6 installed side by side in my computer. Now when I install a package using "pip install", how can I know in which python's site-packages is my package going to be installed?
Thank you.
When you have both version 2 and 3 installations pip and pip3 differentiate the target installtion.
For installing anything on Python 3(versions 3.5 and above) use pip3
for Python 2.7 use pip
Make sure python path is set in environment variables too.
also you can use where pip or which pip as #mshsayem mentioned.
Additional Reference
if you use virtualenv, the modules are located in:
{path_to_your_virtualenv}/lib/python{your_python_version}/site-packages/
and if you don't use virtualenv, normally are installed in:
/usr/local/lib/python{your_python_version}
You have to use pip3 for install python3 modules.
Check where a specific package is installed by:
pip3 show <package_name>
List all installed packages with install locations by:
pip3 list -v
Check the install location used by default when installed without sudo:
pip3 --version
and the location for packages installed with sudo, meaning system-wide installation:
sudo pip3 --version
You can find the location of pip by which pip. Then you view the pip executable header using head `which pip` or using your preferred editor. You can find the python interpreter location on the first line. You may have a pip2 and a pip3 executable.
By the way, you can run pip as a python module by python -m pip <command>. In this way, you can specify your python interpreter.
The answer to you question is divided to two parts:
1. Which python version the native terminal selects for me?
2. How do I specify which python version to use?
Which python version the native terminal selects for me?
In windows, the default pip that will be used is the one associated with the default python version you use. You can edit it in the PATH environmental variable (Start->find-type "Environmental" and click "Edit system variables"). Look the PATH variable and see which version of python is listed. If both versions are listed, windows will select the first.
See more information on system environmental variables here.
In Ubuntu/Linux, usually pip is associated with the native legacy version (2.7), pip3 is associated with Python3.5.x and pip3.6 is associated with Python3.6.x.
However, if you are using Unix OS (such as Ubuntu) or Mac, it is highly recommended to use virtualenv and activate it. See Official documentation to see how to use it. It's true for both Python2.7 and
Python3.6. In short, you will create a lightweight copy of you python installation without any packages, and, your installed packages will be installed within this virtual environment. Once you activate a virtual environment, the pip is associated with this environment.
How do I specify which python version to use?
You have multiple choices to specify in which environment you want to install the package. It depends if you are on Windows/Linux/MAC.
Shortly, you have the following options:
Use an IDE and let it help you manage your packages (e.g. Pycharm). Using PyCharm, you will find it very easy to use its package manager. You can also open the IDE's terminal and when you use pip, it will use the package manager of the selected interpreter. See official documentation.
Use OS native terminal and specify the version. In windows, the easiest way is to go to a command line or powershell, and type "c:\path\to\python.exe -m pip install ". On Ubuntu, use pip/pip3/pip3.6. Again, on Ubuntu it is highly recommended to use venv (virtual environment) since installing wrong package on the wrong version can interrupt the native python (Ubuntu uses python for multiple reasons such as the GNOME GUI).
Use virtual environments. You can look it up, there are plenty of threads explaining on that, as well as the Official documentation.
I recently installed pipenv using the following command: pip3 install --user pipenv. (It's also worth mentioning that I'm following Python's official guide here.) Most online resources seem to indicate that the default installation directory for user-scoped packages is at ~/.local/bin. However, it would seem that my installation of pipenv resides in ~/Library/Python/3.6/bin. I'm concerned that keeping the installation in a version specific directory (i.e. Python 3.6) could lead to problems down the road. What happened? Should I be worried?
This is entirely correct behaviour, and not something that you need to worry about.
Python packages with native compiled extensions are tied to the specific Python version into which it is installed and should not be shared. Because you can't detect a-priori what package will contain native extensions, all Python packages are installed in a version-specific location.
The --user switch installs in the User Scheme location:
With Python 2.6 came the "user scheme" for installation, which means that all Python distributions support an alternative install location that is specific to a user. The default location for each OS is explained in the python documentation for the site.USER_BASE variable. This mode of installation can be turned on by specifying the --user option to pip install.
You can always list your USER_BASE location by running:
python3 -m site
(using the same Python binary as tied to your pip command).
The Python module search path automatically includes the user location, and because that location is Python version (major.minor) specific, won't interfere with other Python versions.
~/Library/Python/3.6/ is the Mac OS X specific path used when you have a framework build. You can override the path by setting the PYTHONUSERBASE environment variable.
In the usr/bin folder, there are three versions of Pythons installed: Python, Python2.6, Python2.7 (the folder names) - not sure what version is for Python folder.
My issue, originally, was that I tried to install the module 'pandas' to run a script, python keeps telling me pandas could not be found.
'pip freeze' shows me pandas is already installed.
However, I could not find pandas using python>>help>>modules.
So I suspected there are multiple pythons installed causing pip installing for one of them, but the default python is a different one.
So my questions are -
1 Which python is the default one that comes with macOS Sierra? (I can confirm pandas currently is installed for Python, not Python2.6 or Python2.7)
2 Can I remove extra Pythons that do not have pandas?
3 How can I find what it the default Python when I type 'Python...' and how to install pandas for that python?
Solved:
Thanks for the comments and reply.
I used "python2.7 install pip" to install pip for python2.7. Then I used command 'pip2.7 install pandas'. This way, pandas is installed for the default python. (The command 'pip install pandas' on my machine, is installing for python 2.6.)
macOS Sierra uses Python 2.7 by default.
You can uninstall a version of Python as described here. However, you shouldn't need to as long as you are managing your packages and environments. If you want to maintain more control over your projects and their packages/versions, you should take a look at virtualenv. Virtualenv creates environments that have their own installation directories that don't share packages with other virtualenv environments. This is a very popular option many people use when managing projects.
You can use pip --version to see which version of Python you are installing a package for. You can also use pip list to see which packages are currently installed.
I highly recommend taking a look at virtualenv as it'll make keeping track of Python environments and their respective packages a lot easier.
Hope this helps!