We Keep Coding

How to reinstall all user packages after updating Python version in Windows?

I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?

Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.

If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page

One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.

Keeping Installed Packages When Updating Python

I've got python 3.7 installed on Windows 10. The recommended way to upgrade to 3.8 appears to be to do a new installation, which means I will have both versions installed. I don't need both versions, but I would like to keep all the packages I installed for version 3.7.
How do I achieve this please? Also will new new path variable for 3.8 replace the one for 3.7?
The process for such a common use case seems strangely complex. Am I missing something?

Simple solution would be in CMD to do
pip freeze > packages.txt
This will write all your current packages to the text file 'packages.txt'
Then uninstall Python 3.7 as you would any Windows program then install Python 3.8 and in CMD do
pip install -r packages.txt
This will install all the packages that you had before.
Though I would recommend using conda as that handless Python versions and packages for you, along with environments.

One way to do this is to run:
python3.7 -m pip freeze > installed.txt
Then, after installing the new Python version you can install the packages with:
python3.8 -m pip install -r installed.txt
There is a chance that the packages you installed for your old Python installation are not compatible with the new version. For that reason it is safer to keep both Python installations and then use virtual environments for each of your projects.
You can create a virtualenv for each of your projects, using the Python version you need for that project, and install your dependencies only in the virtualenv for that specific project. This way you can avoid the situation where project A requires an old version of a certain package but project B requires a newer one. If you install all your packages globally you run into problems in this case.
See also What is a virtualenv, and why should I use one?

I would recommend moving over to conda to manage your environments.
https://docs.conda.io/projects/conda/en/latest/user-guide/install/windows.html
The current thinking for most of the development projects that I've worked on involving python is that the version and libraries are specified on a per project basis. Conda allows you to freeze the environment so that it's more portable. You can generate an environment.yml file that allows someone to recreate your environment from scratch, and you can maintain only the packages needed for a given project.
As per your original question, you can set the PYTHONPATH to point to the old and new directories. I can't guarantee that the libraries will work though since there could be version compatibility issues.

Pip install location

I have python2.7 and python3.6 installed side by side in my computer. Now when I install a package using "pip install", how can I know in which python's site-packages is my package going to be installed?
Thank you.

When you have both version 2 and 3 installations pip and pip3 differentiate the target installtion.
For installing anything on Python 3(versions 3.5 and above) use pip3
for Python 2.7 use pip
Make sure python path is set in environment variables too.
also you can use where pip or which pip as #mshsayem mentioned.
Additional Reference

if you use virtualenv, the modules are located in:
{path_to_your_virtualenv}/lib/python{your_python_version}/site-packages/
and if you don't use virtualenv, normally are installed in:
/usr/local/lib/python{your_python_version}
You have to use pip3 for install python3 modules.

Check where a specific package is installed by:
pip3 show <package_name>
List all installed packages with install locations by:
pip3 list -v
Check the install location used by default when installed without sudo:
pip3 --version
and the location for packages installed with sudo, meaning system-wide installation:
sudo pip3 --version

You can find the location of pip by which pip. Then you view the pip executable header using head `which pip` or using your preferred editor. You can find the python interpreter location on the first line. You may have a pip2 and a pip3 executable.
By the way, you can run pip as a python module by python -m pip <command>. In this way, you can specify your python interpreter.

The answer to you question is divided to two parts:
1. Which python version the native terminal selects for me?
2. How do I specify which python version to use?
Which python version the native terminal selects for me?
In windows, the default pip that will be used is the one associated with the default python version you use. You can edit it in the PATH environmental variable (Start->find-type "Environmental" and click "Edit system variables"). Look the PATH variable and see which version of python is listed. If both versions are listed, windows will select the first.
See more information on system environmental variables here.
In Ubuntu/Linux, usually pip is associated with the native legacy version (2.7), pip3 is associated with Python3.5.x and pip3.6 is associated with Python3.6.x.
However, if you are using Unix OS (such as Ubuntu) or Mac, it is highly recommended to use virtualenv and activate it. See Official documentation to see how to use it. It's true for both Python2.7 and
Python3.6. In short, you will create a lightweight copy of you python installation without any packages, and, your installed packages will be installed within this virtual environment. Once you activate a virtual environment, the pip is associated with this environment.
How do I specify which python version to use?
You have multiple choices to specify in which environment you want to install the package. It depends if you are on Windows/Linux/MAC.
Shortly, you have the following options:
Use an IDE and let it help you manage your packages (e.g. Pycharm). Using PyCharm, you will find it very easy to use its package manager. You can also open the IDE's terminal and when you use pip, it will use the package manager of the selected interpreter. See official documentation.
Use OS native terminal and specify the version. In windows, the easiest way is to go to a command line or powershell, and type "c:\path\to\python.exe -m pip install ". On Ubuntu, use pip/pip3/pip3.6. Again, on Ubuntu it is highly recommended to use venv (virtual environment) since installing wrong package on the wrong version can interrupt the native python (Ubuntu uses python for multiple reasons such as the GNOME GUI).
Use virtual environments. You can look it up, there are plenty of threads explaining on that, as well as the Official documentation.

Changing directory where pip installs packages

I have a python2.6 installed on Oracle linux.
I decided to use a newer python version and installed python 2.7.13.
I have also installed pip to manage packages.
Now if I use pip to install a package it still installs it to the old location: /usr/lib/python2.6/site-packages
Is there a way to force pip to install packages for the newer python 2.7 version?

Posable Repeat Post
"Use a version of pip installed against the Python instance you want to install new packages to.
In many distributions, there may be separate python2.6-pip and python2.7-pip packages, invoked with binary names such as pip-2.6 and pip-2.7. If pip is not packaged in your distribution for the desired target, you might look for a setuptools or easyinstall package, or use virtualenv (which will always include pip in a generated environment).
pip's website includes installation instructions, if you can't find anything within your distribution." ~Charles Duffy
Taken from How to install a module use pip for specific version of?

Did you check Install a Python package into a different directory using pip?.
To install in specified target directory use "pip install --target= "

Oracle Linux uses the system-installed version of Python for almost all of its command-line utilities, so changing that could irreparably damage your system.
Instead of replacing the default Python install, you should strongly consider using Python 2.7 from Software Collections instead.
Using Software Collections means that the install of Python 2.7 is separated from the system install, so you don't run the risk of damaging your system. You can then make it available to your applications via the scl tool.

How to delete extra pythons on Mac OS X Sierra

In the usr/bin folder, there are three versions of Pythons installed: Python, Python2.6, Python2.7 (the folder names) - not sure what version is for Python folder.
My issue, originally, was that I tried to install the module 'pandas' to run a script, python keeps telling me pandas could not be found.
'pip freeze' shows me pandas is already installed.
However, I could not find pandas using python>>help>>modules.
So I suspected there are multiple pythons installed causing pip installing for one of them, but the default python is a different one.
So my questions are -
1 Which python is the default one that comes with macOS Sierra? (I can confirm pandas currently is installed for Python, not Python2.6 or Python2.7)
2 Can I remove extra Pythons that do not have pandas?
3 How can I find what it the default Python when I type 'Python...' and how to install pandas for that python?
Solved:
Thanks for the comments and reply.
I used "python2.7 install pip" to install pip for python2.7. Then I used command 'pip2.7 install pandas'. This way, pandas is installed for the default python. (The command 'pip install pandas' on my machine, is installing for python 2.6.)

macOS Sierra uses Python 2.7 by default.
You can uninstall a version of Python as described here. However, you shouldn't need to as long as you are managing your packages and environments. If you want to maintain more control over your projects and their packages/versions, you should take a look at virtualenv. Virtualenv creates environments that have their own installation directories that don't share packages with other virtualenv environments. This is a very popular option many people use when managing projects.
You can use pip --version to see which version of Python you are installing a package for. You can also use pip list to see which packages are currently installed.
I highly recommend taking a look at virtualenv as it'll make keeping track of Python environments and their respective packages a lot easier.
Hope this helps!