I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.
If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page
One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.
I'm beginner in python, and I have to maintain a python code, which imports a self-made package.
I am able to install this package with pip, but somehow the Sypder(pyhton 3.9) does not accept it.
The error is: "ModuleNotFoundError: No module named 'myModule'
If I type the following command into the IPython console, then it says that the package is already installed.
pip install d:\MyFolder\myModule-0.2.0-py3-none-any.whl
Result:
Processing d:\MyFolder\myModule-0.2.0-py3-none-any.whl
myModule is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
Note: you may need to restart the kernel to use updated packages.
I already tried to solve this problem based this article: Spyder_modulenotfound
The packge is present in the Anaconda3 installation folder: d:\Anaconda3\Lib\site-packages\myPackage-0.2.0.dist-info\ and I added this folder into the PYTHONPATH manager tool. Already restarted the Spyder many times, still no change.
What am I missing? Please help...
Python Version: 3.2.3
I've spent hours working on this issue and I'm still scratching my head furiously out of confusion. I'm testing an application built with python in a windows 7 device. I went and installed python and its dependencies. I also installed pip and the following packages: mysql-connector, and mysqlclient. However, when I tried running the script that imports mysql.connector, the debugger returns:
import mysql.connector
ImportError: No module named mysql.connector
I went and check the list of packages via pip list and saw that the packages we're definitely installed. They are listed as:
mysql-connector-python 8.0.26
mysqlclient 2.0.3
So, I compared this with my windows 10 device that has the script running without issues and saw that the packages were also installed the same way. I thought there were no issues in the package installation.
However, when I checked the python modules in help("modules"), I found something. The mysql and pip are not listed in the windows 7 device whereas, in my personal device, they were there.
I tried uninstalling and reinstalling the packages, yet the issue remains. For some reason, python is not able to detect the packages. I also tried to upgrade the pip via python -m pip install --upgrade pip and as expected, the python can't find the pip. I'm honestly confused at this point. Am I missing something?
UPDATE
It might be worth noting that, since my python version has no pip, I had to install it and copy its path to the system environment variables manually. Could it be connected the source of issue?
I get this error for every single package that I have downloaded from pip whenever I type import [package] or any variation there off. This also means that the packages do not work. This is strange, since it used to work.
Edit: Apparently it works just fine on Jupyter Notebooks, but not in a normal python file.
it seems that you are installing the module in a wrong way, specifically, to improper directory (or python version).
you can alway check if module/package is installed or not by this:
pip list
if you can see the module in list, then it is installed, if not, then you need to install it properly.
sometimes you may have different versions of python installed, you install the package to version X and try to run it on version Z. all you need to do is to clarify your python version, check if the pip is referring to the python version by doing this:
pip -V
it shows the pip version along with the python version it is referring to.
The BEST WAY
always try to create a virtual environment and install packages there.
creating a virtual environment
python -m venv <VENV_NAME>
then activate it:
source <VENV_NAME>/bin/activate # MacOS & Linux
<VENV_NAME>/Scripts/activate # Windows
then, install your packages. there shouldn't be any problems after that.
I recently installed Anaconda on Arch Linux from the Arch repositories. By default, it was set to Python3, whereas I would like to use Python2.7. I followed the Anaconda documentation to create a new Python2 environment. Upon running my Python script which uses Numpy, I got the error No module named NumPy. I found this rather strange, as one of the major points of using Anaconda is easy installation of the NumPy/SciPy stack...
Nevertheless, I ran conda install numpy and it installed. Now, I still cannot import numpy, but when I run conda install numpy it says it is already installed. What gives?
Output of which conda: /opt/anaconda/envs/python2/bin/conda
Output of which python: /opt/anaconda/envs/python2/bin/python
The anaconda package in the AUR is broken. If anyone encounters this, simply install anaconda from their website. The AUR attempts to do a system-wide install, which gets rather screwy with the path.