I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.
If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page
One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.
I have noticed that when I install new modules using pip install example, it installs correctly but when I try importing it in a script it says: ModuleNotFoundError: No module named 'example'. To get the modules working I have to copy the module's folder to the directory of my script. Before this started happening I had installed some modules which also work now so I'm a bit confused.
I'm guessing that there's something wrong with some PATH but I'm not really sure. How can I fix this?
Edit: I'm almost 100% sure that these problems started happening when I installed Anaconda. I have made sure Anaconda's folder is added to PATH but I still have the same issue.
With Anaconda, you would benefit from installing new packages by conda install example instead of using pip.
However, it is likely that you have installed another copy of Python that comes with Anaconda, and you should check which Python you are using by typing python -V in your command prompt or checking where it is installed (for Windows machines, check here). You might have installed the package to use with one version of Python, but your system directed you to the other one.
I have uninstalled my standalone Python distribution after installing Anaconda, so my default version of Python is the version that comes with Anaconda.
For my project requirements I had installed Python packages like Jira and Bitbucket to connect to the servers using their API wrappers. After I installed Anaconda all the existing packages stopped working. Now Im getting No Module Found error for both Bitbucket and JIRA although the modules are already installed
For example :
pip3 install bitbucket-python
gives me that requirement is already satisfied. But when I run the code I get:
No module named "bitbucket" found
The same code was running fine 2 days ago before I installed anaconda.
Please help.
Now you have different versions of python and pip in your system. You have to make sure you are running correct version of the command. In my system with conda installed and activated pip3 refers to pip for system python. and pip for active conda python.
Try this: pip install bitbucket-python
you can use which command to see which binary is getting executed for example on my system.
dhananjay#ideapad:~$ which python
/home/dhananjay/.conda/bin/python
dhananjay#ideapad:~$ which pip
/home/dhananjay/.conda/bin/pip
dhananjay#ideapad:~$ which pip3
/usr/bin/pip3
please help installing SciPy. I think i've tried almost any advice i could find, but still no luck.
I am using Mac High OS Sierra 10.13.1, python 3.7, trying to make this work for IntellIJ IDEA 2017.2 IDE. I have Xcode version 9.2 if that helps.
I've tried instaling from IDE, using package installer -> fails with error status code 1.
I've tried installing using pip3 install scipy
I've installed brew install gcc ( I have version 7.2.0 installed)
I've installed numpy.
I've tried to install using Macports as suggested on official site - didn't help.
I've tried to instal using brew install scipy.
I've tried to install using Conda. It installed somewhere to Conda Dir, but i still cannot access library from python file using import scipy, error: No module named scipy.
Looks like i am just going in loops now, can some one suggest any idea please?
Topics i researched:
SciPy build/install Mac Osx
Can't install Scipy through pip
"failed with error code 1" while installing scipy
Some other ones i lost links to.
Ok looks like i made it work.
This thread: helped me.
It appeared I was actaully able to install scipy package using conda. But my Python didn't see the package. So i had to:
Change Right Click on my project in IDE > Project > New > Python SDK > Add Local
Select Python in Conda dir. In my case it was /Users/[my user ]/miniconda3/bin/python3.6
Restart IDE just in case and my python script was able to see import scipy.
The downside of this - that I cannot install packages the clean way from IDE anymore for some reason. I.e. Tools > Manage Python Packages > + > doesn't find any. And I have to reinstall all the packages i had using conda install [package name] from terminal . But I am fine with it, as long as it works.
Hopefully my quest might be useful for someone.
I have recently begun having troubles using pip to install python packages. I have always used pip but never really understood how it actually works, my experience with it is basically limited to "pip install pkg".
Recently when trying to install openCV on my machine, I followed a few guides that involved changing paths etc. Since making these changes I have been having trouble using pip to install packages correctly.
Now when I run "pip3 install pkg", the install runs fine without any errors. When I try to import the module in python however, python cannot find the package. If I run "pip3 list" in the terminal I get a list of modules that is different to running help('modules') within python.
I think pip is installing the packages to a different location than my version of python is referencing when importing modules?
Is there a way I can change where pip installs to? What did it mean to change paths and how can I avoid this in the future?
Thanks in advance.
EDIT: I should mention that running "python3 -m pip install pkg" installs the packages correctly.
Because you have 2 versions of python installed, the best solution is to install and use virtualenv
A Virtual Environment is a tool to keep all dependencies required by different projects and python versions in separate places. It solves the problem you mentioned and keeps your site-packages directory manageable.