I'm on Ubuntu 22.04. On a default install there is no python version.
I've installed pyenv in order to use specific python-based programs.
Some of the programs I want to install have dependencies that are usually installed using pip. However, because there is no system python on-board, there is no pip.
Now, I could go and install system python, in which case I'd have access to pip... and also install python3 in order to use install python3 deps. But this seems like overkill to me. And for all I know, I might be creating complications that hit me further down the road.
So how do I do dependency management when using managed python versions?
Related
I have a Windows 7 machine running Python 3.8.5 with a very large number of physics/electronics/data analysis/simulation packages. As it turned out, I must have - for some inexplicable reason - installed the 32-bit version of Python instead of the 64-bit one despite having a 64-bit system. And I didn't notice until very recently when I was trying to install some packages that require 64-bit Python. Hence I've now downloaded and installed the latest Python version that is supported by Windows 7, which seems to be 3.8.10.
Question: What is the easiest and also fail-safe way to reinstall all the user packages - that I currently have under 3.8.5 - to 3.8.10?
For some reason, I couldn't find any "canonical" solution for this online. As it seems, Python does not come with any built-in support for updating or system migration and I'm honestly wondering why...
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how. Reason: Doing help('modules') inside the interpreter will list all packages and I don't see a way to "selectively apply" pip to a specific Python version, e.g. something like python-3.8.5 -m pip list --local is not supported.
After getting a list of the user packages, I was thinking to pack it into a batch command pip install package_1 package_2 <...> package_N, thus reinstalling everything to Python 3.8.10. And afterwards uninstalling Python 3.8.5 and removing all environment variables from system PATH.
Is this the proper way to do this?
Anyway, my first idea was to get a list of all user (= "local"?) packages currently installed under 3.8.5, but I don't know how.
Create a list of installed packages with pip freeze > pkglist.txt or pip list --format=freeze. If you already have one, that's great.
Then uninstall 32-bit Python 3.8.5 and clean your path for all Python related variables. Now, install 64-bit Python 3.8.10.
After reinstalling, you can install back all the packages with pip install -r pkglist.txt and it will restore the exact versions of the packages.
If you insist on having both 32-bit and 64-bit versions installed and also have the Python Launcher installed, you could invoke 32 and 64 bit versions separately with py -3.8-64 -m pip and py -3.8-32 -m pip.
I don't see a way to "selectively apply" pip to a specific Python version.
This is possible with the Python Launcher on Windows. But only between major/minor versions and not the patch versions according to its help message.
I would also recommend creating a virtual environment this time before installing the packages and leaving the root environment alone. You can create one named venv with just python -m venv venv, activate it with ./venv/Scripts/activate and proceed with the installation of packages.
Nope, doesn't work. After installing the packages with the newer Python version in PATH, e.g. Jupyter won't start.
If the Jupyter error persists, you could try pinning packages to their most recent patch/minor versions to update them and yet not break your code.
As a last resort, you could try installing Python 3.10 alongside your current Python installation (without uninstall or editing the PATH) and then installing the absolute latest versions of the packages in a 3.10 virtual environment to see if it works for you. You would invoke the two versions with Py Launcher, e.g. py -3.10 and py -3.8.
If I understood correctly, you have multiple packages like NumPy, pandas etc. installed on your machine, and you want to reinstall them "automatically" on a fresh installation of python.
The method (I use) to perform such an operation is by creating a file named setup.py which includes a list of all the packages.
Bellow, I am attaching an example of such a file I use in one of my projects:
from setuptools import setup, find_packages
setup(
name='surface_quality_tools',
version='0.1',
install_requires=["matplotlib", "psutil", "numpy", "scipy", "pandas", "trimesh", "pyglet", "networkx", "protobuf",
"numpy-stl", "sklearn", "opencv-python", "seaborn", "scikit-image", "flask", "tqdm", "pytest"],
package_data={'': ['*.json']},
packages=find_packages(include=[])
)
to run the installation you should open a command prompt from inside the project directory and run:
pip install -e .
You can find a nice example in this blog page
One common way of handling packages in Python is via virtual environments. You can use Anaconda (conda), venv or any of several other solutions. For example, see this post:
https://towardsdatascience.com/virtual-environments-104c62d48c54#:~:text=A%20virtual%20environment%20is%20a,a%20system%2Dwide%20Python).
The way this works in by keeping the Python interpreter separate from the virtual environment that contains all the necessary packages.
Probably the main reason Python doesn't feature migration tools (at least as part of standard library) is because pip - the main package tool - doesn't handle conflict resolution all too well. When you update a version of Python it might so happen (especially with niche packages) that some of them won't work any more and pip often won't be able to solve the dependencies. This is why it's a good idea to keep a separate venv for different Python versions and different projects.
The other tool you could use for easy migration is Docker which is a semi-virtual machine working on top of your host OS and containing usually some linux distribution, Python along with the necessary packages necessary for running and development.
It takes a bit of time to set up a container image initially but afterwards setting everythin on a new machine or in the cloud becomes a breeze.
Listing currently installed packages is done via pip freeze command, the output of which you can then pipe into a file to keep a record of project requirements, for example pip freeze > requirements.txt.
I've got python 3.7 installed on Windows 10. The recommended way to upgrade to 3.8 appears to be to do a new installation, which means I will have both versions installed. I don't need both versions, but I would like to keep all the packages I installed for version 3.7.
How do I achieve this please? Also will new new path variable for 3.8 replace the one for 3.7?
The process for such a common use case seems strangely complex. Am I missing something?
Simple solution would be in CMD to do
pip freeze > packages.txt
This will write all your current packages to the text file 'packages.txt'
Then uninstall Python 3.7 as you would any Windows program then install Python 3.8 and in CMD do
pip install -r packages.txt
This will install all the packages that you had before.
Though I would recommend using conda as that handless Python versions and packages for you, along with environments.
One way to do this is to run:
python3.7 -m pip freeze > installed.txt
Then, after installing the new Python version you can install the packages with:
python3.8 -m pip install -r installed.txt
There is a chance that the packages you installed for your old Python installation are not compatible with the new version. For that reason it is safer to keep both Python installations and then use virtual environments for each of your projects.
You can create a virtualenv for each of your projects, using the Python version you need for that project, and install your dependencies only in the virtualenv for that specific project. This way you can avoid the situation where project A requires an old version of a certain package but project B requires a newer one. If you install all your packages globally you run into problems in this case.
See also What is a virtualenv, and why should I use one?
I would recommend moving over to conda to manage your environments.
https://docs.conda.io/projects/conda/en/latest/user-guide/install/windows.html
The current thinking for most of the development projects that I've worked on involving python is that the version and libraries are specified on a per project basis. Conda allows you to freeze the environment so that it's more portable. You can generate an environment.yml file that allows someone to recreate your environment from scratch, and you can maintain only the packages needed for a given project.
As per your original question, you can set the PYTHONPATH to point to the old and new directories. I can't guarantee that the libraries will work though since there could be version compatibility issues.
I have a python2.6 installed on Oracle linux.
I decided to use a newer python version and installed python 2.7.13.
I have also installed pip to manage packages.
Now if I use pip to install a package it still installs it to the old location: /usr/lib/python2.6/site-packages
Is there a way to force pip to install packages for the newer python 2.7 version?
Posable Repeat Post
"Use a version of pip installed against the Python instance you want to install new packages to.
In many distributions, there may be separate python2.6-pip and python2.7-pip packages, invoked with binary names such as pip-2.6 and pip-2.7. If pip is not packaged in your distribution for the desired target, you might look for a setuptools or easyinstall package, or use virtualenv (which will always include pip in a generated environment).
pip's website includes installation instructions, if you can't find anything within your distribution." ~Charles Duffy
Taken from How to install a module use pip for specific version of?
Did you check Install a Python package into a different directory using pip?.
To install in specified target directory use "pip install --target= "
Oracle Linux uses the system-installed version of Python for almost all of its command-line utilities, so changing that could irreparably damage your system.
Instead of replacing the default Python install, you should strongly consider using Python 2.7 from Software Collections instead.
Using Software Collections means that the install of Python 2.7 is separated from the system install, so you don't run the risk of damaging your system. You can then make it available to your applications via the scl tool.
I am new at Python and Ubuntu. I really struggle to decide the best way of installing Python packages on Ubuntu. I am using VIM so I cannot use Anaconda since they do no "talk".
I have two options:
Install with: pip3 install --user foo. This works find but according to python4astronomers you should never use --user. It will work very bad.
Use apt-get but then I got old packages
Use sudo pip, but that can get bad code into my system.
I do not want to use virtual environments. I want to be able to use all packages I have installed when I use Python. I normally use Jupyter.
I find it strange that Python installation is so difficult on Linux compared with Windows.
Which of these is the correct way?
I don't know what I have did but now when use pip to install a package it install it for python 3 (python3.3 folder) not for python2.7.
Another problem I installed django_debug_toolbar and now my django version is 1.6.4 not 1.3 I installed.
Now I can't remove django 1.6.4 with pip. Do you have a solution?
Learn using virtualenv. It allows you to have different environments with isolated version of Python and set of installed packages. Each created virtual environment is by default having pip installed.
You messed up things (as you know very well) as you probably reinstalled pip for another version of Python.
You might find more versions of pip in your system. Check the version of python they use (on Linux watch the shebang on first line). Use explicit path to proper pip to manage packages for related Python.
Often people install pip and rename it or give an alias - names like pip33 or pip27.
Note, that virtualenv allows creation of different environments (with different Python versions) without need to install virutalenv for each of these Pythons.
With virtualenv I would also highly recommend using virtualenvwrapper which adds a few very handy commands.
My problem came when I installed django_debug_toolbar. when I pip-installed django-debug-toolbar, the latest version of Django was installed automatically.