How can I switch from virtualenv that uses python 2.7 to python virtualenv that uses python 3.5?
Migration from 2.x to 3.x Python has nothing to do with virtualenv. If you already built your project keeping in mind a version change, then it will not be hard.
You can just download the Python35 and install it. After that, just execute
virtualenv -p /path to your Python35 directory/python.exe name_of_env
to create a new virtual environment for Python3.5. You can imagine the new virtual environment as a fresh Python installation with no third packages.
Please note that virtualenv just creates a new environment inside your computer with the Python version you specified in the -p parameter (or if omitted the Python version that is specified in your Path).
You can then install the desired packages for your project after activating the new virtual environment (./name_of_env/Scripts/activate) by executing pip install package_name
Although keep in mind that version migration is not the simplest thing. Many things can go wrong and especially the packages version support. Most of the packages support Python 3.x but not all of them.
Related
I've installed Python 3.7, and since installed python 3.8.
I've added both their folders and script folders to PATH, and made sure 3.8 is first as I'd like that to be default.
I see that the Python scripts folder has pip, pip3 and pip3.8 and the python 3.7 folder has the same (but with pip3.7 of course), so in cmd typing pip or pip3 will default to version 3.8 as I have that first in PATH.
This is great, as I can explicitly decide which pip version to run. However I don't know how to do to the same for Python. ie. run Python3.7 from cmd.
And things like Jupyter Notebooks only see a "Python 3" kernel and don't have an option for both.
How can I configure the PATH variables so I can specify which version of python3 to run?
What OS are you running? If you are running linux and used the system package panager to install python 3.8 you should be able to invoke python 3.8 by typing python3.8. Having multiple binaries named python3 in your PATH is problematic, and having python3 in your PATH point to python 3.8 instead of the system version (which is likely a lower version for your OS) will break your system's package manager. It is advisable to keep python3 in your PATH pointing to whatever the system defaults to, and use python3.8 to invoke python 3.8.
The python version that Jupyter sees will be the version from which you installed it. If you want to be able to use Jupyter with multiple python versions, create a virtual environment with your desired python version and install Jupyter in that environment. Once you activate that specific virtual env you will be sure that the jupyter command that you invoke will activate the currect python runtime.
I recommend you use pyenv a great tool for manage multiple python versions on the same system. Once installed you need to create a virtualenv, then activate the virtualenviroment and there you can install any libraries you want in a safe way.
By the way also come with an automatic installer pyenv-installer
Regards
I want to have multiple installs of Python: 2.1, 2.4, 2.7, 3.x
My IDE is Eclipse (Helios)/Pydev on MacOSX, which works great. I have a couple of Python codebases that are/will be running on different versions of Python. Also, I like Eclipse PyDev's crosslinking from source-code to documentation.
The standard recommendation seems to be: use virtualenv, and keep the installs totally separate from the builtin MacPython (2.6.1). Eclipse should never be pointing to the MacPython install. (Should PYTHONPATH even be set in such an environment?)
Before I get on with virtualenv, is there anything else I should know about this?
virtualenv doesn't impose any overhead, and I shouldn't be worried with occasional comments about breakage to nose, coverage etc?
I'm interested in hearing from Eclipse or Pydev users on MacOS.
Also if anyone has other tips on migrating a Python codebase from 2.1 -> 2.7.
A good way is to use macport to install the different version of pythons. It will have different versions of all packages for all versions of pythons that you want. They will be installed in /opt/local/.
So in Eclipse with PyDev for a particular project you can right click into the name of the project -> properties: There, in the left you choose PyDev - Interpreter/Grammar. And you click the link Click here to configure an interpreter not lister. You click on new on the top right of the new window. You give the name of the version of python you want to create the interpreter for, if you have already installed it with all required package through macport. And in the field Intepreter Executable you give the path: /opt/local/bin/pythonX.X .
After, in the previous option window: PyDev - Intepreter/Grammar, you can choose in the Interpreter menu the python version that you just installed.
For the shell, and the default path pointing to /usr/bin/python you must use: python_select (installed through macport) instead of playing with the env variables. Afterward you can use macport to update/install new packages, make sure everything is alway clean.
Having Home Brew already installed in your system, I recommend you pyenv. Most of the following information comes from this quick installation/use guide:
Installing pyenv + pyenv-virtualenv
Go to your terminal and install pyenv and the external libraries needed by Python:
brew update
brew install pyenv openssl readline sqlite3 xz zlib
Add pyenv init to your shell to enable shims and autocompletion running the following command:
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
Install the pyenv-virtualenv plugin so you can create virtual environments for each version of Python:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
Reset your terminal to apply the changes exec "$SHELL" or just close your terminal and open a new one.
Gettings different Python versions
We can list all available versions with pyenv install --list:
pyenv install --list
Available versions:
2.7.15
3.0.1
..
3.7.2
For installing the Python version 3.7.2:
pyenv install 3.7.2
With pyenv versions you can see all the Python versions installed in your computer:
root#Host ~$ pyenv versions
* 3.7.2
Creating a virtual environment
In the previous step we have downloaded the Python 3.7.2 interpreter, we can now use it to create an isolated virtual environment. This is very useful for software development, keeping each project completely isolated from the others:
pyenv virtualenv 3.7.2 MY_VIRTUALENV_NAME
To list all your virtual environments:
pyenv virtualenvs
To activate a virtual environment:
pyenv activate MY_VIRTUALENV_NAME
To deactivate the currently active virtual environment:
pyenv deactivate
From the README text file of python
Installing multiple versions
On Unix and Mac systems if you intend
to install multiple versions of Python
using the same installation prefix
(--prefix argument to the configure
script) you must take care that your
primary python executable is not
overwritten by the installation of a
different version. All files and
directories installed using "make
altinstall" contain the major and
minor version and can thus live
side-by-side. "make install" also
creates ${prefix}/bin/python which
refers to ${prefix}/bin/pythonX.Y. If
you intend to install multiple
versions using the same prefix you
must decide which version (if any) is
your "primary" version. Install that
version using "make install". Install
all other versions using "make
altinstall".
For example, if you want to install
Python 2.5, 2.6 and 3.0 with 2.6 being
the primary version, you would execute
"make install" in your 2.6 build
directory and "make altinstall" in the
others.
Virtualenv is an option but you could use the above mentioned option instead of venv which seems much simpler.
Personally, I use conda to create multiple environments (mostly, you create a new env, activate it and install the packages you want there -- using conda itself if possible or pip if it's not available in conda). See: https://conda.io/docs/installation.html.
After you have an environment created, you just need to add a new interpreter inside PyDev which points to the interpreter in the created environment. Ssee: http://www.pydev.org/manual_101_interpreter.html.
EDIT FOR CLARITY: This question is in reference to venv, which is often confused with virtualenv.
What are the consequences of removing the system's Python installation which was the source used to create virtual environments on the system using venv?
Ultimately, what I'm looking to do is remove the 3.7 installation and install 3.8 when it is released, but I'm unsure of how strongly tethered a virtual environment is to the system environment.
Some system details that may be relevant when asking this question:
OS = Red Hat Enterprise Linux 7
3.7 installed from source at Python.org, not the rh-Pythonx.x packge in the yum repo.
Python 3.7 is installed to the default location, /usr/local/bin/python3.7
/usr/local/bin/python3 is a link to /usr/local/bin/python3.7
Given the above details, I am under the impression that Python 3.8 would install to /usr/local/bin/python3.8 and python3 would then be pointing to that version.
Does the virtual environment have enough within it to be a stable 3.7 environment of its own? Or will it fall apart, trying to reference a system install that no longer exists?
tl;dr:
What happens to all those virtual environments that were created from a Python source that is no longer existent / replaced with a newer version?
Bad news: You're using venv, and venv virtual environments are not completely standalone. Even when created with --copies, all it copies is the python executable itself, not the standard libraries, not even (if your install creates a shared libpython rather than a static one) libpython; it relies on the system copy. The virtual environment will break if the Python install it was based on goes away. How it breaks will differ based on how it was created. For example, if you created it with:
python3 -mvenv path/to/venv
when python3 meant Python 3.7, then replaced python3 with Python 3.8, then you can fix up the virtual env for the new version with:
python3 -mvenv --upgrade path/to/venv
but your installed third party packages will be (effectively) gone (they'll be in path/to/venv/lib/python3.7, but Python 3.8 will only be looking in path/to/venv/lib/python3.8), so you'll have to reinstall them.
If you created the virtual environment with:
python3.7 -mvenv path/to/venv
then it's completely broken (at least as documented), the --upgrade switch is only documented to work for upgrading when Python was upgraded in place; since the new Python would not be named python3.7, you can't upgrade in place. That said, --upgrade really only works well when upgrading micro versions (from 3.7.1 to 3.7.2 or the like), due to aforementioned per-minor version lib/pythonX.Y directories, so either way, you're best off just creating a new virtual environment from scratch.
To be clear, the third party virtualenv package does not have this limitation if and only if the system Python installation statically linked libpython. Oddly, while the --always-copy flag will make it copy the main binary and the standard library modules, it doesn't cause libpython itself (the interpreter core) to be copied, so if the main binary depends on a system copy of libpython.so, then removing the system copy breaks the virtual environment. If you did use --always-copy and your python executable statically linked libpython.a (ldd /path/to/python3 should show no libpython dependency), then yes, virtualenv makes much more heavyweight (in local tests on 3.6, forcing copies via the appropriate switch, freshly created venv environments were ~11 MB, while virtualenv environments were ~48 MB; sadly, my python dynamically links libpython.so, so it still wouldn't work) virtual environments that should survive the removal of the system installed copy of Python.
In any event, it's probably best to leave your Python 3.7 install in place, and just upgrade to 3.8 without removing 3.7 (are you really hard up for a few dozen MB of disk space?). Even if you replace python3 with the new 3.8 install, python3.7, libpython3.7m.so.1.0 or the like and the rest of the 3.7 standard library will continue to exist for the virtual environment to depend on; at worst you might need to manually change the symlink in the virtual environment to point to /path/to/python3.7 rather than /path/to/python3 to keep using the old version (including all your installed third party packages).
An alternative to trying to keep the old virtual environment working is to just back up the installed state of that virtual environment, delete it, install the new Python, create a fresh virtual environment, and then use the backed up state to reinstall all your packages in the upgraded virtual env. An example might be:
$ source ~/path/to/venv/bin/activate
$ pip freeze > installed_libs.txt
$ deactivate
$ rm -rf ~/path/to/venv
$ ... install new Python/remove old Python ...
$ python3 -mvenv ~/path/to/venv
$ pip install -r installed_libs.txt # Optionally add --upgrade to install latest, not fixed versions
A virtualenv includes a copy of the python binary itself, so nothing should happen. See a more detailed explanation here
virtualenv solves this problem by creating a completely isolated virtual environment for each of your programs. An environment is simply a directory that contains a complete copy of everything needed to run a Python program, including a copy of the python binary itself, a copy of the entire Python standard library, a copy of the pip installer, and (crucially) a copy of the site-packages directory mentioned above. When you install a package from PyPI using the copy of pip that's created by the virtualenv tool, it will install the package into the site-packages directory inside the virtualenv directory. You can then use it in your program just as before.
Edit (accounting for venv vs virtualenv): According to the venv-documentation, it can either copy or symlink the binary:
The venv module provides support for creating lightweight “virtual environments” with their own site directories, optionally isolated from system site directories. Each virtual environment has its own Python binary (which matches the version of the binary that was used to create this environment) and can have its own independent set of installed Python packages in its site directories.
I have installed Python 2.7, pip and virtualenv on my computer and I have created a virtual environment with this same version of Python. I installed some packages in this new virtual environment too.
Later, I decided to install Python 3.6 on the same computer to create a virtual environment that works with version 3.6. I downloaded Python 3.6 and installed it by selecting Add Python 3.6 to PATH and Install for All users and I selected the option to automatically install pip. As I had previously installed virtualenv, I have not had to reinstall it. Should I?
The problem is that, once installed, when creating a virtual environment I've got an error.
As I already had a version of Python installed I have specified that I want to use Python 3.6 in the virtual environment that I am creating:
virtualenv env2_py36 -p c: \ Python36 \ python.exe
The error already says that this usually happens when you install more than one version of Python on the same computer and warns that it is necessary to check the option Install for All Users to work (as I did) and also recommends placing the appropiate PythonXX.dll file in the Scripts folder of the virtual environment. However, I see that there is already a python36.dll file there.
Could anybody tell me what should I do to fix this problem?
On the other hand, I had previously created another virtual environment using Python 2.7. I use PyScripter to run my scripts but I don't know how to connect PyScripter to my virtual environment. It's still connected to the Python2.7's system installation.
This works for me:
virtualenv --python=python3.6 yourenvname
(3.6 is the current latest version of Python, but you can use whichever version you'd like.)
Using Python I require both python 2.7 and python 3.5 for different packages. I am trying to install the following package NepidemiX. I get an error when I do this as I have a newer version of python installed.
To combat this I am trying to create a virtual environment. To do this I am using the virtualenv package.
I have created and activated this and am now faced with
(my_project)Your-Computer:your_project UserName$)
In my terminal.
How do I now proceed to install my package from here? Do I need to install python 2.7 in this environment first, or do I simply copy the desired package into the environment ... ?
Please could you instruct me how to correctly set this up?
Many thanks!
Virtual environment is only for libraries. It uses python versions installed on your computer. You can specify the version of python by using the -p attribute while creating the environment, for ex. virtualenv -p python3 env creates a python 3 enviroment (provided you have it installed in your computer and on the PATH). Check this answer.
After you activate the environment (source /env/bin/activate), just pip install libraries, and the environment takes care of installing the correct version.