I have renamed python.exe to python37.exe to avoid conflict with other versions. It works for running python, but if I run pip37.exe (located in /Scripts) I get the following error:
Fatal error in launcher: Unable to create process using '"c:\python37-32\python.exe" "C:\Python37-32\Scripts\pip37.exe"
Is there a way to keep python.exe renamed to python37.exe, but keep all python tools working?
This sounds like A BAD IDEA.
There are tools designed to help you manage exactly this sort of thing. The best of which imho is pyenv: https://github.com/pyenv/pyenv
It's quite simple to install. It takes a bit of getting used to – wrapping your head around virtual environments – but it makes it all so much easier to work with ultimately.
E.g. On my system I have the following versions of python:
pyenv versions
system
2.7.10
* 3.5.6 (set by /Users/.pyenv/version)
3.5.6/envs/core4
3.6.4
3.6.4/envs/core5
core4
core5
The one with the asterisk is currently the global version, which will be the one used from any default shell. I can change that using pyenv global 3.6.4 for example. I can also create virtual environments. E.g. core4 and core5 are virtual environments I created for specific projects. Each of these will have their own different libraries installed by pip install and differing python versions. You can activate a virtualenv for a given shell session e.g. pyenv activate core5.
And if you're thinking "what on earth does this have to do with Windows", look here: https://duckduckgo.com/?q=Windows+Subsystem+for+Linux&atb=v93-1__&ia=web and here: http://timmyreilly.azurewebsites.net/python-pip-virtualenv-installation-on-windows/
On Windows Python installs PyLauncher. You don't need virtual environments or renaming tricks. py.exe is in the standard Windows path, has command line switches to pick the Python version to use, and enables using "shebangs" to specify what version of Python to run for scripts:
py script.py # Run the latest Python installed (or specified by PY_PYTHON environment variable).
py -2 script.py # Run the latest Python 2 version installed.
py -3 script.py # Run the latest Python 3 version installed.
py -2.7 script.py # Run the specific Python version.
py -2.7-32 script.py # Run the 32-bit specific Python version.
py -0 # List Python versions installed.
Scripts can use shebangs similar to Linux:
#!python2
#!python3
#!python2.7
#!python2.7-32
To run pip with a specific version:
py -2.7 -m pip install ...
If you need still would like a virtual environment with a specific Python version, you can specify the version (e.g. -3) and use:
py -3 -m venv <my_env_name> # to create an virtual environment
<my_env_name>/scripts/activate # to activate that environment
Activation adds the virtual environment to the path, so python (not py) will run in that environment. the Scripts directory in that environment will also be added to the path, so pip can be run directly as well to install packages in that environment.
Related
I am currently running python 3.9 on my system. Due to the incompatibility of some of python libraries like numba on python 3.9 I will have to use previous versions. What cmd commands will I have to use to create a virtual environment so that I could run python 3.6 on cmd separate from python 3.9. I have a windows 10 64 bit system, I dont use conda or anything.
CMD Output:
C:\Users\maste>python -V
Python 3.9.1
In Windows you can utilize the Python Launcher for Windows. You can just install multiple Python versions from Python.org, and then use
py -3.6 -m venv venv
To create a virtual environment called "venv" (the last argument is the name). After that just activate your virtual environment and python will point to the Python 3.6 in your virtual environment.
Instead of the py launcher, you can also just use the full file path of the Python 3.6 (assuming Powershell, hence &):
& "C:\path\to\python 3.6\python.exe" -m venv venv
Lastly, you don't have to activate the virtual environment, if you don't want to. You can just run
<path-to-project>\venv\Scripts\python.exe myscript.py
I've installed Python 3.7, and since installed python 3.8.
I've added both their folders and script folders to PATH, and made sure 3.8 is first as I'd like that to be default.
I see that the Python scripts folder has pip, pip3 and pip3.8 and the python 3.7 folder has the same (but with pip3.7 of course), so in cmd typing pip or pip3 will default to version 3.8 as I have that first in PATH.
This is great, as I can explicitly decide which pip version to run. However I don't know how to do to the same for Python. ie. run Python3.7 from cmd.
And things like Jupyter Notebooks only see a "Python 3" kernel and don't have an option for both.
How can I configure the PATH variables so I can specify which version of python3 to run?
What OS are you running? If you are running linux and used the system package panager to install python 3.8 you should be able to invoke python 3.8 by typing python3.8. Having multiple binaries named python3 in your PATH is problematic, and having python3 in your PATH point to python 3.8 instead of the system version (which is likely a lower version for your OS) will break your system's package manager. It is advisable to keep python3 in your PATH pointing to whatever the system defaults to, and use python3.8 to invoke python 3.8.
The python version that Jupyter sees will be the version from which you installed it. If you want to be able to use Jupyter with multiple python versions, create a virtual environment with your desired python version and install Jupyter in that environment. Once you activate that specific virtual env you will be sure that the jupyter command that you invoke will activate the currect python runtime.
I recommend you use pyenv a great tool for manage multiple python versions on the same system. Once installed you need to create a virtualenv, then activate the virtualenviroment and there you can install any libraries you want in a safe way.
By the way also come with an automatic installer pyenv-installer
Regards
I want to have multiple installs of Python: 2.1, 2.4, 2.7, 3.x
My IDE is Eclipse (Helios)/Pydev on MacOSX, which works great. I have a couple of Python codebases that are/will be running on different versions of Python. Also, I like Eclipse PyDev's crosslinking from source-code to documentation.
The standard recommendation seems to be: use virtualenv, and keep the installs totally separate from the builtin MacPython (2.6.1). Eclipse should never be pointing to the MacPython install. (Should PYTHONPATH even be set in such an environment?)
Before I get on with virtualenv, is there anything else I should know about this?
virtualenv doesn't impose any overhead, and I shouldn't be worried with occasional comments about breakage to nose, coverage etc?
I'm interested in hearing from Eclipse or Pydev users on MacOS.
Also if anyone has other tips on migrating a Python codebase from 2.1 -> 2.7.
A good way is to use macport to install the different version of pythons. It will have different versions of all packages for all versions of pythons that you want. They will be installed in /opt/local/.
So in Eclipse with PyDev for a particular project you can right click into the name of the project -> properties: There, in the left you choose PyDev - Interpreter/Grammar. And you click the link Click here to configure an interpreter not lister. You click on new on the top right of the new window. You give the name of the version of python you want to create the interpreter for, if you have already installed it with all required package through macport. And in the field Intepreter Executable you give the path: /opt/local/bin/pythonX.X .
After, in the previous option window: PyDev - Intepreter/Grammar, you can choose in the Interpreter menu the python version that you just installed.
For the shell, and the default path pointing to /usr/bin/python you must use: python_select (installed through macport) instead of playing with the env variables. Afterward you can use macport to update/install new packages, make sure everything is alway clean.
Having Home Brew already installed in your system, I recommend you pyenv. Most of the following information comes from this quick installation/use guide:
Installing pyenv + pyenv-virtualenv
Go to your terminal and install pyenv and the external libraries needed by Python:
brew update
brew install pyenv openssl readline sqlite3 xz zlib
Add pyenv init to your shell to enable shims and autocompletion running the following command:
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
Install the pyenv-virtualenv plugin so you can create virtual environments for each version of Python:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
Reset your terminal to apply the changes exec "$SHELL" or just close your terminal and open a new one.
Gettings different Python versions
We can list all available versions with pyenv install --list:
pyenv install --list
Available versions:
2.7.15
3.0.1
..
3.7.2
For installing the Python version 3.7.2:
pyenv install 3.7.2
With pyenv versions you can see all the Python versions installed in your computer:
root#Host ~$ pyenv versions
* 3.7.2
Creating a virtual environment
In the previous step we have downloaded the Python 3.7.2 interpreter, we can now use it to create an isolated virtual environment. This is very useful for software development, keeping each project completely isolated from the others:
pyenv virtualenv 3.7.2 MY_VIRTUALENV_NAME
To list all your virtual environments:
pyenv virtualenvs
To activate a virtual environment:
pyenv activate MY_VIRTUALENV_NAME
To deactivate the currently active virtual environment:
pyenv deactivate
From the README text file of python
Installing multiple versions
On Unix and Mac systems if you intend
to install multiple versions of Python
using the same installation prefix
(--prefix argument to the configure
script) you must take care that your
primary python executable is not
overwritten by the installation of a
different version. All files and
directories installed using "make
altinstall" contain the major and
minor version and can thus live
side-by-side. "make install" also
creates ${prefix}/bin/python which
refers to ${prefix}/bin/pythonX.Y. If
you intend to install multiple
versions using the same prefix you
must decide which version (if any) is
your "primary" version. Install that
version using "make install". Install
all other versions using "make
altinstall".
For example, if you want to install
Python 2.5, 2.6 and 3.0 with 2.6 being
the primary version, you would execute
"make install" in your 2.6 build
directory and "make altinstall" in the
others.
Virtualenv is an option but you could use the above mentioned option instead of venv which seems much simpler.
Personally, I use conda to create multiple environments (mostly, you create a new env, activate it and install the packages you want there -- using conda itself if possible or pip if it's not available in conda). See: https://conda.io/docs/installation.html.
After you have an environment created, you just need to add a new interpreter inside PyDev which points to the interpreter in the created environment. Ssee: http://www.pydev.org/manual_101_interpreter.html.
Presently we have a big-data cluster built using Cloudera-Virtual machines. By default the Python version on the VM is 2.7.
For one of my programs I need Python 3.6. My team is very skeptical about 2 installations and afraid of breaking existing cluster/VM. I was planning to follow this article and install 2 versions https://www.digitalocean.com/community/tutorials/how-to-set-up-python-2-7-6-and-3-3-3-on-centos-6-4
Is there a way "I can package Python 3.6" version in my project, and set the Python home path to my project folder, so that there is no installation that needs to be done on the existing Virtual machine?
Since we have to download python and build source for the Unix version, I want to skip this part on VM, and instead ship the folder which has Python 3.6
It seems that miniconda is what you need.
using it you can manage multiple python environments with different versions of python.
to install miniconda3 just run:
# this will download & install miniconda3 on your home dir
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3
then, create new python3.6 env:
conda create -y -n myproject 'python>3.6'
now, enter the new python3.6 env
source activate myproject
python3
miniconda can also install python packages, including pip packages and compiled packages. you can also copy envs from one machine to another. I encourage you to take a deeper look into it.
ShmulikA's suggestion is pretty good.
Here I'd like to add another one - I use Python 2.7.x, but for few prototypes, I had to go with Python 3.x. For this I used the pyenv utility.
Once installed, all you have to do is:
pyenv install 3.x.x
Can list all the available Python variants:
pyenv versions
To use the specific version, while at the project root, execute the following:
pyenv local 3.x.x
It'll create a file .python-version at the project root, having the version as it's content:
[nahmed#localhost ~]$ cat some-project/.python-version
3.5.2
Example:
[nahmed#localhost ~]$ pyenv versions
* system (set by /home/nahmed/.pyenv/version)
3.5.2
3.5.2/envs/venv_scrapy
venv_scrapy
[nahmed#localhost ~]$ pyenv local 3.5.2
[nahmed#localhost ~]$ pyenv versions
system
* 3.5.2 (set by /home/nahmed/.python-version)
3.5.2/envs/venv_scrapy
venv_scrapy
I found it very simple to use.
Here's a post regarding the installation and basic usage (blog post by me).
For the part:
Since we have to download python and build source for the Unix
version, I want to skip this part on VM, and instead ship the folder
which has Python 3.6
You might look into ways to embed Python interpreter with your Python application:
And for both Windows and Linux, there's bbfreeze or also pyinstaller
from - SOAnswer.
How can I switch from virtualenv that uses python 2.7 to python virtualenv that uses python 3.5?
Migration from 2.x to 3.x Python has nothing to do with virtualenv. If you already built your project keeping in mind a version change, then it will not be hard.
You can just download the Python35 and install it. After that, just execute
virtualenv -p /path to your Python35 directory/python.exe name_of_env
to create a new virtual environment for Python3.5. You can imagine the new virtual environment as a fresh Python installation with no third packages.
Please note that virtualenv just creates a new environment inside your computer with the Python version you specified in the -p parameter (or if omitted the Python version that is specified in your Path).
You can then install the desired packages for your project after activating the new virtual environment (./name_of_env/Scripts/activate) by executing pip install package_name
Although keep in mind that version migration is not the simplest thing. Many things can go wrong and especially the packages version support. Most of the packages support Python 3.x but not all of them.