I've installed pyenv and have different versions of python installed with it:
$ pyenv versions
system
2.7.1
3.2.5
3.5.0
3.5.1
* 3.5.2
I use the following command to switch to python 3.5.2:
pyenv shell 3.5.2
And when I check the python version this is what I get:
$ python --version
Python 3.5.2
But when I run pytest, it still runs under python 2.7.6:
pytest -v
==================================================================== test session starts ====================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.3, py-1.4.31, pluggy-0.4.0 -- /usr/bin/python
Why is pytest running under the older version?
Bottom line: run
python -m pytest, or
py.test-<version> if your alternative Python and pytest are installed with system package manager, or
if your alternative Python has been installed with pyenv, switch with pyenv to that version and make sure you have pytest installed for it. Then you can just run pytest.
since the pip executable is also among the ones being switched, you need to switch to the alternative Python before installing pytest for it, too.
As I can see, /usr/bin/pytest (that belongs to the system package manager's python-pytest package) has a shebang !#/usr/bin/python since it corresponds to the system python's installation.
pyenv, as its README.md says, does not replace /usr/bin/python - because it indeed should not be replaced to avoid breaking system packages.
Instead, it adds its directory to PATH and inserts a launcher there (called "shim") which is what gets invoked when you type "python". As you probably guessed, this hack is ignored by a shebang like the above - as it should.
Running python -m pytest will make whichever python that launches itself use the package from its installation.
Alternatively, pytest for your other Python version may include versioned executables on the PATH named py.test-<version> (e.g. py.test-3 or py.test-3.6) depending on the way you installed it.
If it's from a system package manager's package for nonstandard python - like python36-pytest - this is virtually guaranteed.
I checked that if you install a version with pip, it only creates an unversioned executable (though you can create a versioned one yourself). Moreover, if you install the same package for a different Python version but with the same --prefix, it will overwrite the existing one's executable!
pyenv's suggested way seems to be to install all python versions of interest and packages for them under ~/.pyenv/versions.
This is not applicable for the system's Python but the default /usr/local can be used for it.
Once you switch to an alternative Python version, it claims to create shims for all scripts (including pip!) that are currently installed for that version, so invoking those scripts without a path would run those shims.
So, if a package (and thus its script) is not installed for the alternative version but installed for system version, trying to run its executable would "fall through" to /usr/local with just the result you're seeing now.
I found this related question. For them it worked with this:
python -m pytest tests/my_test.py
I hope it works
I just wanted to run "black" for Python 2.7, but I have installed it in pyenv's Python 3.7.9. It worked correctly with the script /usr/local/bin/black27:
PYENV_VERSION=3.7.9 black -t py27 "$#"
It didn't work with:
pyenv shell 3.7.9
black -t py27 "$#"
nor
pyenv shell 3.7.9
pyenv exec black -t py27 "$#"
Related
I recently started using elementary OS. It's based on Ubuntu.
During installation it installs python 3.6.
I installed python 3.8 manually by installing the following packages: python3.8, python3.8-dev, python3.8-minimal, python3.8-venv.
I also updated the link to the python binary with:
sudo ln -sf /usr/bin/python3.8 /usr/bin/python3
After this a couple of things stopped working.
For example when I tried to execute a non-existing command it didn't print the error message that it cannot found the command, but it displayed a python stack trace. That one I solved with:
cd /usr/lib/python3/dist-packages
sudo ln -s apt_pkg.cpython-36m-x86_64-linux-gnu.so apt_pkg.so
sudo ln -s apt_inst.cpython-36m-x86_64-linux-gnu.so apt_inst.so
As you can see the python error was because it couldn't find the compiled binaries for the apt module.
So this issue was solved, but there are a couple of similar ones, and none of them can be solved this way, as the module binary is not compatible with python 3.8.
Is it possible to remove python 3.6 completely and override it with 3.8 so that the module binaries also get updated? Or can python 3.8 and 3.6 coexist? I would be fine with the /usr/bin/python3 link pointing to python 3.6 and I would manually execute /usr/bin/python3.8 or create a different alias or link for it. However when I print out the sys.path with /usr/bin/python3.8 I get this:
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/home/{username}/.local/lib/python3.8/site-packages
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
The trouble is with the last one. That's where the modules are with the 3.6 compatible so files. Can I somehow force python 3.8 to completely ignore the last module search path (without always stating sys.path.remove in my scripts) ?
I recommend you undo your symlink overwriting your systemwide version of Python 3. As you have found, replacing the python3 executable might create some problems, as it is used under the hood.
In general, you should leave your system python[3] installation alone for this reason and it is common to use virtual environments. This can be done as follows (assuming you are in some project directory):
python3.8 -m venv venv
source venv/bin/activate
The first command will create a virtual environment ('venv') in the directory venv. The second command will 'activate': now python (and in this case, python3 and python3.8) all refer to your original python3.8 in the context of this shell. You will have to repeat this if you launch a new shell.
This will also allow you to install packages using pip without cluttering your system installation. The use of virtual environments and pip is an incredibly common workflow in the Python development world.
In terms of shell and 'global' Python management, you can also use pyenv to manage your Python versions and what is available in the shell. pyenv is quite nice if you want to run a particular version of python, say 3.8.0 but not 3.8.1.
I want to have multiple installs of Python: 2.1, 2.4, 2.7, 3.x
My IDE is Eclipse (Helios)/Pydev on MacOSX, which works great. I have a couple of Python codebases that are/will be running on different versions of Python. Also, I like Eclipse PyDev's crosslinking from source-code to documentation.
The standard recommendation seems to be: use virtualenv, and keep the installs totally separate from the builtin MacPython (2.6.1). Eclipse should never be pointing to the MacPython install. (Should PYTHONPATH even be set in such an environment?)
Before I get on with virtualenv, is there anything else I should know about this?
virtualenv doesn't impose any overhead, and I shouldn't be worried with occasional comments about breakage to nose, coverage etc?
I'm interested in hearing from Eclipse or Pydev users on MacOS.
Also if anyone has other tips on migrating a Python codebase from 2.1 -> 2.7.
A good way is to use macport to install the different version of pythons. It will have different versions of all packages for all versions of pythons that you want. They will be installed in /opt/local/.
So in Eclipse with PyDev for a particular project you can right click into the name of the project -> properties: There, in the left you choose PyDev - Interpreter/Grammar. And you click the link Click here to configure an interpreter not lister. You click on new on the top right of the new window. You give the name of the version of python you want to create the interpreter for, if you have already installed it with all required package through macport. And in the field Intepreter Executable you give the path: /opt/local/bin/pythonX.X .
After, in the previous option window: PyDev - Intepreter/Grammar, you can choose in the Interpreter menu the python version that you just installed.
For the shell, and the default path pointing to /usr/bin/python you must use: python_select (installed through macport) instead of playing with the env variables. Afterward you can use macport to update/install new packages, make sure everything is alway clean.
Having Home Brew already installed in your system, I recommend you pyenv. Most of the following information comes from this quick installation/use guide:
Installing pyenv + pyenv-virtualenv
Go to your terminal and install pyenv and the external libraries needed by Python:
brew update
brew install pyenv openssl readline sqlite3 xz zlib
Add pyenv init to your shell to enable shims and autocompletion running the following command:
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
Install the pyenv-virtualenv plugin so you can create virtual environments for each version of Python:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
Reset your terminal to apply the changes exec "$SHELL" or just close your terminal and open a new one.
Gettings different Python versions
We can list all available versions with pyenv install --list:
pyenv install --list
Available versions:
2.7.15
3.0.1
..
3.7.2
For installing the Python version 3.7.2:
pyenv install 3.7.2
With pyenv versions you can see all the Python versions installed in your computer:
root#Host ~$ pyenv versions
* 3.7.2
Creating a virtual environment
In the previous step we have downloaded the Python 3.7.2 interpreter, we can now use it to create an isolated virtual environment. This is very useful for software development, keeping each project completely isolated from the others:
pyenv virtualenv 3.7.2 MY_VIRTUALENV_NAME
To list all your virtual environments:
pyenv virtualenvs
To activate a virtual environment:
pyenv activate MY_VIRTUALENV_NAME
To deactivate the currently active virtual environment:
pyenv deactivate
From the README text file of python
Installing multiple versions
On Unix and Mac systems if you intend
to install multiple versions of Python
using the same installation prefix
(--prefix argument to the configure
script) you must take care that your
primary python executable is not
overwritten by the installation of a
different version. All files and
directories installed using "make
altinstall" contain the major and
minor version and can thus live
side-by-side. "make install" also
creates ${prefix}/bin/python which
refers to ${prefix}/bin/pythonX.Y. If
you intend to install multiple
versions using the same prefix you
must decide which version (if any) is
your "primary" version. Install that
version using "make install". Install
all other versions using "make
altinstall".
For example, if you want to install
Python 2.5, 2.6 and 3.0 with 2.6 being
the primary version, you would execute
"make install" in your 2.6 build
directory and "make altinstall" in the
others.
Virtualenv is an option but you could use the above mentioned option instead of venv which seems much simpler.
Personally, I use conda to create multiple environments (mostly, you create a new env, activate it and install the packages you want there -- using conda itself if possible or pip if it's not available in conda). See: https://conda.io/docs/installation.html.
After you have an environment created, you just need to add a new interpreter inside PyDev which points to the interpreter in the created environment. Ssee: http://www.pydev.org/manual_101_interpreter.html.
I have renamed python.exe to python37.exe to avoid conflict with other versions. It works for running python, but if I run pip37.exe (located in /Scripts) I get the following error:
Fatal error in launcher: Unable to create process using '"c:\python37-32\python.exe" "C:\Python37-32\Scripts\pip37.exe"
Is there a way to keep python.exe renamed to python37.exe, but keep all python tools working?
This sounds like A BAD IDEA.
There are tools designed to help you manage exactly this sort of thing. The best of which imho is pyenv: https://github.com/pyenv/pyenv
It's quite simple to install. It takes a bit of getting used to – wrapping your head around virtual environments – but it makes it all so much easier to work with ultimately.
E.g. On my system I have the following versions of python:
pyenv versions
system
2.7.10
* 3.5.6 (set by /Users/.pyenv/version)
3.5.6/envs/core4
3.6.4
3.6.4/envs/core5
core4
core5
The one with the asterisk is currently the global version, which will be the one used from any default shell. I can change that using pyenv global 3.6.4 for example. I can also create virtual environments. E.g. core4 and core5 are virtual environments I created for specific projects. Each of these will have their own different libraries installed by pip install and differing python versions. You can activate a virtualenv for a given shell session e.g. pyenv activate core5.
And if you're thinking "what on earth does this have to do with Windows", look here: https://duckduckgo.com/?q=Windows+Subsystem+for+Linux&atb=v93-1__&ia=web and here: http://timmyreilly.azurewebsites.net/python-pip-virtualenv-installation-on-windows/
On Windows Python installs PyLauncher. You don't need virtual environments or renaming tricks. py.exe is in the standard Windows path, has command line switches to pick the Python version to use, and enables using "shebangs" to specify what version of Python to run for scripts:
py script.py # Run the latest Python installed (or specified by PY_PYTHON environment variable).
py -2 script.py # Run the latest Python 2 version installed.
py -3 script.py # Run the latest Python 3 version installed.
py -2.7 script.py # Run the specific Python version.
py -2.7-32 script.py # Run the 32-bit specific Python version.
py -0 # List Python versions installed.
Scripts can use shebangs similar to Linux:
#!python2
#!python3
#!python2.7
#!python2.7-32
To run pip with a specific version:
py -2.7 -m pip install ...
If you need still would like a virtual environment with a specific Python version, you can specify the version (e.g. -3) and use:
py -3 -m venv <my_env_name> # to create an virtual environment
<my_env_name>/scripts/activate # to activate that environment
Activation adds the virtual environment to the path, so python (not py) will run in that environment. the Scripts directory in that environment will also be added to the path, so pip can be run directly as well to install packages in that environment.
I installed a specific version of python with pyenv. When typed pyenv version in terminal, i see 3.5.0 (set by /Users/lcherukuri/.python-version). But when I typed python3, I got python3 command not found. How to fix this? pip3 is Also not found
pyenv manages shim executables for commands like python3 and pip3. If pyenv's shims aren't available in your shell, it usually means one of two things:
pyenv isn't fully installed
or
pyenv shell features aren't active
As your pyenv command is working but the shims aren't, it most likely means the shell features aren't activated. As of writing, the correct way is to ensure the init command output is evaluated. On macOS, you can add the following to your ~/.bash_profile:
eval "$(pyenv init -)"
Older installation instructions might not include that step or just have you add pyenv's bin directory to the PATH, which is not enough. If you used pyenv-installer, this step is hinted at in a warning at the end of the installation process.
If you installed both python 2.x and python 3.x using pyenv, run the following to enable both versions to be found globally (python, python2 and python3 aliases).
Add the specific versions you are using:
pyenv global 3.8.3 2.7.18
pyenv is just a Python version manager. It may be able to see a Python 3.X installed even if python3 isn't installed in your $PATH.
You need to add python3 to your $PATH. You can see how to do that here.
By default, MacOS uses python3 to differentiate between the native pre-installed python (which is Python 2.7) and any post-installed Python 3.X distributions. The same goes for pip and pip3.
From the pyenv documentation on managing versions:
Locating the Python Installation
Once pyenv has determined which
version of Python your application has specified, it passes the
command along to the corresponding Python installation.
Each Python version is installed into its own directory under $(pyenv
root)/versions.
For example, you might have these versions installed:
$(pyenv root)/versions/2.7.8/
$(pyenv root)/versions/3.4.2/
$(pyenvroot)/versions/pypy-2.4.0/
As far as pyenv is concerned, version names
are simply the directories in $(pyenv root)/versions.
I had the python3 in path. I also executed pyenv global 2.x.x 3.x.x. But I still got the same error.
What eventually worked for me is executing this line in the project root (with whichever version replacing 3.X.X)
pyenv shell 3.X.X
Note: This sets the shell specific python version so it's not really a solution to the problem posted. Just a workaround to get python3 working.
I have Python 3.4 installed on my Linux computer.
sudo apt-get install python3.4
However, when I run python -V, it shows that Python 2.7.6 is being used.
How do I tell the system to use the updated version of Python?
The answer to this question for a windows computer is at How to update version of Python?, but I couldn't find an answer for Linux.
On Linux, installations of python3 installed by the package manager (e.g. apt) can be called as python3. You might need to specify the version - e.g. python3.5 if the package manager has installed more than one, or you've compiled your own installations from source.
you can specify the version in the shebang.
write #!/usr/bin/env python3.
when run via ./my_script.py it will run in python3.
otherwise run it via python3 my_script.py.
if you just want to start an interactive python shell start it with python3
you also can be more specific with the version. just replace python3 with python3.4 (if installed)
In ubuntu various python executable are places under /usr/bin/ and might look like
/usr/bin/python
/usr/bin/python3.2
/usr/bin/python3.4
etc. so when you execute a command python -v it looks for a file with that name in that location. so to choose your version specify it like python3.4 -v
you can replace the simlink /usr/lib/python with /usr/bin/python3.4 to make that "default"