I've recently installed both Pyenv and Poetry and want to create a new Python 3.8 project. I've set both the global and local versions of python to 3.8.1 using the appropriate Pyenv commands (pyenv global 3.8.1 for example). When I run pyenv version in my terminal the output is 3.8.1. as expected.
Now, the problem is that when I create a new python project with Poetry (poetry new my-project), the generated pyproject.toml file creates a project with python 2.7:
[tool.poetry]
name = "my-project"
version = "0.1.0"
description = ""
authors = ["user <user#email.com>"]
[tool.poetry.dependencies]
python = "^2.7"
[tool.poetry.dev-dependencies]
pytest = "^4.6"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
It seems that Poetry defaults back to the system version of Python. How do I change this so that it uses the version installed with Pyenv?
Edit
I'm using MacOS, which comes bundled with Python 2.7. I think that might be causing some of the issues here. I've reinstalled Python 3.8 again with Pyenv, but when I hit Poetry install I get the following error:
The currently activated Python version 2.7.16 is not supported by the project (^3.8).
Trying to find and use a compatible version.
[NoCompatiblePythonVersionFound]
Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.
Should I create an environment explicitly for the project using Pyenv or should the project be able to access the correct Python version after running pyenv local 3.8.1.? When I do the latter, nothing changes and I still get the same errors.
pyproject.toml is used to define all the dependencies for your project, including the supported python version.
The line your complaining about is just saying that the versions of python supported by the project is python2.7 or greater, this is independent of what versions of python you've installed with pyenv.
python = "^2.7"
If you want to update the versions of python supported by the project you can edit the file directly and run poetry update.
If you want to use multiple versions of python you need to make sure poetry is using the correct dependencies for the version of python you are using. To change the specific version poetry is using you should use poetry env,
poetry env list show the versions of python poetry can use
poetry env use <python> switches poetry to use that version.
For instance on my machine poetry has 3 virtual environments installed and is using the one associated with python3.6:
↪ poetry env list
sipy-a9sqc5pb-py3.6 (Activated)
sipy-a9sqc5pb-py3.7
sipy-a9sqc5pb-py3.8
I'm not sure how these virtual environments with interact with the shivs used by pyenv but their docs have a section relating to it
Managing Virtual Environments
There is a pyenv plugin named pyenv-virtualenv which comes with various features to help pyenv users to manage virtual environments created by virtualenv or Anaconda. Because the activate script of those virtual environments are relying on mutating $PATH variable of user's interactive shell, it will intercept pyenv's shim style command execution hooks. We'd recommend to install pyenv-virtualenv as well if you have some plan to play with those virtual environments.
Alright, I figured the problem. A little embarrassingly, I had not run pyenv shell 3.8.1 before running any of the other commands. Everything works now. Thank you all for your efforts.
you can specify an explicit python executable for poetry using
poetry env use <path to python executable>
This worked for me.
On my machine I was able to fix the "currently activated Python version is not supported by the project" error by reinstalling Poetry:
curl -sSL https://install.python-poetry.org | python3 - --uninstall
curl -sSL https://install.python-poetry.org | python3 -
After that,poetry was able to find the correct version installed by pyenv.
In my case, I had to delete and recreate the virtualenv used by poetry. This is because I added the python version restrictions (e.g. python = ">=3.6.2 <3.7") after creating the virtualenv.
Steps
Delete the original one: run poetry env remove myApp-XkghI9p6-py3.6
Run any poetry step, to create it, or run poetry shell, and confirm poetry run python --version is the correct version.
Even though this issue has been resolved, I am writing this for somebody who comes across this problem again.
After all attempts my python -V always resulted in 2.7 and no discussions mentioned running pyenv shell (surprising to me!)
Adding pyenv to path
$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
In my case I had to add it to .bashrc and not bash_profile.
https://ggkbase-help.berkeley.edu/how-to/install-pyenv/
Worked!
In my case, the environment was messed up in some way that poetry failed to activate the virtualenv properly.
Try using a different shell: perhaps, sh, or zsh. If everything works in that shell, this proves that your environment is as messed up as mine was :)
Use this command in both shells:
$ env
and try to spot the difference
You can remove the python version from pyproject.toml file and then run Poetry install
What worked for me was to run python3.8 -m poetry install.
Related
I understand that the Poetry project is working with a different version of Python than I have installed.
How might I rectify this issue though? Preferably, Poetry and the project can use the latest Python version.
Terminal
cd /mnt/c/Users/me/Documents/GitHub/project
poetry run python -m project
python3 --version
Output
The currently activated Python version 3.8.10 is not supported by the project (3.8.8).
Trying to find and use a compatible version.
NoCompatiblePythonVersionFound
Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.
at ~/.poetry/lib/poetry/utils/env.py:768 in create_venv
764│ python_minor = ".".join(python_patch.split(".")[:2])
765│ break
766│
767│ if not executable:
→ 768│ raise NoCompatiblePythonVersionFound(
769│ self._poetry.package.python_versions
770│ )
771│
772│ if root_venv:
Python 3.8.10
I need to set up a virtual environment.
To do this in Conda:
conda create -n env_name python=3.8.8
I want to have multiple installs of Python: 2.1, 2.4, 2.7, 3.x
My IDE is Eclipse (Helios)/Pydev on MacOSX, which works great. I have a couple of Python codebases that are/will be running on different versions of Python. Also, I like Eclipse PyDev's crosslinking from source-code to documentation.
The standard recommendation seems to be: use virtualenv, and keep the installs totally separate from the builtin MacPython (2.6.1). Eclipse should never be pointing to the MacPython install. (Should PYTHONPATH even be set in such an environment?)
Before I get on with virtualenv, is there anything else I should know about this?
virtualenv doesn't impose any overhead, and I shouldn't be worried with occasional comments about breakage to nose, coverage etc?
I'm interested in hearing from Eclipse or Pydev users on MacOS.
Also if anyone has other tips on migrating a Python codebase from 2.1 -> 2.7.
A good way is to use macport to install the different version of pythons. It will have different versions of all packages for all versions of pythons that you want. They will be installed in /opt/local/.
So in Eclipse with PyDev for a particular project you can right click into the name of the project -> properties: There, in the left you choose PyDev - Interpreter/Grammar. And you click the link Click here to configure an interpreter not lister. You click on new on the top right of the new window. You give the name of the version of python you want to create the interpreter for, if you have already installed it with all required package through macport. And in the field Intepreter Executable you give the path: /opt/local/bin/pythonX.X .
After, in the previous option window: PyDev - Intepreter/Grammar, you can choose in the Interpreter menu the python version that you just installed.
For the shell, and the default path pointing to /usr/bin/python you must use: python_select (installed through macport) instead of playing with the env variables. Afterward you can use macport to update/install new packages, make sure everything is alway clean.
Having Home Brew already installed in your system, I recommend you pyenv. Most of the following information comes from this quick installation/use guide:
Installing pyenv + pyenv-virtualenv
Go to your terminal and install pyenv and the external libraries needed by Python:
brew update
brew install pyenv openssl readline sqlite3 xz zlib
Add pyenv init to your shell to enable shims and autocompletion running the following command:
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
Install the pyenv-virtualenv plugin so you can create virtual environments for each version of Python:
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
Reset your terminal to apply the changes exec "$SHELL" or just close your terminal and open a new one.
Gettings different Python versions
We can list all available versions with pyenv install --list:
pyenv install --list
Available versions:
2.7.15
3.0.1
..
3.7.2
For installing the Python version 3.7.2:
pyenv install 3.7.2
With pyenv versions you can see all the Python versions installed in your computer:
root#Host ~$ pyenv versions
* 3.7.2
Creating a virtual environment
In the previous step we have downloaded the Python 3.7.2 interpreter, we can now use it to create an isolated virtual environment. This is very useful for software development, keeping each project completely isolated from the others:
pyenv virtualenv 3.7.2 MY_VIRTUALENV_NAME
To list all your virtual environments:
pyenv virtualenvs
To activate a virtual environment:
pyenv activate MY_VIRTUALENV_NAME
To deactivate the currently active virtual environment:
pyenv deactivate
From the README text file of python
Installing multiple versions
On Unix and Mac systems if you intend
to install multiple versions of Python
using the same installation prefix
(--prefix argument to the configure
script) you must take care that your
primary python executable is not
overwritten by the installation of a
different version. All files and
directories installed using "make
altinstall" contain the major and
minor version and can thus live
side-by-side. "make install" also
creates ${prefix}/bin/python which
refers to ${prefix}/bin/pythonX.Y. If
you intend to install multiple
versions using the same prefix you
must decide which version (if any) is
your "primary" version. Install that
version using "make install". Install
all other versions using "make
altinstall".
For example, if you want to install
Python 2.5, 2.6 and 3.0 with 2.6 being
the primary version, you would execute
"make install" in your 2.6 build
directory and "make altinstall" in the
others.
Virtualenv is an option but you could use the above mentioned option instead of venv which seems much simpler.
Personally, I use conda to create multiple environments (mostly, you create a new env, activate it and install the packages you want there -- using conda itself if possible or pip if it's not available in conda). See: https://conda.io/docs/installation.html.
After you have an environment created, you just need to add a new interpreter inside PyDev which points to the interpreter in the created environment. Ssee: http://www.pydev.org/manual_101_interpreter.html.
I installed a specific version of python with pyenv. When typed pyenv version in terminal, i see 3.5.0 (set by /Users/lcherukuri/.python-version). But when I typed python3, I got python3 command not found. How to fix this? pip3 is Also not found
pyenv manages shim executables for commands like python3 and pip3. If pyenv's shims aren't available in your shell, it usually means one of two things:
pyenv isn't fully installed
or
pyenv shell features aren't active
As your pyenv command is working but the shims aren't, it most likely means the shell features aren't activated. As of writing, the correct way is to ensure the init command output is evaluated. On macOS, you can add the following to your ~/.bash_profile:
eval "$(pyenv init -)"
Older installation instructions might not include that step or just have you add pyenv's bin directory to the PATH, which is not enough. If you used pyenv-installer, this step is hinted at in a warning at the end of the installation process.
If you installed both python 2.x and python 3.x using pyenv, run the following to enable both versions to be found globally (python, python2 and python3 aliases).
Add the specific versions you are using:
pyenv global 3.8.3 2.7.18
pyenv is just a Python version manager. It may be able to see a Python 3.X installed even if python3 isn't installed in your $PATH.
You need to add python3 to your $PATH. You can see how to do that here.
By default, MacOS uses python3 to differentiate between the native pre-installed python (which is Python 2.7) and any post-installed Python 3.X distributions. The same goes for pip and pip3.
From the pyenv documentation on managing versions:
Locating the Python Installation
Once pyenv has determined which
version of Python your application has specified, it passes the
command along to the corresponding Python installation.
Each Python version is installed into its own directory under $(pyenv
root)/versions.
For example, you might have these versions installed:
$(pyenv root)/versions/2.7.8/
$(pyenv root)/versions/3.4.2/
$(pyenvroot)/versions/pypy-2.4.0/
As far as pyenv is concerned, version names
are simply the directories in $(pyenv root)/versions.
I had the python3 in path. I also executed pyenv global 2.x.x 3.x.x. But I still got the same error.
What eventually worked for me is executing this line in the project root (with whichever version replacing 3.X.X)
pyenv shell 3.X.X
Note: This sets the shell specific python version so it's not really a solution to the problem posted. Just a workaround to get python3 working.
Presently we have a big-data cluster built using Cloudera-Virtual machines. By default the Python version on the VM is 2.7.
For one of my programs I need Python 3.6. My team is very skeptical about 2 installations and afraid of breaking existing cluster/VM. I was planning to follow this article and install 2 versions https://www.digitalocean.com/community/tutorials/how-to-set-up-python-2-7-6-and-3-3-3-on-centos-6-4
Is there a way "I can package Python 3.6" version in my project, and set the Python home path to my project folder, so that there is no installation that needs to be done on the existing Virtual machine?
Since we have to download python and build source for the Unix version, I want to skip this part on VM, and instead ship the folder which has Python 3.6
It seems that miniconda is what you need.
using it you can manage multiple python environments with different versions of python.
to install miniconda3 just run:
# this will download & install miniconda3 on your home dir
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3
then, create new python3.6 env:
conda create -y -n myproject 'python>3.6'
now, enter the new python3.6 env
source activate myproject
python3
miniconda can also install python packages, including pip packages and compiled packages. you can also copy envs from one machine to another. I encourage you to take a deeper look into it.
ShmulikA's suggestion is pretty good.
Here I'd like to add another one - I use Python 2.7.x, but for few prototypes, I had to go with Python 3.x. For this I used the pyenv utility.
Once installed, all you have to do is:
pyenv install 3.x.x
Can list all the available Python variants:
pyenv versions
To use the specific version, while at the project root, execute the following:
pyenv local 3.x.x
It'll create a file .python-version at the project root, having the version as it's content:
[nahmed#localhost ~]$ cat some-project/.python-version
3.5.2
Example:
[nahmed#localhost ~]$ pyenv versions
* system (set by /home/nahmed/.pyenv/version)
3.5.2
3.5.2/envs/venv_scrapy
venv_scrapy
[nahmed#localhost ~]$ pyenv local 3.5.2
[nahmed#localhost ~]$ pyenv versions
system
* 3.5.2 (set by /home/nahmed/.python-version)
3.5.2/envs/venv_scrapy
venv_scrapy
I found it very simple to use.
Here's a post regarding the installation and basic usage (blog post by me).
For the part:
Since we have to download python and build source for the Unix
version, I want to skip this part on VM, and instead ship the folder
which has Python 3.6
You might look into ways to embed Python interpreter with your Python application:
And for both Windows and Linux, there's bbfreeze or also pyinstaller
from - SOAnswer.
I am researching possibility to upgrade to Python 3.6 in our project.
Right now we are using Python 3.5.2 from ppa:fkrull/deadsnakes on Ubuntu 14.04. The PPA doesn't have Python 3.6 yet and it's not clear when it will be available.
I don't want to install yet another PPA.
And I am trying to find a more general approach.
I found people suggesting to use pyenv which compiles Python from source, which sounds interesting, because I can upgrade Python any time without waiting until repo maintainer adds it. Also I can easily install other Python flavors like PyPy.
I am not ready to use pyenv as virtual environment yes, so I am wondering if it's possible to use it to compile and install Python globally so that I can just use it.
The documentation is a little confusing because there is no python-build binary added in PATH after installation.
python-build is a pyenv plugin (installed by default). Documentation and more info is here: https://github.com/pyenv/pyenv/tree/master/plugins/python-build.
How to install system-wide Python for all users: 1) Login as root and 2) install required Python version to /usr/local/python-X.Y.Z.
sudo ~/.pyenv/plugins/python-build/bin/python-build 3.6.1 /usr/local/python-3.6.1/
Now you can use this Python version as a normal user, for example you can create virtualenv for your project:
/usr/local/python-3.6.1/bin/python -m venv /var/www/my-app/.env/
https://github.com/yyuu/pyenv/wiki/Common-build-problems#installing-a-system-wide-python
Installing a system-wide Python
If you want to install a Python interpreter that's available to all
users and system scripts (no pyenv), use /usr/local/ as the install
path. For example:
sudo python-build 3.3.2 /usr/local/
I've contributed a package for python3.6 in deadsnakes for trusty / xenial :)
https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes/+packages?field.name_filter=python3.6&field.status_filter=published&field.series_filter=
By combining the hints from the other answers and reading through the documentation, I found a nice way to do exactly what you want that should work well in a CI system or in a Docker container or on a developer machine if they haven't already installed python3.x via Apt or Yum or Homebrew.
Assuming you have all the dependencies required to build your desired version of Python 3.x (anything above 3.4 requires some extra packages the pyenv-installer doesn't always warn you about), you can run the commands below to get a new system wide Python that should be executable by all users, which makes it easy to pass to virtualenv creations with python3.6 -m venv yourvenv.
curl https://pyenv.run | bash # or
wget -O - https://pyenv.run | bash
export PATH="$HOME/.pyenv/bin:$PATH"
$(pyenv which python-build) 3.6.10 /usr/local/
which python3.6
python3.6 --version
# If you get an error running the above commands, it probably means
# /usr/local/bin isn't in your PATH yet
# on Debian/Ubuntu and maybe others the /etc/environment or
# /etc/login.defs file puts it in the path when a user logs in
echo $PATH
export PATH="/usr/local/bin:$PATH"
python3.6 --version