I am running Ubuntu 20.04 with python 3.6, 3.7, and 3.8 installed.
I am trying to install some packages using pip on 3.6 and 3.7 versions using 'python3.7 -m pip install package' but, I am getting this error:
ModuleNotFoundError: No module named 'distutils.util
I already have python3-distutils and python3-distutils-extra installed but pip only works for python 3.8.
How can I make pip work for installing packages on python 3.6 and 3.7?
What about things like deadsnakes or pyenv, would those help? Otherwise you might need to add the apt repositories for older Ubuntu versions (say 18.04), but I am not sure what the side effects might be. This is a question that I would rather ask on Ask Ubuntu or Super User.
(from bounty description)
I want a definitive solution that does NOT involve someone telling me to install:
python-distutils
python3-distutils
python3-distutils-extra
I will be handing out FREE DOWNVOTES to anyone who mentions installing these.
Well, I have some bad news for you. There is no "official" way to get distutils on the Debian-provided python* packages without installing these packages, not unless you go outside of Debian's default package repositories.
This is a consequence of Debian's decision to break up various parts of Python's standard library into separate packages, on the basis of their policy to split things into runtime packages and development packages. They consider distutils to fall in the "development" part, and thus, don't distribute it as part of the standard python package.
Basically, your options are:
install a python*-distutils package.
This is what your OS maintainers recommend doing in your situation.
You've not explained why you'll "be handing out free downvotes" for anyone recommending this, but this is genuinely the "most correct" solution.
install Python, from somewhere that does not break up the standard library.
This would mean using a package source other than the official debian repositories.
The easiest source would be the deadsnakes ppa.
compile your own Python!
This isn't actually that difficult, and tools like pyenv make it easy to compile and manage multiple python versions.
hack around the problem!
You could... download the sources from CPython's repo, and place the Lib/distutils folder somewhere on the import path for pythonX.Y? This is a 100% hack though and I strongly recommend not doing this. If something bad happens, because you did this, I'm not responsible.
I've been looking for an answer to this question for almost as long as you, and finally found a solution.
Be warned, I may have screwed up my python3.8 installation as a side-effect, and it's possible the package will be uninstalled again and break things as soon as you run apt-update.
Also, it may be possible to reduce the number of commands, but here's what I ended up doing:
Run apt list to see a list of available packages:
$ apt list -a python3-distutils
Listing... Done
python3-distutils/focal-updates,focal-updates,focal-security,focal-security,now 3.8.5-1~20.04.1 all [installed]
python3-distutils/focal,focal 3.8.2-1ubuntu1 all
python3-distutils/bionic-updates,bionic-updates 3.6.9-1~18.04 all
python3-distutils/bionic,bionic 3.6.5-3 all
Then download the "bionic" deb package,
because it actually contains multiple versions of distutils,
for Pythons 3.6, 3.7 & 3.8:
$ apt download python3-distutils/bionic
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-distutils all 3.6.9-1~18.04 [144 kB]
Fetched 144 kB in 0s (661 kB/s)
Finally, install the deb package:
$ sudo dpkg --install python3-distutils_3.6.9-1~18.04_all.deb
dpkg: warning: downgrading python3-distutils from 3.8.5-1~20.04.1 to 3.6.9-1~18.04
(Reading database ... 193375 files and directories currently installed.)
Preparing to unpack python3-distutils_3.6.9-1~18.04_all.deb ...
Unpacking python3-distutils (3.6.9-1~18.04) over (3.8.5-1~20.04.1) ...
Setting up python3-distutils (3.6.9-1~18.04) ...
At this point, I was finally able to use pip with python3.6.
Note-1: The printouts of apt list -a above depend on whether you run an Ubuntu or a Debian distribution, and which apt-repos you have activated in /etc/apt/sources.list*.
Note-2: As seen in the case of python3-distutils/bionic, above, the name of the package does not always coincide with its contents.
To view them, you have to download it and inspect it with dpkg -C <pafkage-file>, or from remote with apt-file list python3-distutils.
Note-3: In case you cannot find the exact distutils version for a specific Python release, you may install an earlier version and symlink to it.
For example in Debian-unstable("sid") and "bullseye" as of January 2020, the python3-distutils package contained in their apt-repos has files just for Python3.9, and the previous apt-repos from "buster" contain files for Python3.7 only.
So if you wish to install distutils for Python3.8, you have to download the "buster" package and execute:
$ sudo dpkg --unpack python3-distutils_3.7.3-1_all.deb
$ sudo rm /usr/lib/python3.8/distutils/distutils/
$ sudo ln -w /usr/lib/python3.{7,8}/distutils/distutils/
The instructions here worked for me: https://www.linuxcapable.com/how-to-install-python-3-7-on-ubuntu-20-04-lts/
Namely this one sudo apt install python3.7-distutils
Was trying to install checkov and getting the ModuleNotFound error.
After the above, it installed just fine following these instructions
Here is what has worked for me on Ubuntu 22.04 for Python 3.6.
pip won't install as distutils is not available.
install python3.6 via apt
check the exact version with python3.6 --version
find a matching tag here: https://github.com/python/cpython.git, i.e. 3.6.9
adapt and call git clone --depth=1 -b v3.6.9 https://github.com/python/cpython.git
export PYTHONPATH=`pwd`/cpython/Lib
curl https://bootstrap.pypa.io/pip/3.6/get-pip.py | python3.6 - --user
python3.6 -m pip list
If the old version is not available, cd into the cpython source directory and use e.g. ./configure --enable-optimizations --prefix=$HOME/.local and make -j4 and make altinstall (set some PREFIX to install to when calling configure, so it won't clobber the system, but note that the install path has to be in your search paths)
Related
I want to install python 3.8.12 on my ubuntu .
I followed this link: https://tecadmin.net/install-python-3-8-ubuntu/ and installed python in my Download folder and I can't find it at /usr/bin/python3.8.12
Is it right that all the python versions need to be found at /usr/bin ?
Do I need to copy it manually to /usr/bin ?
Is there a way I can install python 3.8.12 with apt-get ?
That article is outdated, most likely written at a time where Python 3.8 was not yet available on the stable Ubuntu repositories.
You can install the latest Python available with just apt install python3 (3.9 at the moment).
If you really want 3.8, I suggest you take a look at the deadsnakes PPA. It contains an archive of all Python packages for Ubuntu, and also newer versions that haven't hit Ubuntu stable yet (like 3.10).
You can add this PPA and install Python 3.8 with just three commands:
add-apt-repository ppa:deadsnakes/ppa
apt update
apt install python3.8
Regarding your other two bullet points:
Yesn't. The thing is that, by default, /usr/bin is included in your PATH environment variable, which what enables you to just type python ... instead of providing the absolute path for the binary. It is also expected by some scripts, I suppose.
You could, and you could also symlink it. The reason why it wasn't put there already is because the article used altinstall, which prevents exactly that. It is way more preferable that you install stuff through your package manager, however.
I would like to install the newest version of docutils via pip, but it can't figure out how to upgrade the system version installed via apt.
$ sudo --set-home python2 -m pip install --upgrade docutils
Collecting docutils
Using cached https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl
Installing collected packages: docutils
Found existing installation: docutils 0.14
ERROR: Cannot uninstall 'docutils'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
$ apt-cache show python-docutils | head -n 3
Package: python-docutils
Architecture: all
Version: 0.14+dfsg-3
None of the solutions I've thought of or found on the web appeal:
Delete the apt version with rm -rf /usr/lib/python2.7/dist-packages/docutils*. This silences pip but means:
Installed files on the system no longer match what the Debian packaging system thinks
I might break dependencies of system software on docutils 0.14
Any updates to the Debian package will cause apt to reinstall
Other problems discussed in this answer
pip install --force-reinstall. (Same problems.)
pip install --ignore-install. (Same problems.)
Is there a way to get a default environment that works for me with the newest versions of stuff from pip but has no chance of breaking any system software? The same answer above suggests using one of virtualenv, venv, pyenv, pipenv. I tried pipenv and it doesn't want to install individual packages listed on the commandline using --system and I don't know whether creating a Pipfile will actually solve this problem.
I would rather not have to manually switch environments somehow to use the apt-installed packages versus the pip-installed packages. Is there a way to get only apt-installed software to use one environment and otherwise use the environment with the pip-installed stuff?
I would rather not have to manually switch environments somehow to use the apt-installed packages versus the pip-installed packages. Is there a way to get only apt-installed software to use one environment and otherwise use the environment with the pip-installed stuff?
Ideally, one should use either the system version or the pip version.
Per the Debian Python Policy,
As long as you don't install other versions of Python in your path, Debian's Python versions won't be affected by a new version.
If you install a different micro version of the version of Python you have got installed, you will need to be careful to install all the modules you use for that version of Python too.
So far adding the following to ~/.bashrc seems work well:
if [ ! -d ~/venv/python3 ]; then
python3 -m venv --system-site-packages ~/venv/python3
fi
if [ -d ~/venv/python3 ]; then
VIRTUAL_ENV_DISABLE_PROMPT=1 . ~/venv/python3/bin/activate
fi
Most of the system-installed scripts have one of the Pythons in /usr/bin hard-coded instead of using /usr/bin/env python so they are unaffected by this.
I was trying to install Django. Turns out that course's teacher said that we will be working with Python 3.6
I install Python 3.6. Now it's my default, it somewhat replaced the last version I had; which is Python 3.5.
Everything ok until that. But when I want to install Django doing
"pip3 install django", it tells me that the module is already satisfied and therefore installed.
I run "python3" command into my terminal. It runs Python 3.6. I try to import Django, and boom... "No module named 'django'".
Then I realized pip3 was actually installing my modules into Python 3.5 and not 3.6. So what I do is to install pip in Python 3.6.
I download get-pip.py and proceed to execute it with Python 3.6 typing in "python3.6 get-pip.py".
Here is when the damn "zipimport.ZipImportError: can't decompress data; zlib not available" goes in. I've tried a ton of things and no one of them fixed the %^$! problem. I'm really tired.
What I have already tried:
python3.6 -m pip install django, which output is "/usr/local/bin/python3.6: No module named pip"
apt install zlib, which output is "E: Unable to locate package zlib"
apt install zlib1g-dev, which says that it's already installed; the problem persists though.
I also came across this problem (while creating a simple installer for pyenv). Here's how I solved it for Mac and Linux:
Ubuntu 20.04, 18.04
You need the zlib development files, and probably zlib itself too:
sudo apt install -y zlib1g-dev zlibc
If you're missing zlib, it's likely that the next problem you'll run into is with openssl, so it's probably best to get that now as well:
sudo apt install -y libssl-dev
sudo apt install -y libssl1.1 || sudo apt install -y libssl1.0
macOS
I believe this comes with XCode CLI Tools (or at least I didn't have to custom install it Big Sur):
xcode-select --install
For me it worked in RHEL:
$ yum install zlib-devel
Suggested solutions (installing zlib1g-dev or zlib-devel) seem to resolve the issue in most cases. Here is one edge case I've encountered recently: whatever you are trying to run might use zlib via symlink which might be broken.
In my case I was trying to run a build of a 3rd-party software which already had python and all necessary libs being emebedded into it. It was packaged as a tar.gz archive. Unpacking the archive on a Windows machine and then copying the contents to another linux machine destroyed all the symlinks (if you do ls -l in a folder with symlinks you would see that all of them have size 0 and do not point to anything). Copying tar.gz to the linux machine directly and unpacking it there resolved the issue.
P.S. I know it's an edge case scenario but it took me and one more developer quite a while to figure it out so I think it's worth mentioning here, just in case someone gets as unlucky as I got.
Its solves my issue for centos 7.6 :-
yum install zlib-deve
I have a BeagleBone Black board. The OS is Debian GNU/Linux version 9. Can you use HomeBrew to install Python? Or is HomeBrew only for Mac OS?
Let's start with this last part of the question:
Or is HomeBrew only for Mac OS?
Stack Overflow's timestamp shows that this question was asked March 7, 2019. A short while before that, a reasonable answer to that would have been, “yes, Homebrew is intended for macOS, but Linuxbrew is a fork of Homebrew that does what the name implies: modifies Homebrew to work on Linux”.
However, on February 2, 2019 -- a little over a month before the question was asked -- Homebrew announced version 2.0.0. In the announcement on their blog, they opened with this sentence:
Today I’d like to announce Homebrew 2.0.0. The most significant
changes since 1.9.0 are official support for Linux and Windows 10
(with Windows Subsystem for Linux), brew cleanup running
automatically, no more options in Homebrew/homebrew-core, and removal
of support for OS X Mountain Lion (10.8) and older.
and further explained:
Homebrew on Linux was previously called “Linuxbrew”. You can install
it in your home directory, so it does not require sudo, and use it to
install software that your host distribution’s package manager does
not provide. Homebrew on Linux uses its own repository for formulae:
Homebrew/linuxbrew-core
https://brew.sh/2019/02/02/homebrew-2.0.0/
Then, on February 20, 2019, the README.md file this message was added to the top of the README.md file at the root of the Linuxbrew github repository:
Linuxbrew has been merged into Homebrew
Linuxbrew/brew has been merged into
Homebrew/brew! Existing
installations of Linuxbrew will be automatically migrated to Homebrew.
Linuxbrew/brew will no longer be updated. See the Homebrew
documentation of Linuxbrew and the
Homebrew 2.0.0 blog
post.
https://github.com/Linuxbrew/brew/pull/942/files
So in short, yes. As of a short while before this question was posted, Homebrew officially supports Linux. Debian GNU/Linux version 9, the specific distribution mentioned in the question, is included in that. But before you could use it to to install python, you would most likely need to install Homebrew itself, as it isn't installed on a Debian system by default.
Debian 9 even includes a 'linuxbrew-wrapper' package, which would have been built well before linuxbrew was merged into homebrew.
https://packages.debian.org/stretch/linuxbrew-wrapper
Will it work well now, given the merge of linuxbrew back into homebrew? Well, I have an up-to-date Debian 9 VM in front of me, so let's give it a try:
$ sudo apt install linuxbrew-wrapper
[...]
The following additional packages will be installed:
fonts-lato libruby2.3 rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit
ruby2.3 rubygems-integration zip
Suggested packages:
ri ruby-dev bundler
The following NEW packages will be installed:
fonts-lato libruby2.3 linuxbrew-wrapper rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert
ruby-test-unit ruby2.3 rubygems-integration zip
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 6,447 kB of archives.
After this operation, 27.8 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
[...]
$
So is it installed now? Let's ask dpkg:
$ dpkg --status linuxbrew-wrapper
Package: linuxbrew-wrapper
Status: install ok installed
[...]
Description: Homebrew package manager for Linux
Linuxbrew is a fork of Homebrew, the Mac OS package manager, for Linux.
[...]
This package provides Linuxbrew install scripts instead of linuxbrew itself.
Homepage: http://linuxbrew.sh/
So it installed the package, but the package is a 'wrapper' with install scripts, and the package still has the old linuxbrew description. What happens if we try to run brew?
$ type brew
brew is /usr/bin/brew
$ brew
========================================================================
For the convenience of using linuxbrew it is recommended to update the
following environment variables after installing linuxbrew.
PATH, MANPATH, INFOPATH
Set HOMEBREW_BOTTLE_DOMAIN environment variable if you prefer some
homebrew mirror as your upstream.
(example: /usr/share/doc/linuxbrew-wrapper/examples/profile)
========================================================================
==> This script will install:
/home/parallels/.linuxbrew/bin/brew
/home/parallels/.linuxbrew/Library/...
/home/parallels/.linuxbrew/share/doc/homebrew
/home/parallels/.linuxbrew/share/man/man1/brew.1
/home/parallels/.linuxbrew/share/zsh/site-functions/_brew
/home/parallels/.linuxbrew/etc/bash_completion.d/brew
/home/parallels/.cache/Homebrew/
Press RETURN to continue or any other key to abort
==> Downloading and installing Linuxbrew...
[...]
==> Migrating from Linuxbrew/brew to Homebrew/brew
Linuxbrew/brew has been merged into Homebrew/brew!
Linuxbrew/brew will no longer be updated.
Your git remote has been changed from
https://github.com/Linuxbrew/brew
to https://github.com/Homebrew/brew
See the blog post at https://brew.sh/2019/02/02/homebrew-2.0.0/
Updating Homebrew...
[...]
==> Tapping homebrew/core
Cloning into '/home/parallels/.linuxbrew/Library/Taps/homebrew/homebrew-core'...
[...]
Warning: /home/parallels/.linuxbrew/bin is not in your PATH.
==> Installation successful!
==> Next steps
Install the Linuxbrew dependencies:
Debian, Ubuntu, etc.:
`sudo apt-get install build-essential`
Fedora, Red Hat, CentOS, etc.:
`sudo yum groupinstall 'Development Tools'`
See http://linuxbrew.sh/#dependencies for more information.
Add to your ~/.bash_profile by running
echo 'export PATH="/home/parallels/.linuxbrew/bin:$PATH"' >>~/.bash_profile
echo 'export MANPATH="/home/parallels/.linuxbrew/share/man:$MANPATH"' >>~/.bash_profile
echo 'export INFOPATH="/home/parallels/.linuxbrew/share/info:$INFOPATH"' >>~/.bash_profile
We recommend you install GCC by running `brew install gcc`.
Run `brew help` to get started
Further documentation: https://git.io/brew-docs
==> Homebrew has enabled anonymous aggregate user behaviour analytics
Read the analytics documentation (and how to opt-out) here:
https://git.io/brew-analytics
Homebrew 2.0.6
Homebrew/linuxbrew-core (git revision 69e4; last commit 2019-03-31)
Cool, it automatically updated itself to transition from Linuxbrew to Homebrew. Notably, it still uses the directory name ~/.linuxbrew to keep its files. All we need to do to complete in install is:
Install any apt dependencies mentioned in the install output. Homebrew builds things from source, and your Linux distribution can provide most of the development toolchain it wants.
Updating some 'path' environment variables. Homebrew is adding a new location on your filesystem for installed software, separate from places like /bin and /usr/bin since your OS package manager manages the contents of those directories. Adding Homebrew's install location to these paths allows Hombrew-installed software to be found.
brew install gcc. While they've requested most of the development toolchain be supplied by your OS, they've apparently decided it's important to ask you to install a brew-managed compiler. This makes some sense, since the compiler used by your machine to build other homebrew formulae would presumably be in sync with the compiler they're testing with when they publish those formulae.
The original question also asked:
Can you use HomeBrew to install Python?
Let's try it:
$ brew install python
==> Installing dependencies for python: patchelf, pkg-config, gdbm, openssl, gpatch, ncurses, readline, sqlite, xz, bzip2, libffi and zlib
[...]
==> Installing python
==> Downloading https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
[...]
==> Caveats
Python has been installed as
/home/parallels/.linuxbrew/bin/python3
[...]
See: https://docs.brew.sh/Homebrew-and-Python
[...]
parallels#debian-gnu-linux-vm:~$
Looks like you can!
So, why would you want to bother with using Homebrew to install python when you can do it with apt? Well there's a whole discussion and debate around questions like that but here's the short answer...
The python installed by apt:
$ /usr/bin/python3 --version
Python 3.5.3
The python installed by homebrew:
$ ~/.linuxbrew/bin/python3 --version
Python 3.7.3
3.7.3 is the current python version at the time I'm writing this. Apt-managed packages update on a time scale of months or years. Homebrew-managed packages update on a time scale of days or hours.
Why do we see this dramatic difference between apt and homebrew? Think of homebrew as supplying a system layered on top of the one provided by apt. Lots of OS-level utilities are relying on the apt-managed python install, and the Debian team is being careful not to break them. But maybe you're working on something that needs new features that only the newest python releases have. That's the kind of problem Homebrew is aiming to solve.
On Linux, you can use a fork of Homebrew called Linuxbrew:
The Homebrew package manager may be used on Linux and Windows 10, using Windows Subsystem for Linux (WSL). Homebrew is referred to as Linuxbrew when running on Linux or Windows. It can be installed in your home directory, in which case it does not use sudo.
No, you can not use HomeBrew to install on Debian Linux because it is only for MacOs.
Also python is already installed in Debian Linux distributions, such as Ubuntu. However, it is possible to install a specific version, available from the Linux distribution, with the apt-get command. We can check the names of Python (pre-built) available packages with the apt-cache command. However this way of installing additional Python versions can be a source of potential problems (conflicts with the original installed version of the OS distribution). Therefore be warned on this.
$ apt-cache search python | less
$ sudo apt-get install pythonX.X
Rather, it is preferable to proceed with an installation of the wished version Python source code.
for more information and instructions on how to install from source, follow this link:
http://www.rasadacrea.com/en/web-training-courses/howto-install-python
Homebrew only works on MacOS.
Debian uses apt, however, another package manager, and from a quick google search it looks like your board has it.
If there is a version of python that works it should be installable through a command like
sudo apt-get update
sudo apt-get install python
for python3:
sudo apt-get install python3
However, it's entirely possible that python is already bundled with the OS, in which case you wouldn't have to install python. To check if/what version of python is installed, you can do:
python --version
or for python3:
python3 --version
My question is much like several others. Having manually compiled Python, sqlite3 is missing:
The main difference is that I'm using a Debian Linux system (unlike in this question: OS X 10.8.2 python 3 import sqlite error), and Python3 (unlike in this and a bunch of other questions: Cannot import SQLite with Python 2.6).
I was hoping for some guidance on which direction to troubleshoot in. According to some of the Linux-but-older-Python questions, an error like the one I'm getting could be caused by a missing resource during linking or something (_sqlite3.so). I have two such files on my system, both of them in older Python installations, ... but nothing related to Python3. Or is one of these good enough? Or they say to install the libsqlite3-dev package, then to re-compile Python. I did this, but I don't see how just having this package on my system will affect the compilation process. And indeed, it didn't. A second compile gave me a second Python without sqlite3.
I wish I could just do apt-get install python3, but Debian, in its stability, only has Python 3.2, where I need the latest version. Thoughts?
You need to install libsqlite3 (Debian based) or sqlite-devel (RedHat based) and the associated header files before compiling Python because Python needs to find them during the compile process.
Did you make sure to run:
./configure
make
make install
In this specific order? With no missing steps?
After apt-get install libsqlite3-dev
then
./configure --prefix=/opt/python3.7.4 --with-ssl --with-pydebug
make
make install
Note: You might need apt-get install libssl-dev aslo, openssl version must above 1.0.2 if you are compiling python3.7
For me, I'm using ubuntu 14.04 (trusty) I can't find a libssl-dev package to meet the requirement compiling python3.7 with ssl support. I modify my /etc/apt/sourcelist.d
deb http://cn.archive.ubuntu.com/ubuntu/ xenial main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ xenial main restricted
after install a newer libssl-dev, then change it back to the original one
deb http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted
If you only have limited user access (no root or sudo permission) you can install to a local, user accessible, environment like so:
tar -xvf sqlite-autoconf-3270200.tar.gz
cd sqlite-autoconf-3270200
./configure --prefix=$HOME/.local
make && make install
This will install on your ~/.local tree.
Add ~/.local/bin to your path if missing.