Missing sqlite3 after Python3 compile - python

My question is much like several others. Having manually compiled Python, sqlite3 is missing:
The main difference is that I'm using a Debian Linux system (unlike in this question: OS X 10.8.2 python 3 import sqlite error), and Python3 (unlike in this and a bunch of other questions: Cannot import SQLite with Python 2.6).
I was hoping for some guidance on which direction to troubleshoot in. According to some of the Linux-but-older-Python questions, an error like the one I'm getting could be caused by a missing resource during linking or something (_sqlite3.so). I have two such files on my system, both of them in older Python installations, ... but nothing related to Python3. Or is one of these good enough? Or they say to install the libsqlite3-dev package, then to re-compile Python. I did this, but I don't see how just having this package on my system will affect the compilation process. And indeed, it didn't. A second compile gave me a second Python without sqlite3.
I wish I could just do apt-get install python3, but Debian, in its stability, only has Python 3.2, where I need the latest version. Thoughts?

You need to install libsqlite3 (Debian based) or sqlite-devel (RedHat based) and the associated header files before compiling Python because Python needs to find them during the compile process.
Did you make sure to run:
./configure
make
make install
In this specific order? With no missing steps?

After apt-get install libsqlite3-dev
then
./configure --prefix=/opt/python3.7.4 --with-ssl --with-pydebug
make
make install
Note: You might need apt-get install libssl-dev aslo, openssl version must above 1.0.2 if you are compiling python3.7
For me, I'm using ubuntu 14.04 (trusty) I can't find a libssl-dev package to meet the requirement compiling python3.7 with ssl support. I modify my /etc/apt/sourcelist.d
deb http://cn.archive.ubuntu.com/ubuntu/ xenial main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ xenial main restricted
after install a newer libssl-dev, then change it back to the original one
deb http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted

If you only have limited user access (no root or sudo permission) you can install to a local, user accessible, environment like so:
tar -xvf sqlite-autoconf-3270200.tar.gz
cd sqlite-autoconf-3270200
./configure --prefix=$HOME/.local
make && make install
This will install on your ~/.local tree.
Add ~/.local/bin to your path if missing.

Related

How to install python-distutils for old python versions

I am running Ubuntu 20.04 with python 3.6, 3.7, and 3.8 installed.
I am trying to install some packages using pip on 3.6 and 3.7 versions using 'python3.7 -m pip install package' but, I am getting this error:
ModuleNotFoundError: No module named 'distutils.util
I already have python3-distutils and python3-distutils-extra installed but pip only works for python 3.8.
How can I make pip work for installing packages on python 3.6 and 3.7?
What about things like deadsnakes or pyenv, would those help? Otherwise you might need to add the apt repositories for older Ubuntu versions (say 18.04), but I am not sure what the side effects might be. This is a question that I would rather ask on Ask Ubuntu or Super User.
(from bounty description)
I want a definitive solution that does NOT involve someone telling me to install:
python-distutils
python3-distutils
python3-distutils-extra
I will be handing out FREE DOWNVOTES to anyone who mentions installing these.
Well, I have some bad news for you. There is no "official" way to get distutils on the Debian-provided python* packages without installing these packages, not unless you go outside of Debian's default package repositories.
This is a consequence of Debian's decision to break up various parts of Python's standard library into separate packages, on the basis of their policy to split things into runtime packages and development packages. They consider distutils to fall in the "development" part, and thus, don't distribute it as part of the standard python package.
Basically, your options are:
install a python*-distutils package.
This is what your OS maintainers recommend doing in your situation.
You've not explained why you'll "be handing out free downvotes" for anyone recommending this, but this is genuinely the "most correct" solution.
install Python, from somewhere that does not break up the standard library.
This would mean using a package source other than the official debian repositories.
The easiest source would be the deadsnakes ppa.
compile your own Python!
This isn't actually that difficult, and tools like pyenv make it easy to compile and manage multiple python versions.
hack around the problem!
You could... download the sources from CPython's repo, and place the Lib/distutils folder somewhere on the import path for pythonX.Y? This is a 100% hack though and I strongly recommend not doing this. If something bad happens, because you did this, I'm not responsible.
I've been looking for an answer to this question for almost as long as you, and finally found a solution.
Be warned, I may have screwed up my python3.8 installation as a side-effect, and it's possible the package will be uninstalled again and break things as soon as you run apt-update.
Also, it may be possible to reduce the number of commands, but here's what I ended up doing:
Run apt list to see a list of available packages:
$ apt list -a python3-distutils
Listing... Done
python3-distutils/focal-updates,focal-updates,focal-security,focal-security,now 3.8.5-1~20.04.1 all [installed]
python3-distutils/focal,focal 3.8.2-1ubuntu1 all
python3-distutils/bionic-updates,bionic-updates 3.6.9-1~18.04 all
python3-distutils/bionic,bionic 3.6.5-3 all
Then download the "bionic" deb package,
because it actually contains multiple versions of distutils,
for Pythons 3.6, 3.7 & 3.8:
$ apt download python3-distutils/bionic
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-distutils all 3.6.9-1~18.04 [144 kB]
Fetched 144 kB in 0s (661 kB/s)
Finally, install the deb package:
$ sudo dpkg --install python3-distutils_3.6.9-1~18.04_all.deb
dpkg: warning: downgrading python3-distutils from 3.8.5-1~20.04.1 to 3.6.9-1~18.04
(Reading database ... 193375 files and directories currently installed.)
Preparing to unpack python3-distutils_3.6.9-1~18.04_all.deb ...
Unpacking python3-distutils (3.6.9-1~18.04) over (3.8.5-1~20.04.1) ...
Setting up python3-distutils (3.6.9-1~18.04) ...
At this point, I was finally able to use pip with python3.6.
Note-1: The printouts of apt list -a above depend on whether you run an Ubuntu or a Debian distribution, and which apt-repos you have activated in /etc/apt/sources.list*.
Note-2: As seen in the case of python3-distutils/bionic, above, the name of the package does not always coincide with its contents.
To view them, you have to download it and inspect it with dpkg -C <pafkage-file>, or from remote with apt-file list python3-distutils.
Note-3: In case you cannot find the exact distutils version for a specific Python release, you may install an earlier version and symlink to it.
For example in Debian-unstable("sid") and "bullseye" as of January 2020, the python3-distutils package contained in their apt-repos has files just for Python3.9, and the previous apt-repos from "buster" contain files for Python3.7 only.
So if you wish to install distutils for Python3.8, you have to download the "buster" package and execute:
$ sudo dpkg --unpack python3-distutils_3.7.3-1_all.deb
$ sudo rm /usr/lib/python3.8/distutils/distutils/
$ sudo ln -w /usr/lib/python3.{7,8}/distutils/distutils/
The instructions here worked for me: https://www.linuxcapable.com/how-to-install-python-3-7-on-ubuntu-20-04-lts/
Namely this one sudo apt install python3.7-distutils
Was trying to install checkov and getting the ModuleNotFound error.
After the above, it installed just fine following these instructions
Here is what has worked for me on Ubuntu 22.04 for Python 3.6.
pip won't install as distutils is not available.
install python3.6 via apt
check the exact version with python3.6 --version
find a matching tag here: https://github.com/python/cpython.git, i.e. 3.6.9
adapt and call git clone --depth=1 -b v3.6.9 https://github.com/python/cpython.git
export PYTHONPATH=`pwd`/cpython/Lib
curl https://bootstrap.pypa.io/pip/3.6/get-pip.py | python3.6 - --user
python3.6 -m pip list
If the old version is not available, cd into the cpython source directory and use e.g. ./configure --enable-optimizations --prefix=$HOME/.local and make -j4 and make altinstall (set some PREFIX to install to when calling configure, so it won't clobber the system, but note that the install path has to be in your search paths)

Can't fix "zipimport.ZipImportError: can't decompress data; zlib not available" when I type in "python3.6 get-pip.py"

I was trying to install Django. Turns out that course's teacher said that we will be working with Python 3.6
I install Python 3.6. Now it's my default, it somewhat replaced the last version I had; which is Python 3.5.
Everything ok until that. But when I want to install Django doing
"pip3 install django", it tells me that the module is already satisfied and therefore installed.
I run "python3" command into my terminal. It runs Python 3.6. I try to import Django, and boom... "No module named 'django'".
Then I realized pip3 was actually installing my modules into Python 3.5 and not 3.6. So what I do is to install pip in Python 3.6.
I download get-pip.py and proceed to execute it with Python 3.6 typing in "python3.6 get-pip.py".
Here is when the damn "zipimport.ZipImportError: can't decompress data; zlib not available" goes in. I've tried a ton of things and no one of them fixed the %^$! problem. I'm really tired.
What I have already tried:
python3.6 -m pip install django, which output is "/usr/local/bin/python3.6: No module named pip"
apt install zlib, which output is "E: Unable to locate package zlib"
apt install zlib1g-dev, which says that it's already installed; the problem persists though.
I also came across this problem (while creating a simple installer for pyenv). Here's how I solved it for Mac and Linux:
Ubuntu 20.04, 18.04
You need the zlib development files, and probably zlib itself too:
sudo apt install -y zlib1g-dev zlibc
If you're missing zlib, it's likely that the next problem you'll run into is with openssl, so it's probably best to get that now as well:
sudo apt install -y libssl-dev
sudo apt install -y libssl1.1 || sudo apt install -y libssl1.0
macOS
I believe this comes with XCode CLI Tools (or at least I didn't have to custom install it Big Sur):
xcode-select --install
For me it worked in RHEL:
$ yum install zlib-devel
Suggested solutions (installing zlib1g-dev or zlib-devel) seem to resolve the issue in most cases. Here is one edge case I've encountered recently: whatever you are trying to run might use zlib via symlink which might be broken.
In my case I was trying to run a build of a 3rd-party software which already had python and all necessary libs being emebedded into it. It was packaged as a tar.gz archive. Unpacking the archive on a Windows machine and then copying the contents to another linux machine destroyed all the symlinks (if you do ls -l in a folder with symlinks you would see that all of them have size 0 and do not point to anything). Copying tar.gz to the linux machine directly and unpacking it there resolved the issue.
P.S. I know it's an edge case scenario but it took me and one more developer quite a while to figure it out so I think it's worth mentioning here, just in case someone gets as unlucky as I got.
Its solves my issue for centos 7.6 :-
yum install zlib-deve

ignoring ensurepip failure pip requires ssl/tls error in Ubuntu 18.04

Getting ignoring ensurepip failure pip requires ssl/tls error when trying to install python and pip in Ubuntu 18.04
Trying to run sudo make install gets the above error.
# Download Python
curl -O https://www.python.org/ftp/python/3.4.2/Python-3.4.2.tgz
tar -xzvvf Python-3.4.2.tgz
cd Python-3.4.2
export CFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib -I/usr/local/opt/zlib/include -L/usr/local/opt/zlib/lib"
# Required Dependencies
sudo apt-get install libssl-dev openssl
sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libpq-dev zlib1g-dev
# Prepare to install Python
./configure
make -j4
sudo make install
This is a python compilation issue that has been fixed as of Python 3.4.5. Best practice would be to upgrade to 3.4.5 and follow the compilation steps again.
For those coming here for Python <3.5.3 with the same issue, this same issue has been fixed as of 3.5.3 (see here).
I am not sure if other Python versions are affected, but have seen in the Patch notes that in later versions also a fix was made for Windows and MacOSX. Please check the change log for the Python major version you are using to see which fix applies to you. the change logs can be found here: (Python 3.4, Python 3.5)
This is due to Debian 9 using OpenSSL 1.1.0., However, OpenSSL 1.1.0 support in the SSL module was only added to Python 2.7.13, 3.5.3 and 3.6+
Thus, it makes the python under those versions cannot be correctly linked to ssl library. Issues https://github.com/pyenv/pyenv/issues/945
So you have to manually add those libraries when you are compiling them.
My suggestion
Since your system is already compiled with the default python, unintentionally compiled different version of python into global executable may cause many hidden problems, especially some commands used behind on system-default python.
So why not you use pyenv, to control those version of pythons? pyenv is like a python-version control programs, it uses shims, through a process called rehashing, pyenv maintains shims in that directory to match every Python command across every installed version of python, pip, and so on.
For more documents, please reading: https://github.com/pyenv/pyenv.
To install pyenv, please following the provided reference.
Solution
After many hours struggling, I finally found a solution to perfectly solve those version of pythons confliction problems, copy and paste the following script to a new file, and make it executable, then you can compile and install those pythons. While if you want to install them in other way, say, without using pyenv, please change the last second line commands to fit your needs.
#!/bin/bash -e
# Note: it is a script to solve Ubuntu 18.04 LTS
# different version of pythons compiling
# header confliction problems
#
# The idea is got from #JustAnotherArivist
# From URL: https://github.com/pyenv/pyenv/issues/945
#
# The script used in here is with slightly modifications
# to fit many different SSL header versions
# First under your home directory make OpenSSL library
# and extract useful package
mkdir ~/libssl1.0-dev
cd ~/libssl1.0-dev
apt-get download libssl1.0-dev
ar x libssl1.0-dev* data.tar.xz
tar -xf data.tar.xz --strip-components=2
# Second, specifically get your current system's SSL headers
# and make symbolic-links
libcrypto=$(ls /usr/lib/x86_64-linux-gnu/ | grep libcrypto.so......)
libssl=$(ls /usr/lib/x86_64-linux-gnu/ | grep libssl.so......)
ln -s /usr/lib/x86_64-linux-gnu/${libcrypto} ~/libssl1.0-dev/lib/x86_64-linux-gnu
ln -s /usr/lib/x86_64-linux-gnu/${libssl} ~/libssl1.0-dev/lib/x86_64-linux-gnu
# Set your CFLAGS LDFLAGS compile options
# And use pyenv install the python version <3.4.5 or <3.5.3
# Note: it is a one line command
# Please change the version of python that you want to compile
CFLAGS="-I${HOME}/libssl1.0-dev/include -I${HOME}/libssl1.0-dev/include/x86_64-linux-gnu" \
LDFLAGS="-L${HOME}/libssl1.0-dev/lib/x86_64-linux-gnu" \
pyenv install 3.4.2
# Remove tempor libssl1.0-dev direcotory
rm -rf ~/libssl1.0-dev

python update to 2.7.13 on centos

I'm new for linux and python ,yesterday I update my python from 2.6.6 to 2.7.13,but when i finished these command "./configure 、make、make install " ,then run "python“ ,I got "command not found ". thanks for help
I completely agree with the answer provided by #holdenweb. Just to add some more details considering for a tyro:
If you need access to a newer version of Python you must compile it yourself and install it side-by-side with the system version.
Here are the steps necessary to install Python 2.7. Execute all the commands below as root. Either log in as root temporarily or use sudo.
Install development tools
In order to compile Python you must first install the development tools:
The first one is :
yum groupinstall "Development tools"
You also need a few extra libraries installed before compiling Python or else you will run into problems later when trying to install various packages:
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel
Download, compile and install Python
The --no-check-certificate is optional
cd /opt
wget --no-check-certificate https://www.python.org/ftp/python/2.7.6/Python-
2.7.6.tar.xz
tar xf Python-2.7.6.tar.xz
cd Python-2.7.6
./configure --prefix=/usr/local
make && make altinstall
It is important to use altinstall instead of install, otherwise you will end up with two different versions of Python in the filesystem both named python.
After running the commands above your newly installed Python 2.7 interpreter will be available as /usr/local/bin/python2.7 and the system version of Python 2.6.6 will be available as /usr/bin/python and /usr/bin/python2.6.6.
Now Python is an essential component of many operating systems, the safest rule is leave the system's Python to the system, and don't modify it yourself. That way you can be sure you aren't going to interfere with any OS code that depends on a specific version of Python.
If you want some other version of Python to be your own default, install it somewhere like /usr/local/bin (the default installation for most Linux systems will put it there by default) and then put that directory at the front of your shell's PATH to ensure that the python command gets your version instead of the system's (which will remain as /usr/bin/python).
I would recommend that you re-link /usr/bin/python to point to /usr/bin/python2.6 and then the PATH adjustment mentioned above (but in your case adding /usr/local/python2.7.13/bin, which is where you seem to have installed your updated Python) should be all you need to do.

Uninstall python built from source?

I've installed python 2.6 from source, and somehow later mistakenly installed another python 2.6 from a package manager too.
I can't find a way to uninstall a python that was built from source, is this possible/easy?
Running ubuntu 10.04.
You can use checkinstall to remove Python. The idea is:
Install checkinstall
Use
checkinstall to make a deb of your
Python installation
Use dpkg -r to
remove the deb.
See this post for more details.
PS. Note that Ubuntu must always have at least one installation of Python installed, or else major pieces of your OS stop working. Above, I'm assuming it's safe to remove the Python built from source, without removing the Python that was installed by the package manager.
PPS. If you accidentally erase all Python installations from your Ubuntu machine, all is not lost. Instructions on how to recover from this situation can be found here.
I did the following and reinstall using 'make install' and it worked.
whereis python3.6
rm -rf /usr/local/lib/python3.6
rm -rf /usr/local/bin/python3.6*
make install
Have you looked into
make uninstall
I believe this should work for you, assuming you have the python 2.6 source and the make file has uninstall available (it should).
http://www.linuxquestions.org/questions/linux-newbie-8/source-uninstall-with-make-uninstall-howto-230225/
In the future it may be prudent to use sudo checkinstall.
Below command removed all the things it installed for me.
make -n install
Do you still have the source directory where you compiled Python before? If so, you can CD into that directory and run sudo make uninstall.
If you don't have it still, you could re-create it by going through the build steps again--download, extract, configure, and make--but end with sudo make uninstall instead of sudo make install, of course.

Categories