We Keep Coding

Building Python and OpenSSL from source, but ssl module fails

I'm trying to build Python and OpenSSL from source in a container. Both seem to build correctly, but Python does not successfully create the _ssl module.
I've found a few guides online that say to un-comment and lines from Python-3.X.X/Modules/Setup and add the --openssldir=/usr/local/ssl flag to the ./configure step for OpenSSL. I do these in my dockerfile. This has had the effect that, during the ./configure output for Python, I see the following line.
checking for X509_VERIFY_PARAM_set1_host in libssl... yes
Yet I receive the following errors:
[91m*** WARNING: renaming "_ssl" since importing it failed: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_ssl.cpython-38-x86_64-linux-gnu.so)
[0m[91m*** WARNING: renaming "_hashlib" since importing it failed: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_hashlib.cpython-38-x86_64-linux-gnu.so)
[0m
Python build finished successfully!
...
Following modules built successfully but were removed because they could not be imported:
_hashlib _ssl
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
If ./configure finds X509..., why am I still getting the hashlib and ssl errors?
The full Dockerfile, FWIW:
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && apt-get install -y apt-utils gcc make zlib1g-dev \
build-essential libffi-dev checkinstall libsqlite3-dev
RUN wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz && \
tar xzf openssl-1.1.1d.tar.gz && \
cd openssl-1.1.1d && \
./config -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)' --prefix=/usr/local/ssl --openssldir=/usr/local/ssl && \
make && \
make test && \
make install
RUN wget -q https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz && \
tar -xzf Python-3.8.2.tgz && \
cd Python-3.8.2 && \
./configure && \
make && \
make install
USER jenkins

Following modules built successfully but were removed because they could not be imported:
_hashlib _ssl
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
It seems like installation issue when building openssl from source. For build failure on _ssl module, try extra options like --with-openssl, CFLAGS and LDFLAGS when configuring Python using the script ./configure, e.g.
./configure --with-openssl=/PATH/TO/YOUR/OPENSSL_INSTALL_FOLDER/ \
--enable-optimizations \
--with-ssl-default-suites=openssl \
CFLAGS="-I/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/include" \
LDFLAGS="-L/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/"
Also try this command openssl version, if it reports error like this :
/usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found
that means there is linking problem on your openssl library, I'm not sure if you're on Linux or other system, but for Linux system, you can manually modify the links to openssl library to fix the problem as described in my answer at here.
Reference
Building Python 3.7.1 - SSL module failed
Python 3.7.0 wont compile with SSL Support 1.1.0

I reckon that Jenkins Image comes with some openssl version installed that is not 1.1.1, hence you find X509... in libssl but cant build.
Regarding said config option, you can spin up the container with bash as CMD, copy the config from within the container to the machine where the Image lies, edit ist and bake your version of the config into the Image.

I had the same problem and after 3+ hours of searching THIS is what actually worked:
Error: "ssl module is not available" when installing package with pip3
———————————————————-
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/pip/
The fix:
———————————————————-
sudo yum install openssl11 openssl11-devel
mkdir /usr/local/openssl11
cd /usr/local/openssl11
ln -s /usr/lib64/openssl11 lib
ln -s /usr/include/openssl11 include
The above code creates an alternate path for the latest openssl11 and symlinks it in a place that matches the folder structure that python expects.
Then proceed with the original steps in the guide
(https://linuxstans.com/how-to-install-python-centos/)
and ./configure like this within your python install folder:
./configure -–enable-optimizations -–with-openssl=/usr/local/openssl11
make altinstall
the altinstall is important so that you don't overwrite the system default python. You'll have to invoke it as python3.10 from command line thereafter.
I tested this on CentOS 7 and Python-3.10.8.
ref:
https://linuxstans.com/how-to-install-python-centos/?unapproved=4338&moderation-hash=9903c8ebd6634e5bbbad96716e283f8b#comment-4338

pip cannot confirm SSL certificate: SSL module is not available

I am trying to install RPi.GPIO for Python 3.6 on my Raspberry Pi Zero W, but somehow it won't connect to the python.org website. I have pip installed for 2.7, 3.0, and 3.6 installed, so when I go and do something like:
sudo pip3.6 install RPi.GPIO
I get this error:
pi#raspberrypi:~ $ sudo pip3.6 install RPi.GPIO
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting RPi.GPIO
Could not fetch URL https://pypi.python.org/simple/rpi-gpio/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
Could not find a version that satisfies the requirement RPi.GPIO (from versions: )
No matching distribution found for RPi.GPIO
I did try sudo apt-get install openssl and so on, but it still doesn't work. I can access the website perfectly on my desktop and phone, but my Raspberry Pi won't at all. Any idea what I can do?

Before installing python 3.6 you need to install the libraries needed.
sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
Do you have these installed on the PI?
More details here

I encounter the same problem，finally I solved it by following
Check whether the openssl-devel has been installed.
My OS is centos6 and the installing command is as following
yum install openssl-devel -y
Edit python source configure file
vi /{yourpythonsource}/Modules/Setup
. Uncomment some of content as following
# Socket module helper for socket(2)
_socket socketmodule.c timemodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
#SSL=/usr/local/ssl
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto
recompile python.
cd /{yourpythonsource}
make && make install
Now pip3 should be ok.

Python 3 No module named '_ssl'

The Problem
While I run you python3 application, it shows
File "/usr/local/lib/python3.6/ssl.py", line 101, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
What I've tried
install the dependencies
yum install openssl-devel
I also edited the setup.py file and recomplie python3
# Detect SSL support for the socket module (via _ssl)
search_for_ssl_incs_in = [
'/usr/local/ssl/include',
'/usr/local/include/openssl', #I've added this line
'/usr/contrib/ssl/include/'
]
I've complied the openssl with the path configuration
#tar -xzvf openssl-***.tar.gz
#./config --prefix=/usr/local --openssldir=/usr/local/openssl
#make & make install
CentOS 7
Python 3.6

I found some solution:
if you use centos,try:
s1
yum install openssl-devel -y
then when you compile, append --with-ssl,just like this
./configure prefix=/usr/local/share/python3 --with-ssl
s2
-- install depend library, make share compile is fluent
yum install -y zlib zlib-devel openssl-devel sqlite-devel bzip2-devel libffi libffi-devel gcc gcc-c++
(ubuntu)sudo apt-get install libz-dev
wget --no-check-certificate http://www.openssl.org/source/openssl-1.1.1.tar.gz
tar -zxvf openssl-1.1.1.tar.gz
cd openssl-1.1.1
./config --prefix=$HOME/openssl shared zlib
make && make install
-- configure shared ld library path so that compile can find it
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/openssl/lib" >> $HOME/.bash_profile
source $HOME/.bash_profile
(zsh user has some different with .zsh_profile)
-- compile with openssl path
./configure prefix=/usr/local/share/python3 --with-openssl=$HOME/openssl

I faced the same issue, I installed python from source and didn't enabled ssl option while compiling. So I find the solution in the following article. You need to find ssl section Modules/Setup.dist and uncomment that section. Hope this will help someone.

pyenv: BUILD FAILED (Ubuntu 15.04 using python-build 20160509)

I have already installed the pyenv on my system, and the command pyenv install --list goes well. But when I download some Python versions using pyenv install 2.7.11, it turns out as follows:
Downloading Python-2.7.11.tgz...
-> https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz
error: failed to download Python-2.7.11.tar.gz
BUILD FAILED (Ubuntu 15.04 using python-build 20160509)
I didn't find any similar problem on the official Common build problems.
Is it that pyenv didn't catch the new Ubuntu update?

The command pyenv install -v 2.7.11 gave me a similar error.
Installing the pyenv requirements solved my issue :
sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev
(See https://github.com/pyenv/pyenv/wiki/Common-build-problems)
Then :
pyenv install 2.7.11

To install python by using pyenv, the command pyenv install -v 2.7.11 could show more message than pyenv install 2.7.11. So it will be easy for me to locate the error. This time I got
curl: (77) error setting certificate verify locations: CAfile:
/etc/pki/tls/certs/ca-bundle.crt
CApath: none
This problem has already been solved in curl: (77) error
# sudo apt-get install ca-certificates
The issue was that curl expected the certificate to be at the path /etc/pki/tls/certs/ca-bundle.crt but could not find it because it was at the path /etc/ssl/certs/ca-certificates.crt.
Copying my certificate to the expected destination by running
sudo cp /etc/ssl/certs/ca-certificates.crt /etc/pki/tls/certs/ca-bundle.crt
worked for me. You will need to create folders for the target destination if they do not exist by running
sudo mkdir -p /etc/pki/tls/certs
If needed, modify the above command to make the destination file name match the path expected by curl, i.e. replace /etc/pki/tls/certs/ca-bundle.crt with the path following "CAfile:" in your error message.

The default installing curl don't support https protocol. You can check with "curl --version". you can find http but no https.
So you need to reinstall curl refering how to install curl and libcurl.
./configure --prefix=$YOUR_DIR --with-ssl
make
make install
or find where you install the ssl for example /opt/OpenSSL
./configure --with-ssl=/opt/OpenSSL
make
make install
how to install curl and libcurl

I was facing a similar issue. I was using the fedora 29 and trying to install the python 2.7.5. Below was the error output.
BUILD FAILED (Fedora 29 using python-build 1.2.9-35-gb6109093)
Inspect or clean up the working tree at /tmp/python-build.20190405111845.17497
./python -E -S -m sysconfig --generate-posix-vars
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
make: *** [Makefile:464: pybuilddir.txt] Segmentation fault (core dumped)
I was searching for a solution then stumbled upon this thread.. which says:--
issue was a longstanding in Python < 2.7.15, and recently triggered with recent compiler. They fixed it in 3.x, and backported into 2.7.15. So instead of just gathering the patch back, maybe it's best to update our 2.x version to 2.7.15 with the cross compilation patches. I'm a little bit afraid of the implication (ssl certificates issues on 2.7.x at some point).
I tried installing 2.7.15 with pyenv install 2.7.15 and it installs without errors.

pip install reportlab error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

use ubuntu virtualenv. I try to install reportlab
the command is
pip install reportlab
in this directory
(company2)stefano#stefano-X550EP:~/htdocs/company2$
the error is
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
actually the pip list is:
argparse (1.2.1)
Django (1.7.7)
html5lib (0.999)
pip (1.5.4)
pisa (3.0.33)
PyPDF2 (1.24)
setuptools (2.2)
wsgiref (0.1.2)
xhtml2pdf (0.0.6)
I need reportlab to use xhtml2pdf because now the django project gives me this error:
No module named reportlab.lib.colors

without your full error log, it is impossible to tell. But I bet you are just missing python-dev.
try installing it:
$ sudo apt-get install python-dev
then pip install reportlab again.
hope that helps.
see: installing Reportlab (error: command 'gcc' failed with exit status 1 )

The sudo apt-get install python-dev is almost correct. But if you are using Python 3.x you need to change it to install python3-dev. This made all the difference in my django, postgresql, venv setup.

I was also having the same issue while installing readability-lxml a python package. Then I tried installing following packages, and it worked for me.
sudo apt install libpq-dev python-dev libxml2-dev libxslt1-dev libldap2-dev libsasl2-dev

It majorly depends on the type of packages you suppose to install. Frequently its failing due to the missing of libsasl2-dev a package for authentication abstraction library which uses in the Ubuntu version
First, install:
sudo apt-get install libsasl2-dev
then run:
sudo pip install sasl

I know I am late. But after searching all over this is what solved mine. It need c++ compiler for reportlab. Try Installing g++, the c++ compiler
sudo apt install g++