We Keep Coding

Problem when installing Python from source, SSL package missing even though openssl installed

The Problem
Trying to install Python-3.11.1 from source on Zorin OS (Ubuntu16 based) I get the following errors when I try to pip install any package into a newly created venv:
python3.11 -m venv venv
source venv/bin/active
pip install numpy
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
Could not fetch URL https://pypi.org/simple/numpy/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/numpy/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy
Obviously, the SSL package seems to be missing, however I made sure to have both openssl and libssl-dev installed before installing python. More specifically, I made sure to have all packages installed lined out here.
The Exact Steps I Took To Install
Make sure all packages that are required are installed (the once above)
cd .../python-installs
Download Python from python.org
tar -xvzf Python-3.11.1.tgz
cd Python-3.11.1 and then
./configure \
--prefix=/opt/python/3.11.1 \
--enable-shared \
--enable-optimizations \
--enable-ipv6 \
--with-openssl=/usr/lib/ssl \
--with-openssl-rpath=auto \
LDFLAGS=-Wl,-rpath=/opt/python/3.11.1/lib,--disable-new-dtags
make <- Note that I get a lot off error messages from gcc here, very similar to this, however it seems its successful at the end
make altinstall
Parts of this installation process are from [1], [2]
Running python3.11 seems to work fine, however I cannot pip install anything into a venv created by Python3.11.1.
Other Possible Error Sources
Before trying to reinstall Python3.11.1, I always made sure to delete all files in the following places that were associated with Python3.11.1:
/usr/local/bin/...
/usr/local/lib/...
/usr/local/man/man1/...
/usr/local/share/man/man1/...
/usr/local/lib/pkgconfig/...
/opt/python/...
I also tried adding Python-3.11.1 to PATH by adding
PATH=/opt/python/3.11.1/bin:$PATH
to /etc/profile.d/python.sh, but it didn't seem to do much in my case.
When configuring the python folder I am using --with-openssl=/usr/lib/ssl, though perhaps I need to use something else? I tried --with-openssl=/usr/bin/openssl but that doesn't work because openssl is a file and not a folder and it gives me an error message and doesn't even configure anything.
Conclusion
From my research I found that most times this error relates to the openssl library not being installed (given that python versions >= 3.10 will need it to be installed), and that installing it and reinstalling python seemed to fix the issue. However in my case it doesn't, and I don't know why that is.
The most likely cause is that something is wrong with my openssl configuration, but I wouldn't know what.
Any help would be greatly appreciated.

After some more research, I realized that I didn't have libbz2-dev installed, which is obviously the first thing one should check if they get the errors above but oh well. For anyone who still finds himself struggling, here are my complete steps I took:
Make sure the following libraries are installed
apt show libbz2-dev
apt show openssl
apt show libssl-dev
# Other libraries that might also be needed
apt show liblzma-dev
cd .../python-installs
Download the target Python version from python.org as Gzipped tar ball
tar -xvzf Python-3.11.1.tgz
sudo mkdir opt/python
sudo mkdir opt/python/3.11.1
cd Python-3.11.1 and then
./configure --prefix=/opt/python/3.11.1 \
--enable-optimizations
make <- Note that I still get a lot of error messages from gcc, also get a always_inline not in line error message
sudo make altinstall
Add PATH=/opt/python/3.11.1/bin:$PATH to the file /etc/profile.d/python.sh
reboot
Then to test if the error is gone one can for example test:
python3.11 -m venv venv
source venv/bin/active
pip install pandas
python3.11
import pandas
exit()
If there are no errors then everything worked out. Obviously the version needs to be changed to the actual target version of yours.
Note
If you your newly installed python version does not appear in terminal, it might be because the file /etc/profile.d/python.sh already existed with the Path to the python version (for example, if you had to install it multiple times). In that case, delete the file (or at least the PATH to the target version) and then recreate it. After rebooting it should appear in terminal.

Pip Install Pulp Error: Do I need admin rights to install modules?

I am fairly new to python so pardon my lack of jargon. I am trying to install the pulp module onto a work pc. I have downloaded the package from the pypi.org website into the program data folder on this pc.
I have tried:
pip install pulp
import pulp
python.exe -m pip install pulp
And this is the error that pops up every time.
pip install pulp
Collecting pulp
Note: you may need to restart the kernel to use updated packages.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired'))': /simple/pulp/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired'))': /simple/pulp/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired'))': /simple/pulp/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired'))': /simple/pulp/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 authenticationrequired'))': /simple/pulp/
ERROR: Could not find a version that satisfies the requirement pulp (from versions: none)
ERROR: No matching distribution found for pulp
I just would like to get pulp installed so that I can access the functions that come with it.

You need to tell pip about your proxy to use it that way. Something like this:
pip install --proxy http://user:password#host:port pulp
You may have to ask IT about the details.
Alternatively tell pip to install the file you downloaded:
pip install /path/to/downloaded/file
But you might find problems if it requires other packages and your internet connection is not working.

This ultimately may be a duplicate of the following, assuming local install:
python-pip-install-from-local-dir and
installing-python-packages-from-local-file-system-folder-to-virtualenv-with-pip
However, to answer in general, when you want to install a python module, you have a few options. The simple approach is generally to use pip (pip2 for explicit python2, pip3 for explicit python3) which automatically tries to download the module from the python module repositories.
As Goyo points out in their answer, you need to have access to the Internet for that to happen, and as such must specify settings to work through your proxy.
If, however, you have downloaded the module onto your computer already, you can also use it directly, in two ways.
Firstly, as per python-pip-install-from-local-dir, you can specify a package to install.
If you want a quick use, without pip involved (custom package, out of date, etc.), you can also locate your site packages folder with:
python -m site --user-site
Running without --user-site will also show the root site for your distro.
Extract your module and copy it into the user site folder, eg. for python3 "~/.local/lib/python3.4/site-packages". I create sym-links for python, python3 etc. to point to a generic python-site-packages directory, to help different versions locate the correct spot:
lrwxrwxrwx python -> /home/user/.local/lib/python-site-packages
lrwxrwxrwx python2 -> /home/user/.local/lib/python-site-packages
lrwxrwxrwx python2.7 -> /home/user/.local/lib/python-site-packages
lrwxrwxrwx python3 -> /home/user/.local/lib/python-site-packages
lrwxrwxrwx python3.4 -> /home/user/.local/lib/python-site-packages
drwxrwxr-x python-site-packages
Then you can include the module in your python code. However, having bypassed pip means that you won't necessarily have the latest module, and other modules will not know that the requirement has been met.
To check officially installed modules:
pydoc -g (python2)
pydoc3 -b (python3)

I presume that you are using a Windows machine.
Choose the Run as administrator option when you open cmd and just type pip, in order to ensure that your pip is working perfectly.
If you encounter no errors, then type pip install pulp and the module will be downloaded.
Simply import the module as import pulp

pip Install throwing No matching distribution found for

I am on my work PC (corporate proxy) and i am unable to install anything using pip install.
I have tried the following:
pip install QuantLib-Python
pip install --proxy 'webproxy.abc.123.nsroot.net:8080' QuantLib-Python
pip install --proxy "user:pass#webproxy.abc.123.nsroot.net:8080" QuantLib-
Python
no matter the library i always receive this message. I am on Python 3.7.3
Collecting QuantLib-Python
Note: you may need to restart the kernel to use updated packages.
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x0000000002BEAFD0>: Failed to establish a new connection: [Errno 11004] getaddrinfo failed')': /simple/quantlib-python/
Retrying
No matching distribution found for QuantLib-Python
On my Home PC i ran the command pip install QuantLib-Python with the version of python and it worked perfectly.
please help!

Which repositories are pip packages installed from?

I need to download a package using pip. I ran pip install <package> but got the following error:
[user#server ~]$ pip install sistr_cmd
Collecting sistr_cmd
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.
VerifiedHTTPSConnection object at 0x7f518ee0cd90>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/sistr-cmd/
Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.
VerifiedHTTPSConnection object at 0x7f518ee0c290>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/sistr-cmd/
Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.
VerifiedHTTPSConnection object at 0x7f518ee0c510>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/sistr-cmd/
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.
VerifiedHTTPSConnection object at 0x7f518ee0cf10>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/sistr-cmd/
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.
VerifiedHTTPSConnection object at 0x7f518ee0c190>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/sistr-cmd/
Could not find a version that satisfies the requirement sistr_cmd (from versions: )
No matching distribution found for sistr_cmd
I verified that the source of the problem is the network blocking most sites because I am working behind a proxy (required by the organization). To allow these downloads, I need to compile the list of urls of the source of the downloads and send it to the network admins to unblock.
According to pip documentation (cited and explained in brief in the pip Wikipedia article), “Many packages can be found in the default source for packages and their dependencies — Python Package Index (PyPI)," so I went to the PyPI page for Biopython and found the github repository and the required dependencies for the package. There are also download links on the PyPI page and I want to be sure that all sources for the download are allowed. So does pip install from the original source of a package (the github repository or wherever the original package is hosted), the packages listed in the PyPI page under downloads, or does it search through both?
Thank you in advance for the help.

It appears a network connection, later you may try below:
pip install sistr-cmd
See the difference between - and _ for more info check here

According to user abarnert:
"If you run pip with the --verbose flag, it will show you all the links it considers. Generally speaking, it goes to https://pypi.org/simple/{PACKAGE} first. That page will have links to downloads for wheels and source for all versions that the maintainer has uploaded, and it will look for the appropriate version's wheel by trying its links in order. Usually there is no github repo listed as a source.
If you want to install everything from github (or any other location) that you know, the simplest thing to do is to manually give the repo to pip instead of just the package names. Ideally, install everything you care about that way on a machine that isn't firewalled from most of the internet, build a requirements.txt file (as explained here), then copy that requirements file to your firewalled machine and try to install it."
Running pip install with the --verbose flag shows extensive detail about where the package is being pulled from and about the network connection and traceback.

Cannot install djangorestframework on ec2 instance with pip

I use a micro ec2 instance with python 2.7.10 installed.
When i try to install djangorestframework with "pip install djangorestframework", it failed and here is the log:
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(, 'Connection to pypi.python.org timed out. (connect timeout=15)')'

The problem is pip will open a https connection to pyip. Re-check outbound security rule of the instance and add https rule