We Keep Coding

Can't install Proj 8.0.0 for cartopy linux

I am trying to install Cartopy on Ubuntu and need to install proj v8.0.0 binaries for Cartopy. However when I try to apt-get install proj-bin I can only get proj v6.3.1. How do I install the latest (or at least v8.0.0) proj for cartopy?

I'm answering my own question here partly to help others with this problem, and partly as an archive for myself so I know how to fix this issue if I come across it again. I spent quite a while trying to figure it out, and wrote detailed instructions, so see below:
Installing cartopy is a huge pain, and I've found using conda to be a very bad idea (it has bricked itself and python along with it multiple times for me)
THIS INSTALLATION IS FOR LINUX.
Step 0. Update apt:
apt update
Step 1. Install GEOS:
Run the following command to install GEOS:
apt-get install libgeos-dev
In case that doesn't do it, install all files with this:
apt-get install libgeos-dev libgeos++-dev libgeos-3.8.0 libgeos-c1v5 libgeos-doc
Step 2. Install proj dependencies:
Install cmake:
apt install cmake
Install sqlite3:
apt install sqlite3
Install curl devlopment package:
apt install curl && apt-get install libcurl4-openssl-dev
Step 3. Install Proj
Trying apt-get just in case it works:
Unfortunately, cartopy requires proj v8.0.0 as a minimum, but if you install proj using apt you can only install proj v6.3.1
Just for reference in case anything changes, this is the command to install proj from apt:
apt-get install proj-bin
I'm fairly sure this is all you need, but in case it's not, this command will install the remaining proj files:
apt-get install proj-bin libproj-dev proj-data
To remove the above installation, run:
apt-get remove proj-bin
or:
apt-get remove proj-bin libproj-dev proj-data
Building Proj from source
So if the above commands don't work (it's not working as of 2022/4/8), then follow the below instructions to install proj from source:
Go to your install folder and download proj-9.0.0 (or any version with proj-x.x.x.tar.gz):
wget https://download.osgeo.org/proj/proj-9.0.0.tar.gz
Extract the tar.gz file:
tar -xf proj-9.0.0.tar.gz
cd into the folder:
cd proj-9.0.0
Make a build folder and cd into it:
mkdir build && cd build
Run (this may take a while):
cmake ..
cmake --build .
cmake --build . --target install
Run to make sure everything installed correctly:
ctest
The test command failed on one test for me (19 - nkg), but otherwise was fine.
You should find the required files in the ./bin directory
Finally:
Move binaries to the /bin directory:
cp ./bin/* /bin
As per Justino, you may also need to move the libraries:
cp ./lib/* /lib
Now after all this, you can finally install cartopy with pip:
pip install cartopy
After doing this, my cartopy still wasn't working. I went home to work on this next week, came back, and all of a sudden it was working so maybe try restarting

The libraries should be copied manually
sudo cp ./lib/* /lib
This works for me

Error installing Dlib library for python

While trying to install Dlib library for Python, using this tutorial http://dlib.net/compile.html.
This tutorial says to do this -
cd examples
mkdir build
cd build
cmake ..
cmake --build . --config Release
I have created a build and when I'm giving the command cmake ..
It shows me this error -
CMake Error: The source directory "C:/Python27/dlib-19.4.0/python_examples"
does not appear to contain CMakeLists.txt.

These examples do not need to be compiled, only the C++ one have to.
You will however have to compile the dlib python interface, you will find information about this right underneath the C++ section, on this page : http://dlib.net/compile.html

Simplest way to install dlib:
just execute below command line.
sudo apt-get install python-dev python-pip
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libx11-dev libatlas-base-dev
sudo apt-get install libgtk-3-dev libboost-python-dev
pip install dlib

Python 3 No module named '_ssl'

The Problem
While I run you python3 application, it shows
File "/usr/local/lib/python3.6/ssl.py", line 101, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
What I've tried
install the dependencies
yum install openssl-devel
I also edited the setup.py file and recomplie python3
# Detect SSL support for the socket module (via _ssl)
search_for_ssl_incs_in = [
'/usr/local/ssl/include',
'/usr/local/include/openssl', #I've added this line
'/usr/contrib/ssl/include/'
]
I've complied the openssl with the path configuration
#tar -xzvf openssl-***.tar.gz
#./config --prefix=/usr/local --openssldir=/usr/local/openssl
#make & make install
CentOS 7
Python 3.6

I found some solution:
if you use centos,try:
s1
yum install openssl-devel -y
then when you compile, append --with-ssl,just like this
./configure prefix=/usr/local/share/python3 --with-ssl
s2
-- install depend library, make share compile is fluent
yum install -y zlib zlib-devel openssl-devel sqlite-devel bzip2-devel libffi libffi-devel gcc gcc-c++
(ubuntu)sudo apt-get install libz-dev
wget --no-check-certificate http://www.openssl.org/source/openssl-1.1.1.tar.gz
tar -zxvf openssl-1.1.1.tar.gz
cd openssl-1.1.1
./config --prefix=$HOME/openssl shared zlib
make && make install
-- configure shared ld library path so that compile can find it
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/openssl/lib" >> $HOME/.bash_profile
source $HOME/.bash_profile
(zsh user has some different with .zsh_profile)
-- compile with openssl path
./configure prefix=/usr/local/share/python3 --with-openssl=$HOME/openssl

I faced the same issue, I installed python from source and didn't enabled ssl option while compiling. So I find the solution in the following article. You need to find ssl section Modules/Setup.dist and uncomment that section. Hope this will help someone.

"The C extension could not be compiled' error. - while installing Flask

I get the error below when installing flask in virtualenv on debian 7. apt-get-install tells me I already have GCC. I tried apt-get install libpcre3-dev but then reinstalled flask with pip install Flask-scss --force-reinstall -I but still got the same error. How do I fix this so that the speedups are used?
The Error:
markupsafe/_speedups.c:12:20: fatal error: Python.h: No such file or directory
compilation terminated.
==========================================================================
WARNING: The C extension could not be compiled, speedups are not enabled.
Failure information, if any, is above.
Retrying the build without the C extension now.
==========================================================================
WARNING: The C extension could not be compiled, speedups are not enabled.
Plain-Python installation succeeded.
==========================================================================

You need to install the python-dev package too; it depends on the right package that contains Python.h:
apt-get install python-dev

sudo apt-get install python-dev
Or
sudo apt-get install python3-dev for python3

For those using Amazon Web Services (AWS) .
Install your system’s Development tool-chain per this reference:
yum (Amazon AMI, RedHat, Centos)
sudo yum groupinstall -y "Development Tools"
apt (Debian, Ubuntu, Mint)
sudo apt-get install -y build-essential

This is a common problem on windows also when install libraries having c extensions. This problem in windows can be solved by install mingw, which stands for Minimalist GNU for Windows.
To install mingw for anaconda:
conda install mingw
To know more about mingw, have a look at http://www.mingw.org/

building Python from source with zlib support

When building Python 3.2.3 from source on Ubuntu 12.04, the zlib module is not available.
I downloaded the official source distribution from python.org, and attempted to build and install it with the following commands.
tar xfa Python3.2.3.tar.bz2
cd Python-3.2.3
./configure --prefix=/opt/python3.2
make
sudo make install
The make command output includes the following.
Python build finished, but the necessary bits to build these modules were not found:
_curses _curses_panel _dbm
_gdbm _sqlite3 _ssl
_tkinter bz2 readline
zlib
After running make install and starting the interpreter, the zlib module cannot be imported.
I confirmed that the zlib1g-dev package is installed on my system.
I also found this similar question, which suggests adding the --with-zlib flag to the ./configure command. However, that returns an error that it is an unrecognized option and has no effect.

I had a similar problem on CentOS 6.3 and python 3.2.3
I solved it by:
Edit /Modules/Setup and uncomment the line:
zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz
change to directory /Modules/zlib:
./configure
make
sudo make install
then compiled my python3.2 source.
and was then able to test import zlib and it all worked fine :)

I am using CentOS 6.6 and was recieving zlib errors. None of the other answers proposed here worked for me (including the fix for CentOS 6.3 of uncommenting a line in Modules/Setup). I have fixed it using the following commands.
yum groupinstall "Development tools"
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
Then configuring and installing python as follows:
./configure --prefix=/usr/local LDFLAGS="-Wl,-rpath /usr/local/lib"
make && make altinstall
I can now import zlib in /usr/local/bin/python2.7 with no problems.
These instructions are slightly modified from an article found here.

The solution is to install the Ubuntu package dpkg-dev.
sudo apt-get install dpkg-dev
The reason is explained here.
In short, recent versions of Ubuntu don't store libz.so in the standard /usr/lib location, but rather in a platform specific location. For example, on my system is is in /usr/lib/x86_64-linux-gnu. This prevents Python's build system from finding it.
The dpkg-dev package installs the dpkg-architecture executable, which enables Python to find the necessary libraries.
The original question was about Python 3.2.3. I also downloaded Python 2.7.3 and confirmed that the same problem exists, and this solution is applicable to it as well.

For anyone who's trying to use a non-system / non-standard zlib (e.g. building your own from source), make sure to pass both CPPFLAGS (not CFLAGS!) and LDFLAGS to ./configure. For example, if your zlib is in /opt/zlib:
./configure CPPFLAGS='-I/opt/zlib/include' LDFLAGS='-L/opt/zlib/lib'
make
sudo make install
I ended up going down the rabbit hole trying to figure out why our Python wasn't building with zlib support and found out that the CPython setup.py does not look at CFLAGS for include dirs, only CPPFLAGS:
https://github.com/python/cpython/blob/master/setup.py#L562

The only solution that helped me with installing python 3.5.1 was to apt-get zlib1g-dev (and other packages such as python-setuptools and python-pip) and then rebuild python 3.5.1 from source.
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install build-essential python-dev python-setuptools python-pip python-smbus
sudo apt-get install build-essential libncursesw5-dev libgdbm-dev libc6-dev
sudo apt-get install zlib1g-dev libsqlite3-dev tk-dev
sudo apt-get install libssl-dev openssl
cd ~
mkdir build
cd build
wget https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tgz
tar -zxvf Python-3.5.1.tgz
cd Python-3.5.1
./configure
make
sudo make install
Taken from: https://github.com/MrYsLab/xideco/wiki/Installing-Python-3.5
As I undestand new build of python is made with inclusion of previously apt-getted related packages.
So when you browse the content of new Python-3.5.1/lib/site-packages there will be pip and setuptools. More importantly, they will be copied to any virtualenv you make using Python-3.5.1 AND this virtualenv will use THEM insted of system-default. This is very, very important to rememmber when installing new python version. Otherwise one might get into a black hole of errors such as:
zlib not installed;
"pip install ..." executed from virtualenv that installs package to system-default python instead of virtualenv.

I was having the same error while working on MAC
My MAC OS version
$ uname -v
Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64
python3.4 is used here
Issue(s)
zlib not available while using python3.4
$ python3.4 get-pip.py
Traceback (most recent call last):
File "get-pip.py", line 20204, in
main()
File "get-pip.py", line 152, in main
bootstrap(tmpdir=tmpdir)
File "get-pip.py", line 82, in bootstrap
import pip
zipimport.ZipImportError: can't decompress data; zlib not available
Rebuilding Python fails
./configure --with-zlib-dir=/usr/local/lib
...
configure: WARNING: unrecognized options: --with-zlib-dir
...
Solution
Ensure zlib is installed .
By default it will be installed in /usr/lib
ls /usr/lib/libz.*
If not installed,
a. download and install
i)from zlib.net site
or
ii) from a git repo like the below
git clone https://github.com/madler/zlib.git
or
iii). Use the zlib source in the python source directory
Modules/zlib
b. Install zlib
./configure --prefix=/usr/local
make
sudo make install
2.Edit /Module/Setup by uncommenting the line below
"#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz "
3.Rebuild the Python3.4 from source again
cd ${PYTHON_SRC_CODE_DIR}
./configure --prefix=${PYTHON_HOME_DIR}
make
sudo make install
4.Confirm installation
Please note gzip depends on zlib.
nbr_repeation=100
f=open("some_file.txt","at")
for line in range(nbr_repeation):
print('[{}] This file will be compressed using python zlib/gzipmodule'.format(line),file=f)
f.close()
f=open("some_file.txt","rt")
import gzip
gz=gzip.open('some_file.gz', 'wt')
for line in f : gz.write(line)
gz.close() # Like to be clean exit
f.close() # Like a clean exit
"""confirm the creation of the compressed gzip files"""
import os
print([ (file,os.stat(file)[6],"bytes") for file in os.listdir(".") if file.startswith("some")])

sudo apt-get install build-essential python-dev
Even though python-dev is for python2.7 it will still bring in all the necessary dependencies.
You will then need to do:
./configure
make
sudo make install
To rebuild python3

The easiest solution I found, is on python.org:
sudo apt-get build-dep python3.6
If that package is not available for your system, try reducing the minor version until you find a package that is available in your system’s package manager.
If you see something like this: E: You must put some ‘source’ URIs in your sources.list, Open Software & Updates and enable Source code.
I tried explaining details, on a blog post.

sudo apt-get install zlib1g-dev
is what worked for me.

For anyone having the same error on macOS Mojave, this is the easiest solution for installing/linking the header files:
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg
Then just build Python again as usual (also works with pyenv builds).

This is how I've built Python 3.7 on a CentOS 7 machine without devel libraries and installed it into user's ~/.local without sudo. zlib, OpenSSL and readline modules are built.
Download Python source in .tgz format from https://www.python.org/downloads. Download zlib-devel-1.2.7-18.el7.x86_64.rpm, openssl-devel-1.0.2k-19.el7.x86_64.rpm, krb5-devel-1.15.1-50.el7.x86_64.rpm, libcom_err-devel-1.42.9-19.el7.x86_64.rpm, readline-devel-6.2-11.el7.x86_64.rpm from https://centos.pkgs.org for SSL, zlib and readline modules. Put all files into $DIST_PATH.
cd ~
DIST_PATH=<path to downloaded files>
for f in $DIST_PATH/*.rpm; do rpm2cpio $f | cpio -idmv; done
mkdir ~/usr/lib
# symlinks in ~/usr/lib64 are broken, so create new links to system libraries in ~/usr/lib and pass this folder to ./configure
for f in ~/usr/lib64/*.so; do ln -s /lib64/`readlink $f` ~/usr/lib/`basename $f`; done
tar -xzf $DIST_PATH/Python-3.7.13.tgz && cd Python-3.7.13
# That machine has devtoolset-7 with newer version GCC
scl enable devtoolset-7 bash
# curly brackets are important here, otherwise LDFLAGS is -LOME/usr/lib
./configure --enable-optimizations --prefix=$HOME/.local --with-openssl=$HOME/usr CPPFLAGS='-I${HOME}/usr/include' LDFLAGS='-L${HOME}/usr/lib'
make
# (!) altinstall is used, use python3.7 command to access newly built Python
make altinstall
rm -rf ~/usr
Links:
Unpacking RPM packages
krb5.h error
com_err.h
CPPFLAGS, LDFLAGS