We Keep Coding

Installing python extension module : understanding skbuild+setuptools

I am one of the devs of a (fairly large) C++ simulation tool. Disclaimer : I'm more of a physicist than a dev. I wrote Python bindings for that project using pybind11.
I managed to get the Python module to compile with cmake. I then managed to write a setup.py file using skbuild that does compile the Python module :
python3 setup.py sdist bdist_wheel
In _skbuild/linux-x86_64-3.9/cmake-build/lib/ (and in the tar archive dist/cytosim-0.0.0.tar.gz) there is indeed a compiled library : cytosim.cpython-39-x86_64-linux-gnu.so.
However, when I want to install the module :
pip3 install dist
I get an error :
gcc: error: src/py3/dist.c: No such file or directory
I am very confused because I do not have an directory called py3 in src.
Any pointer ? Anything I'm doing wrong ? Thanks !

The command
pip3 install dist
tries (and fails) to install the dist package from the pypi repository.
Maybe try
pip3 install dist/cytosim-0.0.0.tar.gz
instead.

letsencrypt failed with ImportError: No module named interface

I'm using Amazon linux, and I followed some steps for using letsencrypt that easily found in google search, but all it fails with:
Error: couldn't get currently installed version for /root/.local/share/letsencrypt/bin/letsencrypt:
Traceback (most recent call last):
File "/root/.local/share/letsencrypt/bin/letsencrypt", line 7, in <module>
from certbot.main import main
File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/certbot/main.py", line 11, in <module>
import zope.component
File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/zope/component/__init__.py", line 16, in <module>
from zope.interface import Interface
ImportError: No module named interface
What I do is:
# git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
# /opt/letsencrypt/letsencrypt-auto --debug
That's it. So I tried to fix this, but dozens of solution that I found all won't worked to me.
Most of them said try this:
unset PYTHON_INSTALL_LAYOUT
But still got same error, nothing changes. And someone said that type this:
pip install --upgrade pip
But after typed that, I can't use pip anymore, it failed with some kind of command not found error, so I had recreated my server again.
I also tried to use CertBot, but it gives me exactly same error!
I'm using Linux 4.4.51-40.58.amzn1.x86_64 x86_64, need a help. I spent almost a day, but nothing progressed.
Every solution that I was found were not worked to me. Any advice will very appreciate it.

Removing certbot directory did the trick for me.
rm -rf /opt/eff.org/certbot/
wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto
sudo ./certbot-auto certonly --standalone -d example.com --no-bootstrap

I had same issue and after a long run, a very simple thing solved this issue on my AWS instance:
Move the letsencrypt cache files to another folder (considering you run it as root/sudo):
sudo mv /root/.local/share/letsencrypt /root/.local/share/letsencrypt-old
Downloaded a brand new version of letsencrypt
git clone https://github.com/letsencrypt/letsencrypt
Run the letsencrypt command to test if its working again:
sudo ./letsencrypt-auto --debug
During my search for a solution I also updated pip which might have helped. Although my attempts after pip update did not solved my issue in the same way as cleaning up the letsencnrypt cache folder.
I hope this helps. If not, some links I can share about same issue:
certbot zope.interface error
no module named interface
letsencrypt failed with no module named interface

Part of the issue for me was related to some strange default behavior around 64 bit packages installing, but not being picked up by python. After getting this issue [it's mostly installed at this point], run the following
cd /root/.local/share/letsencrypt
\cp -r ./venv/lib64/* ./venv/lib/
Then retry the command. The install locations of the python virtual environment change with different versions and operating systems, but the general principle has helped me debug two different installs.

I have also faced this issue multiple times and every time I have to repeat these following steps:
Remove cache:
sudo rm -rf /root/.local/share/letsencrypt/
sudo rm -rf /opt/eff.org/certbot/`
then,
unset PYTHON_INSTALL_LAYOUT
Install Let’s Encrypt by cloning the github repository into /opt/letsencrypt
git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
then run installer again
/opt/letsencrypt/letsencrypt-auto --debug
these steps always works for me.

I solved it following the next steps
Download certbot-auto by going to this link.
Delete letsencrypt folder:
sudo rm -rf /opt/eff.org/
Install cryptography module:
sudo python -m pip install cryptography
Run certbot-auto --debug
sudo ./certbot-auto --debug

Installing GDAL Python binding in Ubuntu to use as standalone module

I am trying to export a GeoTiff with Blender using the Blender Python API (based on Python 3), so I've decided to install GDAL on Ubuntu (14.04). What I would like is to get the module as a standalone folder that I could put in the modules directory of Blender (/home/user/.config/blender/2.73/scripts/modules).
The thing is I've run through several different problems trying to install GDAL. I've tried to install from source (for GDAL 2.0.0) here : Official PyPi Gdal
I ran sudo apt-get install libgdal-dev gdal-bin (I list it here because it may be important)
When I am in the extracted GDAL folder, using python setup.py build & python setup.py install, the library installs to /usr/local/lib/python2.7/dist-packages/osgeo. However, when I run python from command line, running from osgeo import osr returns ImportError: No module named _gdal
Following GDAL via pip , I used pip (pip install GDAL) to install the library, and the folder it went to was /usr/lib/python3/dist-packages/osgeo (using pip show ...). Again, running python3 and trying to import results in the same error. Of course, when I copy-paste each folder in the blender module directory, I get the same error in the Blender Python console.
So I decided to compile the sources using ./configure --with-python & make & make install in the source folder. I then copied the folder GDAL-x.x.x/build/lib.linux-x86_64-3.4/osgeo to the blender modules directory and got this time the error when importing : ImportError: /home/yvesu/.config/blender/2.73/scripts/modules/osgeo/_gdal.so: undefined symbol: _Py_ZeroStruct.
Trying to compile with python3 using python3 setup.py build returns the error error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
EDIT 1:
I think I've found the solution : I went to the directory swig/python (not found in a GDAL-1.11.0 folder but gdal-1.11.0 fodler, can't remember where I downloaded it from), ran python3 setup.py build & python3 setup.py install and could finally find the folder in /usr/local/lib/python3.4/dist-packages/GDAL-1.11.0-py3.4-linux-x86_64.egg/osgeo. When I put this osgeo folder oni the Blender modules directory, I was able to import osgeo in Blender. I will report if anything went wrong.
I think I've listed all my attempts at installing GDAL on Ubuntu. Can anyone point me in the right direction? Do you think it is even possible to install it as a standalone module, or do I need linked libraries through LD_LIBRARY_PATH?

Here is the solution I've found :
Download Gdal sources (v2.0.0 is the current stable release) from ftp://ftp.remotesensing.org/gdal/2.0.0/ or http://download.osgeo.org/gdal/2.0.0/ and untar
Go to the directory gdal2.0.0/swig/python
Run python3 setup.py build & python3 setup.py install
Finally find the module folder in, on Ubuntu : /usr/local/lib/python3.4/dist-packages/GDAL-2.0.0-py3.4-linux-x86_64.egg/osgeo
I can now use it in Blender (copying in the modules directory)

Sphinx "sphinxcontrib.httpdomain" package

I downloaded the 'sphinxcontrib.httpdomain' package and put into Sphinx-1.2 directory:
..../Sphinx-1.2/sphinx/sphinxcontrib/httpdomain.py
I modified the conf.py under the extensions
=[.., 'sphinx.sphinxcontrib.httpdomain']
and 'make clean html' failed:
Extension error: Could not import extension
sphinx.sphinxcontrib.httpdomain (exception: No module named
sphinxcontrib.httpdomain)
Any suggestions?

I found the sphinx.sphinxcontrib.httpdomain module inside the Python dependencies. It could be included inside Python or just using the Sphinx builder (modifying the config.pyas pointed on the comment, not tested by me).
To solve it 'as it is':
$ sudo apt-get install python-sphinxcontrib-httpdomain
From a Debian scratch image, I needed the following packages to successfully compile latexpdf:
$ sudo apt-get install python-sphinx python-sphinxcontrib-httpdomain texlive texlive-latex-extra
So I guess that python-sphinxcontrib-httpdomain also solves the problem for make html. Hope it helps!

Install m2crypto on a virtualenv without system packages

I have created a virtual environment without the system packages with python's virtualenv in Ubuntu and installed m2crypto, but when I execute a shell and I try to import M2Crypto i get the following error:
ImportError: /home/imediava/.virtualenvs/myenv/local/lib/python2.7/site-
packages/M2Crypto/__m2crypto.so: undefined symbol: SSLv2_method
From outside the environment I run into the same problem unless from ubuntu I install python-m2crypto with apt-get. I know that I could create the environment with the system packages but I would prefer not to do it.
Is there anyway that I could create a virtual environment without the system packages and then install m2crypto with pip without running into the SSLv2_method?

There seems to be a regression bug from an earlier version of M2Crypto.
After placing M2Crypto's source in your virtualenv, you can try to patch it with the diff code below.
You do this by downloading the source code, untar it via:
tar -xzf M2Crypto-0.21.1.tar.gz
This should create the directory M2Crypto-0.21.1 which will contain the SWIG directory
In SWIG you'll find _ssl.i, which is the file to be patched. In the same directory create a file called _ssl.i.patch for example using the nano editor and paste the complete diff code listed below into it.
Next issue the patch _ssl.i _ssl.i.patch command to merge the patch into the code. (Afterwards you may remove the patch file if you want.)
Finally issue the commands:
python setup.py build
followed by:
python setup.py install
to install manually.
diff code:
--- SWIG/_ssl.i 2011-01-15 20:10:06.000000000 +0100
+++ SWIG/_ssl.i 2012-06-17 17:39:05.292769292 +0200
## -48,8 +48,10 ##
%rename(ssl_get_alert_desc_v) SSL_alert_desc_string_long;
extern const char *SSL_alert_desc_string_long(int);
+#ifndef OPENSSL_NO_SSL2
%rename(sslv2_method) SSLv2_method;
extern SSL_METHOD *SSLv2_method(void);
+#endif
%rename(sslv3_method) SSLv3_method;
extern SSL_METHOD *SSLv3_method(void);
%rename(sslv23_method) SSLv23_method;

You can install this lib in your global environment and then just copy from your global site-packages to virtualenv.

M2Crypto 0.22.3 (the current version in pypi) fixes this problem, so the simplest solution is now simply:
pip install --upgrade M2Crypto
M2Crypto 0.22.3 has been released from martinpaljak's github repository, rather than from the original M2Crypto repository.

I had the same problem with the current release (M2Crypto-0.22.5). The latest RC build worked for me.
pip install M2Crypto==0.22.6rc4

There is a patch slated for 0.26.1.
In the meantime, you can clone the repo, apply the patch, and install from source.
git clone https://gitlab.com/m2crypto/m2crypto.git
(
cd m2crypto
git checkout 0.25.1
curl 'https://gitlab.com/m2crypto/m2crypto/merge_requests/117.diff' | git apply -v
)
pip install -U m2crypto