I have simple python + cython project (hello world example from http://docs.cython.org/src/tutorial/cython_tutorial.html) on my ubuntu 16 x86_64. I can build this project with cython for x86_64.
How can I build the project for armv7 version of ubuntu 15 without using real armv7 board/cpu?
I have arm-linux-gnueabihf-gcc (http://packages.ubuntu.com/xenial/devel/gcc-arm-linux-gnueabihf) and it can compile simple C programs for armv7. How can I change settings of cython to use cross compiler for building shared objects for arm?
Architecture dependent libraries and headers files are needed for cross compiling.
When testing if python3.5-dev package and others could be installed after dpkg --add-architecture armhf and apt-get update (after some modification to sources.list), the result was basically.
python3.5-dev:armhf : Depends: python3.5:armhf (= 3.5.1-10) but it is not going to be installed
apt-get install python3.5:armhf is something that doesn't work, see
The existing proposals allow for the co-installation of libraries and
headers for different architectures, but not (yet) binaries.
One possible solution that does not require "full" virtual machine is provided by QEMU and chroot. A suitable directory for chroot can be created by debootstrap command. After creation, schroot can give access to that environment.
Substitute <DIRECTORY> and <USER> in the following commands:
apt-get install -y debootstrap qemu-user-static binfmt-support schroot
debootstrap --arch=armhf --foreign --include=gcc,g++,python3.5-dev xenial <DIRECTORY>
cp /usr/bin/qemu-arm-static <DIRECTORY>/usr/bin
chroot <DIRECTORY>
/debootstrap/debootstrap --second-stage
echo "deb http://ports.ubuntu.com/ubuntu-ports xenial universe" >> /etc/apt/sources.list
echo "deb http://ports.ubuntu.com/ubuntu-ports xenial multiverse" >> /etc/apt/sources.list
apt-get update
apt-get install -y cython cython3
exit
cat <<END > /etc/schroot/chroot.d/xenial-armhf
[xenial-armhf]
description=Ubuntu xenial armhf
type=directory
directory=/home/xenial-armhf
groups=sbuild,root
root-groups=sbuild,root
users=root,<USER>
END
The environment should be accessible by
schroot -c chroot:xenial-armhf
and for root user session (the user must be in a group listed in root-groups) ,
schroot -c chroot:xenial-armhf -u root
After this, it is also possible to cross compile a cython module:
hello.pyx:
print("hello world")
compiling (python3.5-config --cflags and python3.5-config --libs in chroot for options, note -fPIC):
cython hello.pyx
arm-linux-gnueabihf-gcc --sysroot <DIRECTORY> -I/usr/include/python3.5m -I/usr/include/python3.5m -Wno-unused-result -Wsign-compare -g -fstack-protector-strong -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -c hello.c
arm-linux-gnueabihf-gcc --shared --sysroot <DIRECTORY> -lpython3.5m -lpthread -ldl -lutil -lm hello.o -o hello.so
The module can be then tested
schroot -c chroot:xenial-armhf
python3
import hello
Cross compiling cython based python modules may also work. With setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import os
os.environ['CC'] = 'arm-linux-gnueabihf-gcc'
os.environ['LDSHARED'] = 'arm-linux-gnueabihf-gcc -shared'
sysroot_args=['--sysroot', '/path/to/xenial-armhf']
setup(cmdclass = {'build_ext': build_ext},
ext_modules= [ Extension("hello", ["hello.pyx"],
extra_compile_args=sysroot_args,
extra_link_args=sysroot_args) ])
Building a simple hello world module was possible this way. The file name for the module was wrong, in this case it was hello.cpython-35m-x86_64-linux-gnu.so. After renaming it as hello.so it was possible to import it.
Related
I'm trying to install Python 3.8.5 using pyenv on Centos 6 Docker image. I am getting the below error:
Downloading Python-3.6.12.tar.xz...
-> https://www.python.org/ftp/python/3.6.12/Python-3.6.12.tar.xz Installing Python-3.6.12...
BUILD FAILED (CentOS release 6.10 (Final) using python-build
1.2.22-47-g4c302a0)
Inspect or clean up the working tree at
/tmp/python-build.20210131203049.150 Results logged to
/tmp/python-build.20210131203049.150.log
Last 10 log lines: checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes checking if g++ supports
-c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking
dynamic linker characteristics... (cached) GNU/Linux ld.so checking
how to hardcode library paths into programs... immediate checking size
of size_t... 8 checking for C compiler vendor... gcc -pthread -shared
-L/root/.pyenv/versions/3.6.12/lib -L/root/.pyenv/versions/3.6.12/lib -L/root/.pyenv/versions/3.6.12/lib -I/root/.pyenv/versions/3.6.12/include build/temp.linux-x86_64-3.6/tmp/python-build.20210131203049.150/Python-3.6.12/Modules/pyexpat.o
build/temp.linux-x86_64-3.6/tmp/python-build.20210131203049.150/Python-3.6.12/Modules/expat/xmlparse.o
build/temp.linux-x86_64-3.6/tmp/python-build.20210131203049.150/Python-3.6.12/Modules/expat/xmlrole.o
build/temp.linux-x86_64-3.6/tmp/python-build.20210131203049.150/Python-3.6.12/Modules/expat/xmltok.o
-L/root/.pyenv/versions/3.6.12/lib -L/usr/local/lib -o build/lib.linux-x86_64-3.6/pyexpat.cpython-36m-x86_64-linux-gnu.so
/bin/sh: line 9: 8442 Killed CC='gcc -pthread'
LDSHARED='gcc -pthread -shared -L/root/.pyenv/versions/3.6.12/lib
-L/root/.pyenv/versions/3.6.12/lib ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py $quiet build make: *** [sharedmods] Error 137
Looks like you're missing a dep. Try this:
$ sudo yum install openssl-devel
If that's not enough, several other relevant deps are mentioned on https://joshspicer.com/python37-ssl-issue
Notice that merely having openssl installed
is not sufficient.
You need the -devel form of the package.
It contains essential ingredients, such as *.h header files,
that are necessary for the build to succeed.
Use $ yum list installed to see what you have ATM.
You might find e.g. yum list installed | grep -i openssl convenient.
I want to install the python library scikit-bio via pip using following command:
sudo pip install scikit-bio
on my system:
uname -a
Linux grassgis 3.2.0-69-generic-pae #103-Ubuntu SMP Tue Sep 2 05:15:53 UTC 2014 i686 i686 i386 GNU/Linux
However this causes an error:
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -c skbio/alignment/_ssw/_ssw_wrapper.c -o build/temp.linux-i686-2.7/skbio/alignment/_ssw/_ssw_wrapper.o
In file included from skbio/alignment/_ssw/ssw.h:17:0,
from skbio/alignment/_ssw/_ssw_wrapper.c:355:
/usr/lib/gcc/i686-linux-gnu/4.6/include/emmintrin.h:32:3: error: #error "SSE2 instruction set not enabled"
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/__multiarray_api.h:1532:1: warning: ‘_import_array’ defined but not used [-Wunused-function]
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/__ufunc_api.h:226:1: warning: ‘_import_umath’ defined but not used [-Wunused-function]
error: command 'gcc' failed with exit status 1
I ran already sudo apt-get update and sudo apt-get upgrade to get the most recent versions of installed software.
My GCC version is:
gcc --version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
How can I successfully install the scikit-bio packages for python?
This problem was previously reported by a user with an i686 machine on the scikit-bio issue tracker. The error occurs while compiling SSW, an external C program that is shipped with scikit-bio. The author of SSW recommended passing -msse2 to the compiler to fix the issue.
A fix was merged into the development branch of scikit-bio to include this flag for i686 machines.
If you are installing a release version of scikit-bio, you can specify this flag via CFLAGS on the command line:
CFLAGS=-msse2 pip install scikit-bio
or:
sudo CFLAGS=-msse2 pip install scikit-bio
Alternatively, scikit-bio's setup.py file can be modified to include '-msse2' in SSW's extra_compile_args.
I'm trying to compile a simple python/C example following this tutorial:
http://www.swig.org/tutorial.html
I'm on MacOS using Anaconda python.
however, when I run
gcc -c example.c example_wrap.c -I/Users/myuser/anaconda/include/
I get:
example_wrap.c:130:11: fatal error: 'Python.h' file not found
# include <Python.h>
^
It seems that this problem is reported in a number of questions:
Missing Python.h while trying to compile a C extension module
Missing Python.h and impossible to find
Python.h: No such file or directory
but none seem to provide an answer specific to Anaconda on MacOS
Anyone solved this?
Use the option -I/Users/myuser/anaconda/include/python2.7 in the gcc command. (That's assuming you are using python 2.7. Change the name to match the version of python that you are using.) You can use the command python-config --cflags to get the full set of recommended compilation flags:
$ python-config --cflags
-I/Users/myuser/anaconda/include/python2.7 -I/Users/myuser/anaconda/include/python2.7 -fno-strict-aliasing -I/Users/myuser/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
However, to build the extension module, I recommend using a simple setup script, such as the following setup.py, and let distutils figure out all the compiling and linking options for you.
# setup.py
from distutils.core import setup, Extension
example_module = Extension('_example', sources=['example_wrap.c', 'example.c'])
setup(name='example', ext_modules=[example_module], py_modules=["example"])
Then you can run:
$ swig -python example.i
$ python setup.py build_ext --inplace
(Take a look at the compiler commands that are echoed to the terminal when setup.py is run.)
distutils knows about SWIG, so instead of including example_wrap.c in the list of source files, you can include example.i, and swig will be run automatically by the setup script:
# setup.py
from distutils.core import setup, Extension
example_module = Extension('_example', sources=['example.c', 'example.i'])
setup(name='example', ext_modules=[example_module], py_modules=["example"])
With the above version of setup.py, you can build the extension module with the single command
$ python setup.py build_ext --inplace
Once you've built the extension module, you should be able to use it in python:
>>> import example
>>> example.fact(5)
120
If you'd rather not use the script setup.py, here's a set of commands that worked for me:
$ swig -python example.i
$ gcc -c -I/Users/myuser/anaconda/include/python2.7 example.c example_wrap.c
$ gcc -bundle -undefined dynamic_lookup -L/Users/myuser/anaconda/lib example.o example_wrap.o -o _example.so
Note: I'm using Mac OS X 10.9.4:
$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.3.0
Thread model: posix
I have Centos 6 on my server and I've been trying to install a few packages (modules) for it. I just brought the ones on my windows to the ftp client in /usr/lib/python2.6/site-packages, but when I run my script one of them - lxml gives me an error:
File "plugins/util/http.py", line 12, in <module>
from lxml import etree, html
File "/usr/lib/python2.6/site-packages/lxml/html/__init__.py", line 12, in <module>
ImportError: cannot import name etree
The same code worked perfectly on Windows 7 & Linux Ubuntu 10.04.
Does anyone know why it returns this error? I haven't modified anything, just moved the module from windows to my python2.6 directory on my vps.
#root:
>>> import lxml;print lxml
<module 'lxml' from '/usr/lib/python2.6/site-packages/lxml/__init__.pyc'>
>>>
#ig
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/libxml2 -I/tmp/pip-build/lxml/src/lxml/includes -I/usr/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.6/src/lxml/lxml.etree.o
unable to execute gcc: No such file or directory
error: command 'gcc' failed with exit status 1
Or install it from EPEL (the easiest way):
# install EPEL if you don't have it yet
sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
# instal python-lxml
sudo yum -y install python-lxml
lxml is not a pure Python module. It's mostly written in Cython and hence compiles to native code. The binary files from your Windows machine are incompatible with CentOS (and Linux in general).
Your best bet is to follow the instructions on installing lxml, i.e., install the libxslt-devel and libxml2-devel packages and use pip to compile lxml or compile it manually yourself. If you go for compiling it yourself, there's a previous question with some useful info and further info on the lxml site.
Installation
I don't have a CentOS machine to test this on. The easiest method to install would be via ip, which should be available as a package through Yum. If not, you can install it using the following commands (from the documentation for distribute, a pre-requisite for pip):
curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip
From there, just run:
pip install lxml
And it should install fine.
If you want to skip pip (though, ultimately, pip makes managing dependencies a lot easier, so its well worth doing)
Locate the appropriate version of lxml on PyPI
Download the source from the download URL (should be a tar.gz file)
Extract the tar archive using tar -xzf lxml-<version>.tar.gz
cd into the extracted directory and run python setup.py install
Note that any of the above commands may need to be run as root if your installation is only modifiable by root. Installation commands are python distribute_setup.py, easy_install pip and python setup.py install.
Trying to install Cython on a small VPS running Ubuntu Server. Did
sudo apt-get install gcc
and then
python setup.py install
In the Cython directory, but I get this peculiar error.
running install
running build
running build_py
running build_ext
building 'Cython.Plex.Scanners' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c Cython/Plex/Scanners.c -o build/temp.linux-x86_64-2.6/Cython/Plex/Scanners.o
Cython/Plex/Scanners.c:4:20: error: Python.h: No such file or directory
Cython/Plex/Scanners.c:6:6: error: #error Python headers needed to compile C extensions, please install development version of Python.
error: command 'gcc' failed with exit status 1
Why should I need a 'development version of Python'? Running Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41). Thanks!
You need the developer version of Python - i.e. the Python header files (Python.h)
sudo apt-get install python-dev
You need the headers included in the development version of Python, i.e. python.h. It should be packaged in Ubuntu as python-dev.