Ubuntu 20.04 on WSL 2.0 under windows 10 ARM64 (Surface Pro X with WSL2.0).
pip3 install jupyterlab // failure
Building wheels for collected packages: argon2-cffi
Building wheel for argon2-cffi (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3.8 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp5k32hshp
cwd: /tmp/pip-install-jo5pljfy/argon2-cffi_f65af512bd344c9b956794545e86c839
Complete output (39 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.8
creating build/lib.linux-aarch64-3.8/argon2
copying src/argon2/__main__.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/exceptions.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/_password_hasher.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/_ffi_build.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/low_level.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/_utils.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/_legacy.py -> build/lib.linux-aarch64-3.8/argon2
copying src/argon2/__init__.py -> build/lib.linux-aarch64-3.8/argon2
running build_clib
building 'argon2' library
creating build/temp.linux-aarch64-3.8
creating build/temp.linux-aarch64-3.8/extras
creating build/temp.linux-aarch64-3.8/extras/libargon2
creating build/temp.linux-aarch64-3.8/extras/libargon2/src
creating build/temp.linux-aarch64-3.8/extras/libargon2/src/blake2
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/argon2.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/argon2.o
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/blake2/blake2b.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/blake2/blake2b.o
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/core.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/core.o
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/encoding.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/encoding.o
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/ref.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/ref.o
aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/thread.c -o build/temp.linux-aarch64-3.8/extras/libargon2/src/thread.o
aarch64-linux-gnu-gcc-ar rcs build/temp.linux-aarch64-3.8/libargon2.a build/temp.linux-aarch64-3.8/extras/libargon2/src/argon2.o build/temp.linux-aarch64-3.8/extras/libargon2/src/blake2/blake2b.o build/temp.linux-aarch64-3.8/extras/libargon2/src/core.o build/temp.linux-aarch64-3.8/extras/libargon2/src/encoding.o build/temp.linux-aar aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iextras/libargon2/include -I/usr/include/python3.8 -c build/temp.linux-aarch64-3.8/_ffi.c -o build/temp.linux-aarch64-3.8/build/temp.linux-aarch64-3.8/_ffi.o
build/temp.linux-aarch64-3.8/_ffi.c:50:14: fatal error: pyconfig.h: No such file or directory
50 | # include <pyconfig.h>
| ^~~~~~~~~~~~
compilation terminated.
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for argon2-cffi
Failed to build argon2-cffi
ERROR: Could not build wheels for argon2-cffi which use PEP 517 and cannot be installed directly
How can I build argon2-cffi successfully?
I solved it following these steps:
sudo apt install -y build-essential python3.8-dev // ubuntu 20.04
pip3 install pep517 // or python3.8 -m pip install pep517
pip3 install jupyterlab // successfully
Related
I'm trying to install pydoop on my ubuntu but ended with the following errors.
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pydoop
Running setup.py clean for pydoop
Failed to build pydoop
Installing collected packages: pydoop
Running setup.py install for pydoop ... error
error: subprocess-exited-with-error
× Running setup.py install for pydoop did not run successfully.
│ exit code: 1
╰─> [79 lines of output]
using setuptools version 65.6.3
running install
/home/manoj/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/config.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/hadoop_utils.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/hadut.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/test_support.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/test_utils.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/avrolib.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/jc.py -> build/lib.linux-aarch64-cpython-310/pydoop
copying pydoop/version.py -> build/lib.linux-aarch64-cpython-310/pydoop
creating build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
copying pydoop/mapreduce/binary_protocol.py -> build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
copying pydoop/mapreduce/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
copying pydoop/mapreduce/pipes.py -> build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
copying pydoop/mapreduce/api.py -> build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
copying pydoop/mapreduce/connections.py -> build/lib.linux-aarch64-cpython-310/pydoop/mapreduce
creating build/lib.linux-aarch64-cpython-310/pydoop/hdfs
copying pydoop/hdfs/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs
copying pydoop/hdfs/file.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs
copying pydoop/hdfs/fs.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs
copying pydoop/hdfs/path.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs
copying pydoop/hdfs/common.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs
creating build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/argparse_types.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/submit.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/main.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/script.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
copying pydoop/app/script_template.py -> build/lib.linux-aarch64-cpython-310/pydoop/app
creating build/lib.linux-aarch64-cpython-310/pydoop/utils
copying pydoop/utils/misc.py -> build/lib.linux-aarch64-cpython-310/pydoop/utils
copying pydoop/utils/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop/utils
copying pydoop/utils/conversion_tables.py -> build/lib.linux-aarch64-cpython-310/pydoop/utils
copying pydoop/utils/py3compat.py -> build/lib.linux-aarch64-cpython-310/pydoop/utils
copying pydoop/utils/jvm.py -> build/lib.linux-aarch64-cpython-310/pydoop/utils
creating build/lib.linux-aarch64-cpython-310/pydoop/hdfs/core
copying pydoop/hdfs/core/__init__.py -> build/lib.linux-aarch64-cpython-310/pydoop/hdfs/core
copying pydoop/pydoop.properties -> build/lib.linux-aarch64-cpython-310/pydoop
running build_ext
checking for TLS support
creating /tmp/pydoop_6tyh2mvj/tmp
creating /tmp/pydoop_6tyh2mvj/tmp/pydoop_6tyh2mvj
aarch64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.10 -c /tmp/pydoop_6tyh2mvj/temp.c -o /tmp/pydoop_6tyh2mvj/tmp/pydoop_6tyh2mvj/temp.o
building 'pydoop.native_core_hdfs' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/src
creating build/temp.linux-aarch64-cpython-310/src/libhdfs
creating build/temp.linux-aarch64-cpython-310/src/libhdfs/common
creating build/temp.linux-aarch64-cpython-310/src/libhdfs/os
creating build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix
creating build/temp.linux-aarch64-cpython-310/src/native_core_hdfs
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/common/htable.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/common/htable.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/exception.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/exception.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/hdfs.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/hdfs.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/jni_helper.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/jni_helper.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/os/posix/mutexes.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/mutexes.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/os/posix/thread.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/thread.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/libhdfs/os/posix/thread_local_storage.c -o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/thread_local_storage.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/native_core_hdfs/hdfs_file.cc -o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_file.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/native_core_hdfs/hdfs_fs.cc -o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_fs.o -Wno-write-strings
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_BETTER_TLS -I/usr/lib/jvm/java-8-openjdk-arm64/include -Inative/jni_include -I/usr/lib/jvm/java-8-openjdk-arm64/lib -I/usr/lib/jvm/java-8-openjdk-arm64/include/linux -Isrc/libhdfs -Isrc/libhdfs/include -Isrc/libhdfs/os/posix -I/usr/include/python3.10 -c src/native_core_hdfs/hdfs_module.cc -o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_module.o -Wno-write-strings
aarch64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-aarch64-cpython-310/src/libhdfs/common/htable.o build/temp.linux-aarch64-cpython-310/src/libhdfs/exception.o build/temp.linux-aarch64-cpython-310/src/libhdfs/hdfs.o build/temp.linux-aarch64-cpython-310/src/libhdfs/jni_helper.o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/mutexes.o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/thread.o build/temp.linux-aarch64-cpython-310/src/libhdfs/os/posix/thread_local_storage.o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_file.o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_fs.o build/temp.linux-aarch64-cpython-310/src/native_core_hdfs/hdfs_module.o -L/usr/lib/jvm/java-8-openjdk-arm64/Libraries -L/usr/lib/jvm/java-8-openjdk-arm64/jre/lib/aarch64/server -L/usr/lib/aarch64-linux-gnu -ldl -ljvm -o build/lib.linux-aarch64-cpython-310/pydoop/native_core_hdfs.cpython-310-aarch64-linux-gnu.so -Wl,-rpath,/usr/lib/jvm/java-8-openjdk-arm64/jre/lib/aarch64/server
building 'pydoop.sercore' extension
creating build/temp.linux-aarch64-cpython-310/src/sercore
creating build/temp.linux-aarch64-cpython-310/src/sercore/HadoopUtils
aarch64-linux-gnu-gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/sercore/HadoopUtils -I/usr/include/python3.10 -c src/sercore/HadoopUtils/SerialUtils.cc -o build/temp.linux-aarch64-cpython-310/src/sercore/HadoopUtils/SerialUtils.o -Wno-write-strings -std=c++11 -O3
src/sercore/HadoopUtils/SerialUtils.cc:21:10: fatal error: rpc/types.h: No such file or directory
21 | #include <rpc/types.h>
| ^~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/aarch64-linux-gnu-gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> pydoop
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
I've tried upgrading and downgrading pip versions and I've tried everything that I could. I'm using python version 3.10.6 and pip version 22.3.1
When pip-installing a Python project with C++ parts,
pip3 install . --verbose --user
the typical compile line will be something like
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/eigen3/ -I/usr/local/include/python3.7 -I/home/nschloe/.local/include/python3.7m -I/usr/include/python3.7m -c src/generate.cpp -o build/temp.linux-x86_64-3.7/src/generate.o
Explicitly setting CFLAGS like
OPT="" CFLAGS="" pip3 install . --verbose --user
results in
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/eigen3/ -I/usr/local/include/python3.7 -I/home/nschloe/.local/include/python3.7m -I/usr/include/python3.7m -c src/generate.cpp -o build/temp.linux-x86_64-3.7/src/generate.o
retaining most of the flags.
Question: Where are all the compiler options set? More specifically: How can I remove -g? (It's eating too much memory.)
I was trying to build c through python distutils. I want to replace CC with gcc and follow this page
CC=gcc python setup.py build
Then I got
gcc -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c hello.c -o build/temp.linux-x86_64-2.7/hello.o
creating build/lib.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/hello.o -o build/lib.linux-x86_64-2.7/hello.so
CC only changed to gcc on first stage, and it would become original x86_64-linux-gnu-gcc, I am not sure is there anything missing, thanks.
You must set "CC" with
os.environ["CC"] = "GCC"
However, this sometimes does not work in windows and involves some changes in the config files.
Create file "C:\Python27\Lib\distutils\distutils.cfg" and write this inside
Then insert the following code:
[build]
compiler = gcc
Remove all instances of "-mno-cygwin" gcc option from file "C:\Python27\Lib\distutils\cygwinccompiler.py":
In the same file convert this:
self.set_executables(compiler='gcc -mno-cygwin -O -Wall',
compiler_so='gcc -mno-cygwin -mdll -O -Wall',
compiler_cxx='g++ -mno-cygwin -O -Wall',
linker_exe='gcc -mno-cygwin',
linker_so='%s -mno-cygwin %s %s'
% (self.linker_dll, shared_option,
entry_point))
to this:
self.set_executables(compiler='gcc -O -Wall',
compiler_so='gcc -mdll -O -Wall',
compiler_cxx='g++ -O -Wall',
linker_exe='gcc',
linker_so='%s %s %s'
% (self.linker_dll, shared_option,
entry_point))
I found the answer here, LDSHARED is the variable I need.
The full command would be like this
CC=gcc LDSHARED=gcc -pthread -shared python setup.py build
I am having a problem while trying to compile the shtns library with python support. I do:
./configure --prefix=$HOME/local --enable-python
make
and I get:
(...)
building '_shtns' extension
gcc -march=native -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-lMBuS3/python2.7-2.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/home/david/.local/lib/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c shtns_numpy_wrap.c -o build/temp.linux-x86_64-2.7/shtns_numpy_wrap.o -fopenmp
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-lMBuS3/python2.7-2.7.12=. -fstack-protector-strong -Wformat -Werror=format-security -L/home/david/local/lib -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-lMBuS3/python2.7-2.7.12=. -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/shtns_numpy_wrap.o sht_init.o sht_fly.o sht_omp.o -L/home/david/local/lib -lfftw3_omp -lfftw3 -lm -o build/lib.linux-x86_64-2.7/_shtns.so -L/home/david/local/lib
/usr/bin/ld: /home/david/local/lib/libfftw3.a(assert.o): relocation R_X86_64_PC32 against symbol `stdout##GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Makefile:107: recipe for target '_shtns.so' failed
make: *** [_shtns.so] Error 1
I think those are the relevant lines. It suggests to compile using the -fpic flag, but that is already in my CFLAGS environment variable, as you can see in the first gcc line. The problem is with the 'x86_64-linux-gnu-gcc' bit. Any suggestions?
I have CentOS 7 and I have installed mpicc (it works and compiles for openmpi in C). I also have python 2.7.5 and just installed pip. I'm running this command and get the following errors:
sudo pip install mpi4py
Collecting mpi4py Using cached mpi4py-2.0.0.tar.gz Installing
> collected packages: mpi4py Running setup.py install for mpi4py ...
> error
> Complete output from command /usr/bin/python2 -u -c "import setuptools,
> tokenize;__file__='/tmp/pip-build-x5jD4O/mpi4py/setup.py';exec(compile(getattr(tokenize,
> 'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
> 'exec'))" install --record /tmp/pip-mpMoZO-record/install-record.txt
> --single-version-externally-managed --compile:
> running install
> running build
> running build_src
> running build_py
> creating build
> creating build/lib.linux-x86_64-2.7
> creating build/lib.linux-x86_64-2.7/mpi4py
> copying src/__main__.py -> build/lib.linux-x86_64-2.7/mpi4py
> copying src/__init__.py -> build/lib.linux-x86_64-2.7/mpi4py
> creating build/lib.linux-x86_64-2.7/mpi4py/include
> creating build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/__init__.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/__init__.pyx -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-2.7/mpi4py/include/mpi4py
> copying src/MPI.pxd -> build/lib.linux-x86_64-2.7/mpi4py
> copying src/libmpi.pxd -> build/lib.linux-x86_64-2.7/mpi4py
> running build_clib
> MPI configuration: [mpi] from 'mpi.cfg'
> checking for library 'lmpe' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -llmpe -o _configtest
> /bin/ld: cannot find -llmpe
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> building 'mpe' dylib library
> creating build/temp.linux-x86_64-2.7
> creating build/temp.linux-x86_64-2.7/src
> creating build/temp.linux-x86_64-2.7/src/lib-pmpi
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o
> creating build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi
> gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/mpe.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libmpe.so
> checking for library 'vt-mpi' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt-mpi -o _configtest
> /bin/ld: cannot find -lvt-mpi
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> checking for library 'vt.mpi' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt.mpi -o _configtest
> /bin/ld: cannot find -lvt.mpi
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> building 'vt' dylib library
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o
> gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt.so
> checking for library 'vt-mpi' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt-mpi -o _configtest
> /bin/ld: cannot find -lvt-mpi
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> checking for library 'vt.mpi' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt.mpi -o _configtest
> /bin/ld: cannot find -lvt.mpi
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> building 'vt-mpi' dylib library
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o
> gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-mpi.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-mpi.so
> checking for library 'vt-hyb' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt-hyb -o _configtest
> /bin/ld: cannot find -lvt-hyb
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> checking for library 'vt.ompi' ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c _configtest.c -o _configtest.o
> gcc -pthread _configtest.o -lvt.ompi -o _configtest
> /bin/ld: cannot find -lvt.ompi
> collect2: error: ld returned 1 exit status
> failure.
> removing: _configtest.c _configtest.o
> building 'vt-hyb' dylib library
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o
> gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/src/lib-pmpi/vt-hyb.o -o
> build/lib.linux-x86_64-2.7/mpi4py/lib-pmpi/libvt-hyb.so
> running build_ext
> MPI configuration: [mpi] from 'mpi.cfg'
> checking for MPI compile and link ...
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _configtest.c -o _configtest.o
> _configtest.c:2:17: fatal error: mpi.h: No such file or directory
> #include <mpi.h>
> ^
> compilation terminated.
> failure.
> removing: _configtest.c _configtest.o
> error: Cannot compile MPI programs. Check your configuration!!!
I tried every solution I found so far and none seemed to work, has anyone any idea about this problem, please? Thank you
Run into the same issue and solved with:
yum install openmpi-devel
export CC=/usr/lib64/openmpi/bin/mpicc
pip install mpi4py
it seems you missed to install the development files of OpenMPI on Centos, the line that is the key here is:
_configtest.c:2:17: fatal error: mpi.h: No such file or directory
#include <mpi.h>
you should install the openmpi-devel (or equivalent) through yum and you should be good to reinstall the mpi4py module.
You could also try:
CentOS
yum install python-mpi4py
yum install python3-mpi4py
Debian
apt install python-mpi4py
apt install python3-mpi4py
For anyone else who runs into this, I'm posting this answer just in case you have a similar problem to me. The answer from #francis was the key. On my Centos7.7 system I was able to install mpi4py using the CC environment variable suggested by #iakov-nakhimovski but it was not working and I was getting runtime errors. The problem turned out to be that other software installed on the system had its own python installation in the path which was interfering with the setup of mpi4py. The solution was to remove this other installation from the path and to load the openmpi module which was shipped with the openmpi-devel package (I think).
module load mpi/openmpi-x86_64
Then install mpi4py using pip (note you do not need to explicitly define CC=mpicc since all these tools get put into the path by the environment module). When running a python script which requires mpi4py you also need to load this module.