Libpython error while building YouCompleteMe - python

I am building YouCompleteMe plugin of vim, following this document. When I run make I get the following error.
Linking CXX shared library /home/sagar/.vim/bundle/YouCompleteMe/python/ycm_core.so
/usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython2.7.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
What is this error?
I have installed pyenv to manage python versions. Is it causing problem?

Make the linker point to the .so (shared object) file and not the .a (static lib) file.
You can do this specifying the flag when running cmake:
cmake -G "Unix Makefiles" -DPYTHON_LIBRARY=/usr/local/lib/libpython2.7.so . ~/.vim/bundle/YouCompleteme/cpp
Do mind that even though you're using pyenv, YouCompleteMe build may point to an undesired
python build as they are not correctly auto-detected right now.
If you're having this problem, you should probably also specify the Python header files correctly:
cmake -G "Unix Makefiles" -DPYTHON_LIBRARY=/usr/local/lib/libpython2.7.so -DPYTHON_INCLUDE_DIR=/usr/local/include/python . ~/.vim/bundle/YouCompleteme/cpp
PS=(I'm assuming your headers are in that path, do check before)

Since some paths were different on my system from the accepted answer (both the CMake and the python lib ones) I'm posting an alternate solution for the above problem:
Make sure to have a shared library version of libpython2.7.so
$ locate libpython
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1
Either create a symlink to it from where CMake expects it to be
sudo ln -s "/usr/lib/x86_64-linux-gnu/libpython2.7.so.1" "/usr/lib/libpython2.7.so"
or alternatively, as written in YCM's build script code, you could add additional CMake options to ensure the .so library is properly found
export EXTRA_CMAKE_ARGS="-DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython2.7.so.1"

Related

CMake - Installing issues to make package

For installing dlib, I followed this tutorial : http://www.pyimagesearch.com/2017/03/27/how-to-install-dlib/.
I am on Mac OS X 10.12.5 and using Python 3.5.
I run
$ brew install cmake
$ brew install boost
$ brew install boost-python --with-python3
It works without any error.
But when I try to install dlib with pip install dlib. I have an error :
The C compiler
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc"
is not able to compile a simple test program.
error: cmake configuration failed
ld: can't map file, errno=22 file '/usr/local/opt/qt/lib' for architecture x86_64
For the full error, please see on this link (doesn't want to paste the full error) :
https://gist.github.com/alexattia/3e98685310d90b65031db640d3ea716a
After retracing the error, when I tried to make dlib manually, I have this :
Linking C executable cmTC_05e45
/usr/local/Cellar/cmake/3.8.2/bin/cmake -E cmake_link_script
CMakeFiles/cmTC_05e45.dir/link.txt --verbose=1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-Wl,-search_paths_first -Wl,-headerpad_max_install_names
/usr/local/opt/qt/lib CMakeFiles/cmTC_05e45.dir/testCCompiler.c.o -o
cmTC_05e45
For the full trace expand : https://gist.github.com/alexattia/1e54ffb87c9eb4c811033f5cadd90331
I reinstalled XCode (from Apple Store) and CMake (3.8.2 from the downloaded page), I even installed Qt Creator to have a clean version of Qt, but I still have the same error.
I tried to install it with conda but after the installation, I still don't have the module in python.
Thank you very much for any help.
You commented:
Indeed, in my .bash_profile, I have export LDFLAGS="/usr/local/opt/qt/lib",
export CPPFLAGS="/usr/local/opt/qt/include", export PATH="/usr/local/opt/qt/bin:$PATH".
But even while commenting it, I still have the same error
Neither of your assignments to LDFLAGS or CPPFLAGS makes sense, and the
first one is the cause the linker failure that concerns you.
The value of the environment variable LDFLAGS, if set, is interpreted by your build system
as linkage options. Likewise The value of the environment variable
CPPFLAGS, if set, is interpreted as preprocessor options.
/usr/local/opt/qt/lib is not a linkage option and /usr/local/opt/qt/include
is not a preprocessor option. These are simply directory names. Any argument that
you pass to the linker (or preprocessor, or compiler) that is not an option is
interpeted by the tool as an input file. Thus you have led the linker to believe
that /usr/local/opt/qt/lib is an input file to your linkage.
ld: can't map file, errno=22 file '/usr/local/opt/qt/lib' for architecture x86_64
is what the linker says when it discovers that /usr/local/opt/qt/lib is not
a file at all.
Presumably, you wish to instruct the linker that /usr/local/opt/qt/lib is
a directory in which it should search for libraries required by your linkage.
The linkage option that expresses that intent is:
-L/usr/local/opt/qt/lib
Here are the GCC options for linking
Similarly you intend to instruct that preprocessor that /usr/local/opt/qt/include
is a directory in which it should search for header files. The preprocessor
option to express that is:
-I/usr/local/opt/qt/include
Here are the GCC options for preprocessing
It is abnormal and inadvisable to specify compilation or linkage options
in your bash login profile, as you are doing. Specify such options in the
build system's input files (makefile, cmakelists file or similar), or as arguments to
the build system's configuration. But if you insist on specifying them in
your bash login profile, then you should specify:
LDFLAGS=-L/usr/local/opt/qt/lib
CPPFLAGS=-I/usr/local/opt/qt/include
And once you have made these environment settings in your bash_profile they will
only take effect in new login shells.
I had a similar issue but found out it was due to boost.
Try this.
brew uninstall boost-python
brew uninstall boost
brew install boost-python --with-python3 --without-python
pip3 install dlib

cross compiling gdb for arm with python failed

I want to debug ARM application on devices like Android machine, i prefer to use gdb(ARM version) than gdb with gdbserver to debug, because there is a dashboard , a visual interface for GDB in Python.
It must cooperation with gdb(ARM version) on devices,so i need to cross compiling a ARM version of gdb with python, the command used shows below:
./configure --host=arm-linux-gnueabi --target=arm-linux-gnueabi --with-python=/usr/bin
But finally a error message appeared:
configure:8096: checking whether to use python
configure:8098: result: /usr/bin/
configure:8316: checking for python2.7
configure:8334: arm-linux-gnueabi-gcc -o conftest -g -O2 -I/usr/include/python2.7 -I/usr/include/python2.7 conftest.c -ldl -ltermcap -lm -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions >&5
In file included from /usr/include/python2.7/Python.h:8:0,
from conftest.c:50:
/usr/include/python2.7/pyconfig.h:15:52: fatal error: arm-linux-gnueabi/python2.7/pyconfig.h: No such file or directory
compilation terminated.
Then I find the line 15 in /usr/include/python2.7/pyconfig.h, as below:
# elif defined(__ARM_EABI__) && !defined(__ARM_PCS_VFP)
#include <arm-linux-gnueabi/python2.7/pyconfig.h>
Here is the point, I only have x86_64-linux-gnu/python2.7/pyconfig.h in /usr/include, how can I get the arm-linux-gnueabi/python2.7/pyconfig.h? I already apt-get install python2.7-dev.
I ran into this problem when attempting to cross-compile a python wrapper module built with SWIG, but it looks to me like it will happen to anyone cross compiling python-linked C code on a Debian system.
Apparently the Debian python-dev packages are not set up with the header files to facilitate a cross compile, but it is possible to go get them manually. I'm not sure if this is a python bug or a Debian package bug, and I haven't researched if it applies to other distros.
pyconfig.h sets preprocessor defines to tell the python source code about platform-depended things like endianness and data type sizes, so the proper pyconfig.h is definitely needed to correctly compile python source. Fortunately, the pyconfig.h file should be the only file you need to grab separately, and it is available from the Debian python-dev package for your target platform.
you can download the package file for armeabi or any other architecture from https://packages.debian.org/jessie/libpython2.7-dev and extract the include directory yourself, or you can use the following commands to download the package and copy the proper files for armeabi to /usr/local/include
wget http://security.debian.org/debian-security/pool/updates/main/p/python2.7/libpython2.7-dev_2.7.9-2+deb8u2_armel.deb
dpkg -x libpython2.7-dev_2.7.9-2_armel.deb libpython2.7-dev_2.7.9-2_armel_extracted
sudo cp -r libpython2.7-dev_2.7.9-2_armel_extracted/usr/include/arm-linux-gnueabi/ /usr/local/include/
rm -r libpython2.7-dev_2.7.9-2_armel*
Note that on some cross compilers you will have to add -I /usr/local/include to the compiler options if it does not search this location by default, but to me this is better than modifying /usr/include and risking your changes being wiped out by the OS

Building graph-tool on OSX with Python 3.4

I tried to install graph-tool on Mac OSX 10.10 using homebrew. The brew build process works fine, but when I try to import graph-tool I get the error described in this question.
Another problem with homebrew is that I always builds graph-tool for python2.7 and it installs the packages in the Python 2.7 sit-packages folder. But I want to use it with Python 3.4. These are the reasons why I tried to build graph-tool from source.
The ./configure command automatically uses Python 2.7, too. So I passed it the desired Python version with ./configure PYTHON=python3.4
It then detects the correct version as well as the related paths but crash with the following error:
configure: error:
Could not link test program to Python. Maybe the
main Python library has been installed in some non-standard library
path. If so, pass it to configure, via the LDFLAGS environment
variable.
Example: ./configure LDFLAGS="-L/usr/non-standard-path/python/lib"
======================================================================
ERROR!
You probably have to install the development version of the
Python package for your distribution. The exact name of this package varies
among them.
======================================================================
The error occurs with and without PYTHON variable set.
From the output of ./configure I can see that everything works fine except for the last line, which says:
checking consistency of all components of python development
environment... no
Whats does the above line mean and how do I properly install graph-tool on my maschine?
The error message is explaining exactly what needs to be done. Since python was installed in a non-standard path, you need to pass the flag LDFLAGS="-L/usr/non-standard-path/python/lib" pointing to the directory where the python libraries are located. This is most likely "/usr/local/lib", if you are using homebrew.
I was getting this error when I was trying to install graph-tool using outdated an autoconf / automake / pkg-config combination (installed using yum in CentOS 5.10). Installing those packages from source fixed the problem... although I'm not sure how this related to my python installation....
It worked for me by passing the variable PYTHON_EXTRA_LDFLAGS="-Wl,-stack_size,1000000 -F/usr/local/Cellar/python3/3.6.3/Frameworks -framework CoreFoundation".
In your case, it would be the path to the homebrew installation of python3.4.
The way I found out is that in the config.log, the error message shows the following:
configure:19023: checking python extra libraries
configure:19030: result: -ldl -framework CoreFoundation
configure:19037: checking python extra linking flags
configure:19044: result: -Wl,-stack_size,1000000 -framework CoreFoundation Python.framework/Versions/3.6/Python
configure:19051: checking consistency of all components of python development environment
configure:19079: gcc -o conftest -g -O2 -DNDEBUG -I/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/include/python3.6m -F/usr/local/Cellar/python3/3.6.3/Frameworks/ -Wl,-stack_size,1000000 -framework CoreFoundation Python.framework/Versions/3.6/Python conftest.c -L/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib -lpython3.6m -ldl -framework CoreFoundation -ldl -framework CoreFoundation >&5
clang: error: no such file or directory: 'Python.framework/Versions/3.6/Python'
The error seems to be path 'Python.framework/Versions/3.6/Python', that in a homebrew installation does not exist. I search for the same path in the config.log and I found this line:
PYTHON_EXTRA_LDFLAGS="-Wl,-stack_size,1000000 -framework CoreFoundation Python.framework/Versions/3.6/Python"
So, the solution for me was to pass this variable with the right path.

Cannot build mod_wsgi for python2.7 on Scientific Linux

I have a web app developed with web.py using python 2.7.5.
I need to deploy it on a server running Scientific Linux 6.5. It comes with python 2.6.6 and the system requires this version, so I had to install python 2.7.5 in parallel.
What I need to do is to install mod_wsgi that uses python 2.7.5. I couldn't find a pre-compiled package, so I attempted to build one from source. I downloaded the source of mod_wsgi-3.4.
I ran configure with:
./configure --with-python=/home/mapto/Python-2.7.5/python
and got response:
checking for apxs2... no
checking for apxs... /usr/sbin/apxs
checking Apache version... 2.2.15
configure: creating ./config.status
config.status: creating Makefile
When I run make it fails with:
/usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython2.7.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
apxs:Error: Command failed with rc=65536
.
make: * [mod_wsgi.la] Error 1
How can I sort out this problem? Would it help installing apxs2? If so, how can I do that? (apparently when I executed "yum install httpd-devel" it installed apxs, not apxs2)
This worked for me: apxs:Error: Command failed with rc=65536
I had to also do this, because of the parallel python versions: Python executable not finding libpython shared library

How do I build gevent in python under cygwin?

I want to use python consistently in a variety of environments. One of those environments is cygwin. One of the components I want to use is gevent (http://www.gevent.org/intro.html). Under cygwin, I have python 2.7 running (built locally, with the one line change described here, which is required for it to build: http://www.gossamer-threads.com/lists/python/python/976956).
gevent requires libevent (http://libevent.org/).
libevent seems to build just fine under cygwin (./configure && make && make install).
However, when building gevent (pip install gevent), it fails because libevent built static libraries (such as /usr/local/lib/libevent.a) and the gevent build wants shared libraries. Thus:
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.7.13-i686-2.7/gevent/core.o -L/usr/local/lib/python2.7/config -levent -lpython2.7 -o build/lib.cygwin-1.7.13-i686-2.7/gevent/core.dll
/usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: cannot find -levent
Meanwhile, if I try to get gevent to link statically (CFLAGS='-static' pip install gevent), -levent still fails, along with numerous warnings about how dereferencing type-punned pointer will break strict-aliasing rules and some additional ld failures:
gevent/core.c:21835: warning: dereferencing type-punned pointer will break strict-aliasing rules
....
gevent/core.c:21836: warning: dereferencing type-punned pointer will break strict-aliasing rules
gcc -shared -Wl,--enable-auto-image-base -static build/temp.cygwin-1.7.13-i686-2.7/gevent/core.o -L/usr/local/lib/python2.7/config -levent -lpython2.7 -o build/lib.cygwin-1.7.13-i686-2.7/gevent/core.dll
/usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: cannot find -levent
/usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython2.7
... so...
I think I need to tell libevent to build .dll instead of .a, but libevent's Makefile does not actually have a .a target, and it's not clear to me how the abstractions being used would have to change to accomplish this.
So, taking a step back: how do I install gevent under python 2.7 under cygwin?
Can you try a beta release from http://code.google.com/p/gevent/downloads/list ?
It does not require any external dependencies.

Categories