I have some C code which uses "mex.h". I am trying to create shared library (ie create dll file) so that it can be used in Python by using ctypes library.
Following is the command that I have tried for creating dll file, but I am getting an error.
Command Used
gcc -fPIC -IC:\Progra~1\MATLAB\R2020b\extern\include -IC:\Progra~1\MATLAB\R2020b\extern\lib -lC:\Progra~1\MATLAB\R2020b\bin\win64\libmat.dll -lC:\Progra~1\MATLAB\R2020b\bin\win64\libmx.dll -shared -o whitsmw_Cd1.dll whitsmw_Cd1.c
Error
c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/ld.exe: cannot find -lC:\Progra~1\MATLAB\R2020b\bin\win64\libmat.dll
c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/ld.exe: cannot find -lC:\Progra~1\MATLAB\R2020b\bin\win64\libmx.dll
collect2.exe: error: ld returned 1 exit status
I have checked the paths are correct. I would be helpful if someone can provide some help.
Related
I am cross compiling python 3.7 for ARM using TI-SDK.
I have downloaded the source code on ubuntu and followed the below instructions
Set the path of SDK (export PATH = $PATH:)
./confgure
make
When I run make, I get the errors below. Can you please tell me how to resolve the linker errors?
*ar rc libpython3.5m.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o
ar rc libpython3.5m.a Modules/_threadmodule.o Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/_stat.o Modules/timemodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/zipimport.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o
ranlib libpython3.5m.a
gcc -pthread -Xlinker -export-dynamic -o python Programs/python.o libpython3.5m.a -lpthread -ldl -lutil -lm
libpython3.5m.a(fileutils.o): In function `set_inheritable':
/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:892: undefined reference to `fcntl64'
/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:903: undefined reference to `fcntl64'
libpython3.5m.a(fileutils.o): In function `get_inheritable':
/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:777: undefined reference to `fcntl64'
/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:777: undefined reference to `fcntl64'
libpython3.5m.a(fileutils.o): In function `set_inheritable':
/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:892: undefined reference to `fcntl64'
libpython3.5m.a(fileutils.o):/home/pcadmin/work/OTIS/Python-3.5.9/Python/fileutils.c:903: more undefined references to `fcntl64' follow
collect2: error: ld returned 1 exit status
Makefile:555: recipe for target 'python' failed
make: *** [python] Error 1*
Please comment if more information is required to help me out with this issue.
I found the solution. Thought it may help others if i post it.
The linker error is due to the dependency on libxml2. I have cross compiled libxml2 and linked to python and it worked.
The configure script includes the path of libxml along with other options.
./configure CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar LD=arm-linux-gnueabihf-ld RANLIB=arm-linux-gnueabihf-ranlib --host=arm-linux-gnueabihf --target=arm -prefix=/home/sagar/otis/python3_install --without-sqlite3 --without-pdo-sqlite --without-pear --enable-simplexml --disable-mbregex --enable-sockets --enable-fpm --disable-opcache --enable-libxml --without-zlib --with-libxml-dir=/home/sagar/otis/libxml/_install/ --build=x86_64-linux-gnu --disable-all --disable-ipv6 ac_cv_have_long_long_format=yes ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no
I am trying to staticaly link the "c++ portaudio library" against my "C++ demo module" which is a python callable library (module).
I'm doing this with distutils, and in order to perform the static linking, I've added the libportaudio to the extra_objects argument, as follows:
module1 = Extension(
"demo",
sources=cppc,
# TODO remove os dependency
extra_compile_args=gccArgs,
# link against shared libraries
#libraries=[""]
# link against static libraries
extra_objects=["./clib-3rd-portaudio/libportaudio.a"]) # << I've added the static lib here
Compiling with "python setup.py build" results in the following linker error:
/usr/bin/ld: ./clib-3rd-portaudio/libportaudio.a(pa_front.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
./clib-3rd-portaudio/libportaudio.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
So at this point I've tried the obvious, I've added the -fPIC flagg to gccArgs (note extra_compile_args=gccArgs above), as follows:
gccArgs = [
"-Icsrc",
"-Icsrc/paExamples",
"-Icinc-3rd-portaudio",
"-Icinc-3rd-portaudio/common",
"-Icinc-3rd-portaudio/linux",
"-fPIC"] # << I've added the -fPIC flag here
However this results in the exact same error, so I guess the -fPIC flag is not the root cause. I'm probably missing something trivial, but I'm a bit lost here, hope somebody can help.
As the error message said, you should recompile the external library libportaudio.a with -fPIC argument, NOT your own codes. That's why it doesn't help to add -fPIC to your extra_compile_args.
Several other posts suggest that the file libportaudio.a cannot be used to build shared library, probably because the default build settings of portaudio don't include -fPIC.
To recompile portaudio correctly, download the source and try to run ./configure with -shared option (or something similar). If you cannot find the proper option, then modify the Makefile and append -fPIC to the extra compile options. You can also compile each object file manually and pack them into libportaudio.a.
Since your target file (libdemo.so) is a shared library, you must make sure ANY object codes included inside are compiled with -fPIC option. To understand why you need this option, please refer to:
What does -fPIC mean when building a shared library? and Position Independent Code (PIC) in shared libraries
this implementation of RFD http://www.cse.buffalo.edu/~jcorso/r/snippets.metric_learning.html
fails to build for me. Running the setup.py within the python package, when the following appears:
Building Swig Modules:
building librf...
/tmp/cctKDjwA.s: Assembler messages:
/tmp/cctKDjwA.s:12665: Error: no such instruction: `vfnmadd312ss 52(%r14),%xmm5,%xmm2'
/tmp/cctKDjwA.s:14338: Error: no such instruction: `vfnmadd312ss 84(%rdx),%xmm5,%xmm2'
/tmp/cctKDjwA.s:18244: Error: no such instruction: `vfnmadd312ss 228(%rsp),%xmm1,%xmm3'
/tmp/cctKDjwA.s:18389: Error: no such instruction: `vfmadd312ss 272(%rsp),%xmm1,%xmm0'
The line where it fails (checked separately):
os.system("g++ -march=native -fPIC -O3 -std=c++0x -c src/librf_wrap.cxx src/librf/*.cc src/librf/semaphores/*.cpp -I/usr/include/" + pyver)
I'm running on Ubuntu 12.04 64-bit with i5-4430. Apologies, but I'm unsure what additional info I should add, please suggest.
Thanks for your patience.
It's possible that '-march=native' is incorrectly determining your CPU and generating instructions that are illegal for it.
Could you try without '-march=native' to see if that is the case?
Note that it's possible to see what '-march=native' is choosing exactly, see this website.
I wrote a C++ "python plugin" for an a non-python C++ application.
At some point, this plugin, which is a .so, initializes the python interpreter and opens a python console.
For convenience the "readline" module is then imported and we get this error:
ImportError: /usr/lib/python2.7/lib-dynload/readline.so: undefined symbol: PyOS_InputHook
The link command (generated by cmake) goes:
/usr/bin/c++ -fPIC -Wall -Wextra -O3 -DNDEBUG -Xlinker -export-dynamic -Wl,-fwhole-program /usr/lib/libpython2.7.a -shared -Wl,-soname,libMyplugin.so -o libMyplugin.so [sources] [qt libs] -lGLU -lGL -lX11 -lXext -lc -lc -lpython2.7 -Wl,-rpath,/src:/usr/local/Trolltech/Qt-4.8.4/lib:
nm libMyplugin.so gives the following python-related symbols:
U Py_Finalize
U Py_Initialize
00000000002114a8 B PyOS_InputHook
U PyRun_InteractiveLoopFlags
U PyRun_SimpleStringFlags
We observe that PyOS_InputHook is defined in the BSS section of the plugin. Yet, python's readline.so fails to find it.
The question is why, and how to fix it.
The issue is with how the main application loads the plugin: it uses dlopen() without the flag RTLD_GLOBAL.
This implies that the symbols present in the plugin that are not currently needed (like PyOS_InputHook in this instance) are not resolved and will not be resolved for other shared libraries that will be loaded afterwards (like readline.so in this instance).
To fix this, the flag RTLD_GLOBAL should be used when loading the plugin.
If there is no control over the main application (as in this instance) and on how it uses dlopen(), it is still possible to "reload" the plugin from within the plugin itself using dlopen() with flags RTLD_NOLOAD | RTLD_GLOBAL, so as to resolve all previously unresolved symbols in the currently loaded library.
Doing this solves the issue.
I have written a Python C module (just ffmpeg.c which depends on some FFmpeg libs and other libs) and I am wondering how to link.
I'm compiling with:
cc -std=c99 -c ../ffmpeg.c -I /usr/include/python2.7 -g
I'm trying to link right now with:
ld -shared -o ../ffmpeg.so -L/usr/local/lib -lpython2.7 -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint ffmpeg.o -lc
There is no error. However, when I try to import ffmpeg in Python, I get:
ImportError: ./ffmpeg.so: undefined symbol: avio_alloc_context
Maybe this is already correct. I checked the resulting ffmpeg.so with ldd and it partly links to a wrong FFmpeg. This is strange however because of the -L/usr/local/lib which should take precedence over the default. Maybe because my custom installed FFmpeg (in /usr/local/lib) has for some reason only installed static *.a libs and *.so files have precedence over *.a files.
You should put the libraries that you're linking to after the .o file; i.e.:
ld -shared -o ../ffmpeg.so ffmpeg.o -L/usr/local/lib -lpython2.7 -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint -lc
The linker is dumb, and will not link in code from static libraries that it doesn't think are needed until a dependency arises i.e. the use of avio_alloc_context happens in ffmpeg.o, and because it's not listed after the use of the library, then the linker will not consider the code in the library as needed, so it doesn't get linked in - this is the biggest reason why linking using .a files fails.
You can also use --start-group and --end-group around all the files that you are linking - this allows you to link static libraries that have cross dependencies that just seem impossible to resolve through other means:
ld -shared -o ../ffmpeg.so -L/usr/local/lib -lpython2.7 --start-group -lavutil -lavformat -lavcodec -lswresample -lportaudio -lchromaprint ffmpeg.o --end-group -lc
using .a files is a little bit trickier than .so files, but these two items generally will work around any issues you have when linking.