I've tried to make a C extension to Python. My problem is that I have C function calls inside the C function I have made a C extension for. For example I am using C functions in pmd.h and usb1024LS.h inside these C functions. When I try running my script, I get errors like "undefined symbol: hid_init". Where hid_init is a function.
I have tried running the program in a c main program, and it works.
How do I call C functions from inside other C functions which have an extension?
Thanks!
My code:
test.py - test script:
import ctypes
import myTest_1024LS
ctypes_findInterface = ctypes.CDLL('/home/oysmith/NetBeansProjects/MCCDAQ/usb1024LS_with_py/myTest_1024LS.so').findInterface
ctypes_findInterface.restype = ctypes.c_void_p
ctypes_findInterface.argtypes = [ctypes.c_void_p]
ctypes_findInterface()
setup.py:
from distutils.core import setup, Extension
setup(name="myTest_1024LS", version="0.0", ext_modules = [Extension("myTest_1024LS", ["myTest_1024LS.c"])])
myTest_1024LS.c:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <ctype.h>
#include <sys/types.h>
#include <asm/types.h>
#include <python2.7/Python.h>
#include "pmd.h"
#include "usb-1024LS.h"
#include "myTest_1024LS.h"
void findInterface(void){
int interface;
hid_return ret;
ret = hid_init();
if (ret != HID_RET_SUCCESS) {
fprintf(stderr, "hid_init failed with return code %d\n", ret);
exit(1);
}
if ((interface = PMD_Find_Interface(&hid, 0, USB1024LS_PID)) >= 0) {
printf("USB 1024LS Device is found! interface = %d\n", interface);
} else if ((interface = PMD_Find_Interface(&hid, 0, USB1024HLS_PID)) >= 0) {
printf("USB 1024HLS Device is found! interface = %d\n", interface);
} else {
fprintf(stderr, "USB 1024LS and USB 1024HLS not found.\n");
exit(1);
}
}
PyDoc_STRVAR(myTest_1024LS__doc__, "myTes_1024LS point evaluation kernel");
PyDoc_STRVAR(findInterface__doc__, "find device");
static PyObject *py_findInterface(PyObject *self, PyObject *args);
static PyMethodDef wrapper_methods[] = {
{"findInterface", py_findInterface, METH_VARARGS, findInterface__doc__},
{NULL, NULL}
};
PyMODINIT_FUNC initwrapper(void){
Py_InitModule3("wrapper", wrapper_methods, myTest_1024LS__doc__);
}
static PyObject *py_findInterface(PyObject *self, PyObject *args){
if(!PyArg_ParseTuple(args, "")){
return NULL;
}
findInterface();
return 0;
}
When building C extensions which themselves have to be linked against other shared libraries you'll have to tell which ones to link against in the setup.py. In this case at least the library which exports the hid_init() function. See the Python documentation for more details and examples: Building C and C++ Extensions with distutils. The second example contains arguments to link an extra library to the extension module.
The ctypes ”declarations” are wrong: void is not the same as a void pointer (void*). The findInterface() C function has neither arguments nor a return value, which is ”declared” as:
ctypes_findInterface.argtypes = []
ctypes_findInterface.restype = None
Related
I try to use a c++-class for socket communication in Python. Therefore, I created a class that uses nngpp. When importing the swigged file to python, I get the ImportError: undefined symbol: nng_msg_insert. The definition of the class is:
/* commclass.h */
#include <nngpp/nngpp.h>
#include <nngpp/protocol/req0.h>
#include <nngpp/protocol/rep0.h>
#include <nngpp/msg_body.h>
#include <nngpp/msg_header.h>
#include <nngpp/msg.h>
#include <nngpp/socket.h>
#include <nngpp/view.h>
#include <string>
#include <nlohmann/json.hpp>
//#include <thread>
#include <iostream>
#include <cstdio>
#include "/usr/local/include/nng/nng.h"
//#include <memory>
#include <chrono>
using json = nlohmann::json;
class CommIF
{
private:
nng::socket socket;
nng::msg message;
int msg_size;
public:
CommIF(const std::string option, std::string ipToListen, std::string ipToDial)
{
message = nng::make_msg(0);
if (option.compare("rep") == 0)
{
socket = std::move(nng::rep::v0::open());
}
else if (option.compare("req") == 0)
{
socket = std::move(nng::req::v0::open());
}
else
{
printf("EXCEPTION");
}
socket.listen(ipToListen.c_str());
bool connected = false;
while (connected == false)
{
try
{
socket.dial(ipToDial.c_str());
connected = true;
std::cout << "successfully connected\n";
}
catch (const nng::exception &e)
{
std::cerr << e.what() << "; retry in 1 s" << '\n';
//std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
msg_size = 0;
}
};
The interface file for swig is:
/* commclass.i */
%module commclass
%{
#include "src/commclass.h"
%}
%include "src/commclass.h"
I then start with the command python3 build_commclass.py build_ext --inplace the build process. The file build_commclass.py is the following
from distutils.core import setup, Extension
import os
name = "commclass"
version = "0.0.1"
os.environ["CC"] = "g++"
setup(name = name, version = version, ext_modules = [Extension(
name = '_commclass',
sources = ["commclass.i"],#"src/commclass.h"],
include_dirs = ['src'],#'/home/user1/Documents/extLibs','/usr/local/include'],
swig_opts = ["-c++", "-modern"]
)])
When I now import the the class to python, I get the error mentioned above. I searched a lot on google and stackoverflow and I am quite sure this is a linker issue. I also tried a lot of different things with the compiler and linker options of distutils.core, but I didn't find a solution.
Edit 1:
I now changed the interface file as following
/* commclass.i */
/* module*/
%module commclass
%{
#include "/usr/local/include/nng/nng.h"
#include "src/nngpp/nngpp.h"
#include "src/nngpp/protocol/req0.h"
#include "src/nngpp/protocol/rep0.h"
#include "src/nngpp/socket.h"
#include "src/nngpp/msg.h"
#include "src/nngpp/aio.h"
#include "src/nngpp/aio_view.h"
#include "src/nngpp/msg_body.h"
#include "src/nngpp/msg_header.h"
#include "src/commclass.h"
%}
%include "/usr/local/include/nng/nng.h"
%include "src/commclass.h"
I now get the same error when importing in python, but the undefined symbol changed. It is now nng_aio_set_iov. Both nng_msg_insert and nng_aio_set_iov are defined in the file nng.h which I have now included. I am confused now.
SWIG only generates interfaces for definitions that are directly specified by %include by default. nng_msg_insert is not defined in src/commclass.h. You need additional %include statements to bring in the definition.
Note you can change the default to recurse into all #include statements with the -includeall SWIG flag, but you generally don't want the entire <iostream>, <cstdio>, <string>, etc. interfaces to be wrapped and it likely won't work without a lot of additional effort.
I am trying to extend python with some c++ code. I have been sticking to the guide here
https://docs.python.org/2/extending/extending.html
I use the standard header spammodule.h
#ifndef Py_SPAMMODULE_H
#define Py_SPAMMODULE_H
#ifdef __cplusplus
extern "C" {
#endif
/* Header file for spammodule */
/* C API functions */
#define PySpam_System_NUM 0
#define PySpam_System_RETURN int
#define PySpam_System_PROTO (const char *command)
/* Total number of C API pointers */
#define PySpam_API_pointers 1
#ifdef SPAM_MODULE
/* This section is used when compiling spammodule.c */
static PySpam_System_RETURN PySpam_System PySpam_System_PROTO;
#else
/* This section is used in modules that use spammodule's API */
static void **PySpam_API;
#define PySpam_System \
(*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM])
/* Return -1 on error, 0 on success.
* PyCapsule_Import will set an exception if there's an error.
*/
static int
import_spam(void)
{
PySpam_API = (void **)PyCapsule_Import("spam._C_API", 0);
return (PySpam_API != NULL) ? 0 : -1;
}
#endif
#ifdef __cplusplus
}
#endif
#endif /* !defined(Py_SPAMMODULE_H) */
the actual function is put into a spam.cpp file
#define SPAM_MODULE
#include "spammodule.h"
#include <Python.h>
#include <iostream>
static PyObject * some_other_method(PyObject *self, PyObject *args);
static PyObject * spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
system(command);
sts = 2;
return Py_BuildValue("i", sts);
}
static int
PySpam_System(const char *command)
{
return system(command);
}
static PyMethodDef SpamMethods[] = {
{"system", spam_system, METH_VARARGS,
"Execute a shell command."},
{"some_other_method", some_other_method, METH_VARARGS, "do new stuff"},
{NULL, NULL, 0, NULL} /* Sentinel */
};
In a file method.cpp I put
#include <Python.h>
#include <iostream>
static PyObject * some_other_method(PyObject *self, PyObject *args){
std::cout<<"hello world\n"<<std::endl;
int sts = 2;
return Py_BuildValue("i", sts);
}
This compiles with a warning:
spam.cpp:7:19: warning: ‘PyObject* some_other_method(PyObject*, PyObject*)’ used but never defined [enabled by default]
static PyObject * some_other_method(PyObject *self, PyObject *args);
and fails when importing in a python script.
import spam
spam.system("ls -l")
spam.some_other_method()
However, if I supplant the forward declaration of some_other_method with the actual code
//static PyObject * some_other_method(PyObject *self, PyObject *args);
#include "method.cpp"
it works fine.
Why does it behave this way?
Is there an option to put the actual code of method functions into separate source files and compile those individually?
If your function is static it is only visible in the cpp file where it is defined. You cannot call it from other cpp files (unless you first somehow has gotten a pointer to the function).
You have to remove static from the functions that are to be visible outside their own source file.
I'm having problems getting the NumPy C API to properly initialize. I think I've isolated the problem to calling import_array from a different translation unit, but I don't know why this should matter.
Minimal working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
void* loc_wrap_import_array()
{
import_array();
return (void*) 1;
}
void loc_initialize()
{
loc_wrap_import_array();
}
int main()
{
Py_Initialize();
#ifdef USE_LOC_INIT
loc_initialize();
#else
initialize();
#endif
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
Compiler commands:
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m -DUSE_LOC_INIT
./segissue
# runs fine
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m
./segissue
# segfaults
I've tested this with Clang 3.6.0, GCC 4.9.2, Python 2.7, and Python 3.4 (with a suitably modified wrap_import_array because this is different between Python 2.x and 3.x). The various combinations all give the same result: if I don't call loc_initialize, the program will segfault in the PyArray_DescrFromType call. I have NumPy version 1.8.2. For reference, I'm running this in Ubuntu 15.04.
What baffles me most of all is this C++ NumPy wrapper appears to get away with calling import_array in a different translation unit.
What am I missing? Why must I call import_array from the same translation unit in order for it to actually take effect? More importantly, how do I get it to work when I call import_array from a different translation unit like the Boost.NumPy wrapper does?
After digging through the NumPy headers, I think I've found a solution:
in numpy/__multiarray_api.h, there's a section dealing with where an internal API buffer should be. For conciseness, here's the relevant snippet:
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif
#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)
extern void **PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void **PyArray_API;
#else
static void **PyArray_API=NULL;
#endif
#endif
It looks like this is intended to allow multiple modules define their own internal API buffer, in which each module must call their own import_array define.
A consistent way to get several translation units to use the same internal API buffer is in every module, define PY_ARRAY_UNIQUE_SYMBOL to some library unique name, then every translation unit other than the one where the import_array wrapper is defined defines NO_IMPORT or NO_IMPORT_ARRAY. Incidentally, there are similar macros for the ufunc features: PY_UFUNC_UNIQUE_SYMBOL, and NO_IMPORT/NO_IMPORT_UFUNC.
The modified working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#ifndef MYLIBRARY_USE_IMPORT
#define NO_IMPORT
#endif
#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API
#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#define MYLIBRARY_USE_IMPORT
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
int main()
{
Py_Initialize();
initialize();
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
I don't know what pitfalls there are with this hack or if there are any better alternatives, but this appears to at least compile and run without any segfaults.
I have write a python script that load a dll writed in c++ .
Script is :
from ctypes import *
mydll = cdll.LoadLibrary('C:\\Users\\gpiscite\\Documents\\Project\\DllXPython\\Debug\\DllXPython.dll')
mydll
mydll.fnDllXPython(956, c_char_p("c_char_p"), c_void_p(0), "std::string")
The c++ code is compiled in VisualStudio as a DLL and it is:
// DllXPython.cpp : Defines the exported functions for the DLL application.
//
#include "stdafx.h"
#include <cstdlib>
#include <string>
#include "DllXPython.h"
// This is an example of an exported variable
DLLXPYTHON_API int nDllXPython=0;
// This is an example of an exported function.
DLLXPYTHON_API int fnDllXPython(int aInt, char* apChar, void* apVoid, std::string aString)
//DLLXPYTHON_API PyObject * fnDllXPython(PyObject *self, PyObject *args)
{
return 0;
}
I had attached debug on python process and i saw thad the exception was raised on return 0 execution . The exception raised is:
Debug Assertion Failed! ..... _BLOCK_TYPE_IS_VALID(pHead->nBlockUse) ....
Any idea ? Thanks.
"std::string" is not a std::string. It is merely a char*
Hello and thanks for your help in advance !
I am writing a python wrapper (SWIG 2.0 + Python 2.7) for a C++ code. The C++ code has typedef which I need to access in python wrapper. Unfortunately, I am getting following error when executing my Python code:
tag = CNInt32(0)
NameError: global name 'CNInt32' is not defined
I looked into SWIG documentation section 5.3.5 which explains size_t as typedef but I could not get that working too.
Following is simpler code to reproduce the error:
C++ header:
#ifndef __EXAMPLE_H__
#define __EXAMPLE_H__
/* File: example.h */
#include <stdio.h>
#if defined(API_EXPORT)
#define APIEXPORT __declspec(dllexport)
#else
#define APIEXPORT __declspec(dllimport)
#endif
typedef int CNInt32;
class APIEXPORT ExampleClass {
public:
ExampleClass();
~ExampleClass();
void printFunction (int value);
void updateInt (CNInt32& var);
};
#endif //__EXAMPLE_H__
C++ Source:
/* File : example.cpp */
#include "example.h"
#include <iostream>
using namespace std;
/* I'm a file containing use of typedef variables */
ExampleClass::ExampleClass() {
}
ExampleClass::~ExampleClass() {
}
void ExampleClass::printFunction (int value) {
cout << "Value = "<< value << endl;
}
void ExampleClass::updateInt(CNInt32& var) {
var = 10;
}
Interface file:
/* File : example.i */
%module example
typedef int CNInt32;
%{
#include "example.h"
%}
%include <windows.i>
%include "example.h"
Python Code:
# file: runme.py
from example import *
# Try to set the values of some typedef variables
exampleObj = ExampleClass()
exampleObj.printFunction (20)
var = CNInt32(5)
exampleObj.updateInt (var)
Thanks again for your help.
Santosh
I got it working. I had to use typemaps in the interface file, see below:
- Thanks a lot to "David Froger" on Swig mailing lists.
- Also, thanks to doctorlove for initial hints.
%include typemaps.i
%apply CNInt32& INOUT { CNInt32& };
And then in python file:
var = 5 # Note: old code problematic line: var = CNInt32(5)
print "Python value = ",var
var = exampleObj.updateInt (var) # Note: 1. updated values returned automatically by wrapper function.
# 2. Multiple pass by reference also work.
# 3. It also works if your c++ function is returning some value.
print "Python Updated value var = ",var
Thanks again !
Santosh