I want to call the Python functional eval from a C++11 code, using a function defined there as the argument of eval.
my_module.py
def eval(foo):
return foo(1)
main.cpp
#include <iostream>
#include <vector>
#include <pybind11/embed.h>
#include <pybind11/numpy.h>
namespace py = pybind11;
double foo (double x) { return x; };
auto foo_py = py::vectorize(foo);
int main()
{
py::scoped_interpreter guard{};
py::module m = py::module::import("my_module");
m.attr("eval")(foo_py);
}
The code compiles but I get the following error, which I do not understand:
terminate called after throwing an instance of 'pybind11::cast_error'
what(): Unable to convert call argument '0' of type 'detail::vectorize_helper (*)(double), double, double>' to Python object
I am writing a custom PAM module.
I have written a shared object (.so) file as required for Linux-PAM. What this .so file does, is call embedded Python to open up my facial recognition and , depending on the outcome, will return PAM_SUCCESS or PAM_AUTH_ERR
In the /etc/pam.d/sudo file I have told PAM that the file resides in (/home/berns/2020-ca326-cberns-fileencryption-with-opencv/PAM/pam_authnew.so). This is okay as , when sudo is typed, I can see my own personal error statement being printed saying it could not load the Python file.
The issue I have is, the facial recognition code resides in a total different directory from where the .so file resides. (../code/facial). I have used a chdir command in my C code, but it does'nt seem to change the directory back to where the Python facial files reside to provide success or not.
Is there something I'm missing ?
C code given below:
#define PAM_SM_AUTH
#define PAM_SM_ACCOUNT
#define PAM_SM_SESSION
#define GetCurrentDir getcwd
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <security/pam_appl.h>
#include <security/pam_modules.h>
#include </usr/include/python3.6m/Python.h>
int main(int argc, char** argv){
}
PAM_EXTERN int pam_sm_authenticate( pam_handle_t *pamh, int flags,int argc, const char **argv ){
char *result;
chdir("../code/facial"); // this changes it to the correct directory to execute
//dlopen("/usr/lib/x86_64-linux-gnu/libpython3.6m.so",RTLD_LAZY | RTLD_GLOBAL);
Py_Initialize(); // Starts python interpreter
PyRun_SimpleString("import os\nimport sys\nsys.path.append(os.getcwd())"); // lets python know where we are
PyObject *mymod, *func1, *ret1;
mymod = PyImport_ImportModule("pam_detect"); // This is the .py
if (mymod != NULL){ // check if the file file was loaded
func1 = PyObject_GetAttrString(mymod, "detect"); // hel is the function name in the file you declared earlier
ret1 = PyObject_CallObject(func1, NULL); // Null because the function doesnt take an argument.
result = PyUnicode_AsUTF8(ret1);
//printf("%s\n", result);
if (strcmp(result, "success") == 0){
Py_Finalize();
return PAM_SUCCESS;
}
else{
Py_Finalize();
return PAM_AUTH_ERR;
}
}
else{
printf("Error: can't find file!\n");
}
Py_Finalize();
return 0;
}
Error response from the terminal below
Is this due to child processes not changing the directories outside of their own running space ?
Also worth noting if I compile the C code WITHOUT making it a .so , I can load up the Python files perfectly and recognise my face.
I've tried to make a C extension to Python. My problem is that I have C function calls inside the C function I have made a C extension for. For example I am using C functions in pmd.h and usb1024LS.h inside these C functions. When I try running my script, I get errors like "undefined symbol: hid_init". Where hid_init is a function.
I have tried running the program in a c main program, and it works.
How do I call C functions from inside other C functions which have an extension?
Thanks!
My code:
test.py - test script:
import ctypes
import myTest_1024LS
ctypes_findInterface = ctypes.CDLL('/home/oysmith/NetBeansProjects/MCCDAQ/usb1024LS_with_py/myTest_1024LS.so').findInterface
ctypes_findInterface.restype = ctypes.c_void_p
ctypes_findInterface.argtypes = [ctypes.c_void_p]
ctypes_findInterface()
setup.py:
from distutils.core import setup, Extension
setup(name="myTest_1024LS", version="0.0", ext_modules = [Extension("myTest_1024LS", ["myTest_1024LS.c"])])
myTest_1024LS.c:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <ctype.h>
#include <sys/types.h>
#include <asm/types.h>
#include <python2.7/Python.h>
#include "pmd.h"
#include "usb-1024LS.h"
#include "myTest_1024LS.h"
void findInterface(void){
int interface;
hid_return ret;
ret = hid_init();
if (ret != HID_RET_SUCCESS) {
fprintf(stderr, "hid_init failed with return code %d\n", ret);
exit(1);
}
if ((interface = PMD_Find_Interface(&hid, 0, USB1024LS_PID)) >= 0) {
printf("USB 1024LS Device is found! interface = %d\n", interface);
} else if ((interface = PMD_Find_Interface(&hid, 0, USB1024HLS_PID)) >= 0) {
printf("USB 1024HLS Device is found! interface = %d\n", interface);
} else {
fprintf(stderr, "USB 1024LS and USB 1024HLS not found.\n");
exit(1);
}
}
PyDoc_STRVAR(myTest_1024LS__doc__, "myTes_1024LS point evaluation kernel");
PyDoc_STRVAR(findInterface__doc__, "find device");
static PyObject *py_findInterface(PyObject *self, PyObject *args);
static PyMethodDef wrapper_methods[] = {
{"findInterface", py_findInterface, METH_VARARGS, findInterface__doc__},
{NULL, NULL}
};
PyMODINIT_FUNC initwrapper(void){
Py_InitModule3("wrapper", wrapper_methods, myTest_1024LS__doc__);
}
static PyObject *py_findInterface(PyObject *self, PyObject *args){
if(!PyArg_ParseTuple(args, "")){
return NULL;
}
findInterface();
return 0;
}
When building C extensions which themselves have to be linked against other shared libraries you'll have to tell which ones to link against in the setup.py. In this case at least the library which exports the hid_init() function. See the Python documentation for more details and examples: Building C and C++ Extensions with distutils. The second example contains arguments to link an extra library to the extension module.
The ctypes ”declarations” are wrong: void is not the same as a void pointer (void*). The findInterface() C function has neither arguments nor a return value, which is ”declared” as:
ctypes_findInterface.argtypes = []
ctypes_findInterface.restype = None
Hello and thanks for your help in advance !
I am writing a python wrapper (SWIG 2.0 + Python 2.7) for a C++ code. The C++ code has typedef which I need to access in python wrapper. Unfortunately, I am getting following error when executing my Python code:
tag = CNInt32(0)
NameError: global name 'CNInt32' is not defined
I looked into SWIG documentation section 5.3.5 which explains size_t as typedef but I could not get that working too.
Following is simpler code to reproduce the error:
C++ header:
#ifndef __EXAMPLE_H__
#define __EXAMPLE_H__
/* File: example.h */
#include <stdio.h>
#if defined(API_EXPORT)
#define APIEXPORT __declspec(dllexport)
#else
#define APIEXPORT __declspec(dllimport)
#endif
typedef int CNInt32;
class APIEXPORT ExampleClass {
public:
ExampleClass();
~ExampleClass();
void printFunction (int value);
void updateInt (CNInt32& var);
};
#endif //__EXAMPLE_H__
C++ Source:
/* File : example.cpp */
#include "example.h"
#include <iostream>
using namespace std;
/* I'm a file containing use of typedef variables */
ExampleClass::ExampleClass() {
}
ExampleClass::~ExampleClass() {
}
void ExampleClass::printFunction (int value) {
cout << "Value = "<< value << endl;
}
void ExampleClass::updateInt(CNInt32& var) {
var = 10;
}
Interface file:
/* File : example.i */
%module example
typedef int CNInt32;
%{
#include "example.h"
%}
%include <windows.i>
%include "example.h"
Python Code:
# file: runme.py
from example import *
# Try to set the values of some typedef variables
exampleObj = ExampleClass()
exampleObj.printFunction (20)
var = CNInt32(5)
exampleObj.updateInt (var)
Thanks again for your help.
Santosh
I got it working. I had to use typemaps in the interface file, see below:
- Thanks a lot to "David Froger" on Swig mailing lists.
- Also, thanks to doctorlove for initial hints.
%include typemaps.i
%apply CNInt32& INOUT { CNInt32& };
And then in python file:
var = 5 # Note: old code problematic line: var = CNInt32(5)
print "Python value = ",var
var = exampleObj.updateInt (var) # Note: 1. updated values returned automatically by wrapper function.
# 2. Multiple pass by reference also work.
# 3. It also works if your c++ function is returning some value.
print "Python Updated value var = ",var
Thanks again !
Santosh
I have been trying to wrap an existing C code into a Python module for some time now and I keep running into recurrent errors and a failed build... After an extensive look at the available documentation and posts on this forum I would like to know if someone has an idea why it is not working.
I would like to use the stsci.imagemanip.interp2d.expand2d() function which consists of a python script calling a C function which uses the Python.h library. This under Mac 10.8 64bits with Python 2.7.
I am new to wrapping C functions to Python and I decided to use SWIG and distutils.
My input files are the following:
bilinearinterp.i
/* bilinearinterp.i */
%module bilinearinterp
%{
#define SWIG_FILE_WITH_INIT
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include <stdio.h>
#include <Python.h>
#include <math.h>
#include <numpy/arrayobject.h>
#include "bilinearinterp.h"
%}
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
bilinearinterp.c
/* File: bilinearinterp.c */
#include <Python.h>
#include <stdio.h>
#include <math.h>
#include <numpy/arrayobject.h>
#include "bilinearinterp.h"
static void InterpInfo (float ai, int npts, int *i, float *p, float *q) {
/* arguments:
float ai i: independent variable in same units as i
int npts i: size of array within which *i is an index
int *i o: array index close to ai
float *p, *q o: weights for linear interpolation
*/
DO SOME CALCULATIONS
}
int unbin2d (float *a, float *b, int inx, int iny, int onx, int ony) {
/* arguments:
PyArrayObject *a i: input data
PyArrayObject *b o: output data
*/
DO SOME CALCULATIONS
CALL InterpInfo
return (1);
}
static PyObject * bilinearinterp(PyObject *obj, PyObject *args)
{
PyObject *input, *output;
PyArrayObject *dataa, *datab;
int inx, iny, onx, ony;
int status=0;
if (!PyArg_ParseTuple(args,"OO:bilinearinterp",&input,&output))
return NULL;
dataa = (PyArrayObject *)PyArray_ContiguousFromAny(input, PyArray_FLOAT, 1, 2);
datab = (PyArrayObject *)PyArray_ContiguousFromAny(output, PyArray_FLOAT, 1, 2);
inx = PyArray_DIM(input,0);
iny = PyArray_DIM(input,1);
onx = PyArray_DIM(output,0);
ony = PyArray_DIM(output,1);
status = unbin2d((float *)dataa->data,(float *)datab->data, inx, iny, onx, ony);
Py_XDECREF(dataa);
Py_XDECREF(datab);
return Py_BuildValue("i",status);
}
static PyMethodDef bilinearinterp_methods[] =
{
{"bilinearinterp", bilinearinterp, METH_VARARGS,
"bilinearinterp(input, output)"},
{0, 0} /* sentinel */
};
void initbilinearinterp(void) {
Py_InitModule("bilinearinterp", bilinearinterp_methods);
import_array();
}
bilinearinterp.h
/* File: bilinearinterp.H */
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
setup.py
#!/usr/bin/env python
from distutils.core import setup, Extension
bilinearinterp_module = Extension('_bilinearinterp',
sources = ['bilinearinterp_wrap.c','bilinearinterp.c'],
# Path to locate numpy/arrayobject.h
include_dirs=['/Library/Python/2.7/site-packages/numpy-override/numpy/core/include'])
setup (name = 'bilinearinterp',
version = '1.1',
author = "Space Telescope Science Institute - stsci_python",
description = """bilinear interpretation for 2D array extrapolation""",
ext_modules = [bilinearinterp_module],
py_modules = ["bilinearinterp"],
)
Then I run the following in Terminal:
>>>swig -python bilinearinterp.i
>>>python setup.py build_ext --inplace
The appropriate files are being created:
_bilinearinterp.so bilinearinterp.py bilinearinterp.pyc bilinearinterp_wrap.c
As well as a build directory containing some files in it.
I get warnings when setup.py is running but it seems to complete.
Over the various tests I did two recurrent errors keep coming back:
warning: "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
Which may induces several other warnings associated to numpy
warning: function 'bilinearinterp' has internal linkage but is not defined [-Wundefined-internal] static PyObject * bilinearinterp(PyObject *obj, PyObject *args); bilinearinterp_wrap.c:2971:24: note: used here result = (PyObject *)bilinearinterp(arg1,arg2);
Then when calling the module in a python script I get the following:
>>> from bilinearinterp import bilinearinterp as lininterp
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "bilinearinterp.py", line 28, in <module>
_bilinearinterp = swig_import_helper()
File "bilinearinterp.py", line 24, in swig_import_helper
_mod = imp.load_module('_bilinearinterp', fp, pathname, description)
ImportError: dlopen(./_bilinearinterp.so, 2): Symbol not found: _bilinearinterp
Referenced from: ./_bilinearinterp.so
Expected in: flat namespace
in ./_bilinearinterp.so
Would someone have an idea from where the error could be coming from?
Is it from my usage of SWIG or from setup.py?
Another approach with only a swig interface file and the C code (without including the *.h file) does not work. Based on the swig introduction chapter
bilinearinterp.i
/* bilinearinterp.i */
%module bilinearinterp
%{
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
%}
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
And I compile using:
>>>swig -python -o bilinearinterp_wrap.c bilinearinterp.i
>>>gcc -c -fpic bilinearinterp.c bilinearinterp_wrap.c
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-I/System//Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include
bilinearinterp_wrap.c:2955:19: warning: function 'bilinearinterp' has internal
linkage but is not defined [-Wundefined-internal]
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
^
bilinearinterp_wrap.c:2971:24: note: used here
result = (PyObject *)bilinearinterp(arg1,arg2);
^
1 warning generated.
>>>gcc -bundle -flat_namespace -undefined suppress bilinearinterp.o bilinearinterp_wrap.o -o _bilinearinterp.so
Then I still get the same import error in the python interpreter.
Could it be due to my numpy installation or my gcc arguments?
Thanks
Well, I don't quite understand what you are trying to achieve, but I see some bad practices. For example, you make bilinearinterp everywhere static, and static functions are not exported and only can be used in the same module where implemented. I think it would be helpful to read a bit about SWIG design ideas. For sure it will take a some time, but you will get a broader perspective of what SWIG is and isn't capable of.