We Keep Coding

How to wrapping member function to global function in SWIG

I wonder how to wrapping member function to global function in SWIG.
Example,
test.cpp
class test
{
public:
void foo1();
void foo2();
}
After Wrapping,
test.py
class test:
def foo1():
...
def foo2():
...
how to write *.i file?

First, you need a working set of files. A class definition with no implementation won't work:
test.hpp
class test {
public:
int foo1();
int foo2();
};
test.cpp
#include "test.hpp"
int test::foo1() { return 1; }
int test::foo2() { return 2; }
Then for the .i file:
// Name of the module. For Python this must be the name of the final
// _test.pyd and test.py files after swig processing and compilation.
%module test
%{
// Code to be inserted in the generated wrapper code.
// The wrapper must be able to call the wrapped code so include the header.
#include "test.hpp"
%}
// This tells swig to wrap everything found in "test.cpp".
// Note it *does not* process #include in this file by default, just
// declaration found directly in the file.
%include "test.hpp"
Run SWIG to generate (in this case) test.py and test_wrap.cxx:
swig -python -c++ test.i
Compile the wrapper. I'm doing this directly with MSVC. The Python include and libs directory must be provided to build the Python extension, and the name of the final Python extension should be _.pyd.
cl /EHsc /LD /W3 /Ic:\python310\include /Fe_test.pyd test_wrap.cxx test.cpp /link /libpath:c:\python310\libs
Demo:
C:\>swig -python -c++ test.i
C:\>cl /nologo /EHsc /LD /W3 /Id:\dev\python310\include /Fe_test.pyd test_wrap.cxx test.cpp /link /libpath:d:\dev\python310\libs
test_wrap.cxx
test.cpp
Generating Code...
Creating library _test.lib and object _test.exp
C:\>py
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
>>> t = test.test()
>>> t.foo1()
1
>>> t.foo2()
2

Python in C DLL

I just created a DLL in C with Python in it.
When I export my function with python functions in it, I can't call it in my Python code
But when I export a classic C function without Python code inside, it works perfectly. I don't understand
C DLL
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
__declspec(dllexport) PyObject* getList()
{
PyObject *PList = PyList_New(0);
PyList_Append(PList, Py_BuildValue("i", 1));
return PList;
}
Python Code
import ctypes
lib = ctypes.cdll.LoadLibrary("EasyProtect.dll")
getList = lib.getList
getList.argtypes = None
getList.restype = ctypes.py_object
print(getList())
My Error
print(getList())
OSError: exception: access violation reading 0x0000000000000010

According to [Python.Docs]: class ctypes.PyDLL(name, mode=DEFAULT_MODE, handle=None) (emphasis is mine):
Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.
Thus, this is only useful to call Python C api functions directly.
So, you should replace cdll (CDLL) by pydll (PyDLL). I enhanced your example a bit.
dll00.c:
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API PyObject* createList(unsigned int size, int *pValues);
#if defined(__cplusplus)
}
#endif
PyObject* createList(unsigned int size, int *pValues)
{
PyObject *pList = PyList_New(size);
for (unsigned int i = 0; i < size; ++i) {
PyList_SetItem(pList, i, Py_BuildValue("i", pValues[i]));
}
return pList;
}
code00.py:
#!/usr/bin/env python
import ctypes as ct
import sys
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
IntPtr = ct.POINTER(ct.c_int)
def main(*argv):
dll00 = ct.PyDLL(DLL_NAME)
create_list = dll00.createList
create_list.argtypes = (ct.c_uint, IntPtr)
create_list.restype = ct.py_object
dim = 5
int_arr = (ct.c_int * dim)(*range(dim, 0, -1)) # Intermediary step: create an array
res = create_list(dim, ct.cast(int_arr, IntPtr))
print("\n{:s} returned: {:}".format(create_list.__name__, res))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.")
sys.exit(rc)
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q072231434]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul
[prompt]> dir /b
code00.py
dll00.c
[prompt]>
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.09\include" dll00.c /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.09\libs"
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]>
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj
[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe" code00.py
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 064bit on win32
createList returned: [5, 4, 3, 2, 1]
Done.

When you use the Python C API, the GIL (global interpreter lock) must be held. Use PyDLL for that. Load your DLL with:
lib = ctypes.PyDLL("EasyProtect.dll")
As an aside, your DLL has a reference leak. Py_BuildValue returns a new object, and PyList_Append increments the reference when adding it to the list. Py_DECREF should be called on the object returned by Py_BuildValue.
In this case, though, create the list with the size you want and use PyList_SET_ITEM which steals the new object's reference to initialize the list and is optimized for initializing new list items:
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
__declspec(dllexport) PyObject* getList()
{
PyObject *PList = PyList_New(1); # size 1 list
PyList_SET_ITEM(PList, 0, PyLong_FromLong(1)); # assign offset 0
return PList;
}

C DLL crack when called from Python

I've a DLL that embeds Python interpreter using the C/Python API. The DLL works fine if called one time, but if the DLL is called twice, the code cracks and my program catch memory error.
The C code calling DLL is simple and the call to DLL function(Which calls Python interpreter) is done one time, if the second call(In the code) is not commented the code cracks and this happens only if "Numpy" is called in the Python code.
#include <stdio.h>
#include <conio.h>
#include <math.h>
#include <dll_simples.h>
int main() {
double in[] = { 4,2,5,4,2 };
double out[5] = {};
double a = 0;
double b = 0;
simuser(a,b,in,out);
//simuser(a, b, in, out); IF NOT COMMENTED -> ERROR
return 0;
}
I've commented Py_Finalize() in the DLL as suggested here and this post here affirms that from Py_Finalize() docs"Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize() and Py_Finalize() more than once." So i'd like to know why this happens and if any other thing could be done except calls "Py_Finalize()" only one time in the last call to DLL.

Listing [Python 3.Docs]: ctypes - A foreign function library for Python.
dll00.h:
#pragma once
#if defined(_WIN32)
# if defined DLL0_EXPORTS
# define DLL00_EXPORT_API __declspec(dllexport)
# else
# define DLL00_EXPORT_API __declspec(dllimport)
# endif
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut);
#if defined(__cplusplus)
}
#endif
dll00.c:
#include <stdio.h>
#include "Python.h"
#define DLL0_EXPORTS
#include "dll00.h"
#define C_TAG "From C .dll"
int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut) {
int res = 0;
printf("%s: in function\n", C_TAG);
const int isInit = Py_IsInitialized();
// Modify array calling Python functions
if (!isInit) {
printf("%s: initializing Python interpreter\n", C_TAG);
Py_Initialize();
}
res = PyRun_SimpleString("print(\"From Python (within C .dll): test\")");
for (unsigned int i = 0; i < size; i++) {
pOut[i] = pIn[i] * t + delta;
}
if (!isInit) {
printf("%s: uninitializing Python interpreter\n", C_TAG);
Py_Finalize();
}
return 0;
}
main00.c:
#include <stdio.h>
#include "dll00.h"
#define SIZE 4
int main() {
int res = 0;
double in[SIZE] = { 10.0, 11.0, 12.0, 13.0 };
double out[SIZE] = { 0 };
res = dll00Func00(2, 0.5, SIZE, in, out);
printf("Output array:\n");
for (unsigned int i = 0; i < SIZE; i++) {
printf("%.03f ", out[i]);
}
printf("\n");
return 0;
}
code00.py:
#!/usr/bin/env python
import sys
import ctypes as ct
DLL_NAME = "./dll00.dll"
def main(*argv):
DblPtr = ct.POINTER(ct.c_double)
size = 5
DblArr = ct.c_double * size
dll00 = ct.PyDLL(DLL_NAME)
dll00Func00 = dll00.dll00Func00
dll00Func00.argtypes = (ct.c_double, ct.c_double, ct.c_uint, DblPtr, DblPtr)
dll00Func00.restype = ct.c_int
in_arr = DblArr(*range(size))
out_arr = DblArr()
print("Output array:")
for i in range(size):
print("{:.3f}".format(out_arr[i]), end=" ")
print("\n")
res = dll00Func00(2, 2.5, size, in_arr, out_arr)
print("Output array:")
for i in range(size):
print("{:.3f}".format(out_arr[i]), end=" ")
print()
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main(*sys.argv[1:])
print("\nDone.")
Notes:
In Python, use ctypes.PyDLL as you're (indirectly) calling Python API functions
In the .dll, use [Python 3.Docs]: Initialization, Finalization, and Threads - int Py_IsInitialized()
As a side note, the if test is not needed in Py_Initialize's case as Py_Initialize simply returns if the interpreter is already initialized (so I left it there just for consistency), but it is needed for Py_Finalize as one wouldn't want to uninitialize the interpreter while still in Python. So Py_Initialize / Py_Finalize pair doesn't work on "reference count" (every Py_Initialize call requires an Py_Finalize one)
Calling Py_Initialize / Py_Finalize in the function, seems like an overkill (if the function is being called multiple times). I'd do 2 wrapper functions in the .dll and call:
one at the beginning
the other at the end
of the (C) program
Output:
e:\Work\Dev\StackOverflow\q059937552>sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2017\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.19
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
[prompt]> dir /b
code00.py
dll00.c
dll00.h
main00.c
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.07.06\include" dll00.c /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.07.06\libs"
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]> cl /nologo /MD /W0 main00.c /link /NOLOGO /OUT:main00_064.exe dll00.lib
main00.c
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.h
dll00.lib
dll00.obj
main00.c
main00.obj
main00_064.exe
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Output array:
0.000 0.000 0.000 0.000 0.000
From C .dll: in function
From Python (within C .dll): test
Output array:
2.500 4.500 6.500 8.500 10.500
Done.
[prompt]> set PATH=%PATH%;c:\Install\pc064\Python\Python\03.07.06
[prompt]> main00_064.exe
From C .dll: in function
From C .dll: initializing Python interpreter
From Python (within C .dll): test
From C .dll: uninitializing Python interpreter
Output array:
20.500 22.500 24.500 26.500

Python C API: PyDateTime_FromTimestamp causes segmentation fault

I followed this answer to call PyDateTime_FromTimestamp to create a datetime object in C++. But I got a Segmentation fault when PyDateTime_FromTimestamp is called.
Here is my C++ code:
#include <python3.6/Python.h>
#include <stdio.h>
#include <python3.6/datetime.h>
#include <sys/time.h>
static PyObject *iGetDateTime_PyFn(PyObject *Py_UNUSED(self), PyObject *Py_UNUSED(args)) {
static double doubleValue = 1314761451;
PyObject *floatObj = NULL;
PyObject *timeTuple = NULL;
PyObject *dateTime = NULL;
floatObj = PyFloat_FromDouble(doubleValue);
timeTuple = Py_BuildValue("(O)", floatObj);
printf("timeTuple = %08x\n", (unsigned int)(long long)timeTuple);
printf("PyTuple_Check(timeTuple) = %d\n", PyTuple_Check(timeTuple));
dateTime = PyDateTime_FromTimestamp(timeTuple);
printf("ready to return\n");
return dateTime;
}
static PyMethodDef all_methods[] = {
{ "get_datetime", iGetDateTime_PyFn, METH_VARARGS, NULL },
{ NULL, NULL, 0, NULL }
};
static struct PyModuleDef main_module = {
PyModuleDef_HEAD_INIT,
"cpp",
NULL,
-1,
all_methods
};
PyMODINIT_FUNC PyInit_cpp(void) {
return PyModule_Create(&main_module);
}
I compiled with this command:
g++ --shared -fPIC -o cpp.so t1.cpp
My g++ version is 7.3.0.
In python, I execute:
import cpp
print(cpp.get_datetime())
And I get the following printed:
timeTuple = a7934358
PyTuple_Check(timeTuple) = 1
Segmentation fault (core dumped)
As we can see the timeTuple is successfully constructed and it is checked as a tuple. But we cannot get to the return sentence.

According to [GitHub]: python/cpython - (3.6) cpython/Include/datetime.h (${PYTHON_SRC_DIR}/Include/datetime.h):
PyDateTime_FromTimestamp is a preprocessor macro:
#define PyDateTime_FromTimestamp(args) \
PyDateTimeAPI->DateTime_FromTimestamp( \
(PyObject*) (PyDateTimeAPI->DateTimeType), args, NULL)
PyDateTimeAPI is initialized to NULL (earlier in file)
static PyDateTime_CAPI *PyDateTimeAPI = NULL;
resulting in segfault (Access Violation) when invoking the macro.
the fix requires initializing PyDateTimeAPI via the PyDateTime_IMPORT macro.
#define PyDateTime_IMPORT \
PyDateTimeAPI = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0)
Initially, I discovered this while browsing the code (and I did it in the getDateTimePyFn function), then I came across [Python 3.Docs]: DateTime Objects (emphasis is mine)
Before using any of these functions, the header file datetime.h must be included in your source (note that this is not included by Python.h), and the macro PyDateTime_IMPORT must be invoked, usually as part of the module initialization function.
I modified your code, and I will exemplify on Win (as it'e easier for me, and the behavior is reproducible).
cpp.c:
#include <stdio.h>
#include <Python.h>
#include <datetime.h>
#define MOD_NAME "cpp"
static double doubleValue = 1314761451;
static PyObject *getDateTimePyFn(PyObject *Py_UNUSED(self), PyObject *Py_UNUSED(args)) {
PyObject *floatObj = NULL,
*timeTuple = NULL,
*dateTime = NULL;
floatObj = PyFloat_FromDouble(doubleValue);
if (!floatObj)
{
return NULL;
}
timeTuple = Py_BuildValue("(O)", floatObj);
Py_XDECREF(floatObj);
if (!timeTuple)
{
return NULL;
}
dateTime = PyDateTime_FromTimestamp(timeTuple);
Py_XDECREF(timeTuple);
return dateTime;
}
static PyMethodDef all_methods[] = {
{ "get_datetime", getDateTimePyFn, METH_VARARGS, NULL },
{ NULL, NULL, 0, NULL }
};
static struct PyModuleDef main_module = {
PyModuleDef_HEAD_INIT,
MOD_NAME,
NULL,
-1,
all_methods
};
PyMODINIT_FUNC PyInit_cpp(void) {
PyDateTime_IMPORT; // #TODO - cfati: !!! This initializes the struct containing the function pointer !!!
return PyModule_Create(&main_module);
}
code.py:
#!/usr/bin/env python3
import sys
import cpp
def main():
print("cpp.get_datetime returned: {:}".format(cpp.get_datetime()))
if __name__ == "__main__":
print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
main()
print("Done.")
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q055903897]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***
[prompt]> "c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64
[prompt]> dir /b
code.py
cpp.c
[prompt]> cl /nologo /DDLL /MD /I"c:\Install\x64\Python\Python\03.06.08\include" cpp.c /link /NOLOGO /DLL /LIBPATH:"c:\Install\x64\Python\Python\03.06.08\libs" /OUT:cpp.pyd
cpp.c
Creating library cpp.lib and object cpp.exp
[prompt]> dir /b
code.py
cpp.c
cpp.exp
cpp.lib
cpp.obj
cpp.pyd
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
cpp.get_datetime returned: 2011-08-31 06:30:51
Done.

How to cast a ctypes pointer to an instance of a Python class

Say you have the following C code:
typedef void (*PythonCallbackFunc)(void* userData);
void cb(PythonCallbackFunc pcf, void* userData)
{
pcf(userData);
}
and the following Python 3 code:
import ctypes
class PythonClass():
def foo():
print("bar")
CALLBACK_TYPE = ctypes.CFUNCTYPE(None, ctypes.c_void_p)
def callback(userData):
instanceOfPythonClass = ???(userData) # <-- this part right here
instanceOfPythonClass.foo()
lib = ctypes.cdll.LoadLibrary("path/to/lib.dll")
pc = PythonClass()
lib.cb(ctypes.byref(pc), CALLBACK_TYPE(callback))
Where "path/to/lib.dll" is a compiled binary of the C code up top.
How would one go about casting the userData parameter in "callback" back to an instance of PythonClass, so one could call the function "foo()"?

Based on [Python.Docs]: ctypes - A foreign function library for Python, I did some changes to your code in order to make it work.
dll00.c:
#include <stdio.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#define C_TAG "From C"
#define PRINT_MSG_0() printf("%s - [%s] (%d) - [%s]\n", C_TAG, __FILE__, __LINE__, __FUNCTION__)
typedef void (*PythonCallbackFuncPtr)(void* userData);
DLL00_EXPORT_API void callPython(PythonCallbackFuncPtr callbackFunc, void* userData)
{
PRINT_MSG_0();
callbackFunc(userData);
}
code00.py:
#!/usr/bin/env python
import ctypes as ct
import sys
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
CallbackFuncType = ct.CFUNCTYPE(None, ct.py_object)
class PythonClass():
def foo(self):
print("Dummy Python method")
def callback(userData):
print("From Python: {:s}".format(callback.__name__))
userData.foo()
def main(*argv):
dll = ct.CDLL(DLL_NAME)
callPython = dll.callPython
callPython.argtypes = [CallbackFuncType, ct.py_object]
callPython.rettype = None
instance = PythonClass()
callPython(CallbackFuncType(callback), instance)
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.")
sys.exit(rc)
Notes:
When dealing with Python types, use ctypes.py_object (which is a wrapper over PyObject) rather than ctypes.c_void_p
Always define argtypes (and restype) for C functions that you call from Python (e.g. call_python_func (which wraps callPython)). Check [SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) for more details
PythonClass.foo was missing the 1st (self) argument and thus being just a function defined inside PythonClass instead of a method
Did other non critical changes (mostly renames)
Output:
(py35x64_test) e:\Work\Dev\StackOverflow\q052053434>sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\pc032\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64
[prompt]> dir /b
code00.py
dll00.c
[prompt]> cl /nologo /DDLL dll00.c /link /DLL /OUT:dll00.dll
dll.c
Creating library dll00.lib and object dll00.exp
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj
[prompt]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] 064bit on win32
From C - [dll00.c] (18) - [callPython]
From Python: callback
Dummy Python method