I've a DLL that embeds Python interpreter using the C/Python API. The DLL works fine if called one time, but if the DLL is called twice, the code cracks and my program catch memory error.
The C code calling DLL is simple and the call to DLL function(Which calls Python interpreter) is done one time, if the second call(In the code) is not commented the code cracks and this happens only if "Numpy" is called in the Python code.
#include <stdio.h>
#include <conio.h>
#include <math.h>
#include <dll_simples.h>
int main() {
double in[] = { 4,2,5,4,2 };
double out[5] = {};
double a = 0;
double b = 0;
simuser(a,b,in,out);
//simuser(a, b, in, out); IF NOT COMMENTED -> ERROR
return 0;
}
I've commented Py_Finalize() in the DLL as suggested here and this post here affirms that from Py_Finalize() docs"Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize() and Py_Finalize() more than once." So i'd like to know why this happens and if any other thing could be done except calls "Py_Finalize()" only one time in the last call to DLL.
Listing [Python 3.Docs]: ctypes - A foreign function library for Python.
dll00.h:
#pragma once
#if defined(_WIN32)
# if defined DLL0_EXPORTS
# define DLL00_EXPORT_API __declspec(dllexport)
# else
# define DLL00_EXPORT_API __declspec(dllimport)
# endif
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut);
#if defined(__cplusplus)
}
#endif
dll00.c:
#include <stdio.h>
#include "Python.h"
#define DLL0_EXPORTS
#include "dll00.h"
#define C_TAG "From C .dll"
int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut) {
int res = 0;
printf("%s: in function\n", C_TAG);
const int isInit = Py_IsInitialized();
// Modify array calling Python functions
if (!isInit) {
printf("%s: initializing Python interpreter\n", C_TAG);
Py_Initialize();
}
res = PyRun_SimpleString("print(\"From Python (within C .dll): test\")");
for (unsigned int i = 0; i < size; i++) {
pOut[i] = pIn[i] * t + delta;
}
if (!isInit) {
printf("%s: uninitializing Python interpreter\n", C_TAG);
Py_Finalize();
}
return 0;
}
main00.c:
#include <stdio.h>
#include "dll00.h"
#define SIZE 4
int main() {
int res = 0;
double in[SIZE] = { 10.0, 11.0, 12.0, 13.0 };
double out[SIZE] = { 0 };
res = dll00Func00(2, 0.5, SIZE, in, out);
printf("Output array:\n");
for (unsigned int i = 0; i < SIZE; i++) {
printf("%.03f ", out[i]);
}
printf("\n");
return 0;
}
code00.py:
#!/usr/bin/env python
import sys
import ctypes as ct
DLL_NAME = "./dll00.dll"
def main(*argv):
DblPtr = ct.POINTER(ct.c_double)
size = 5
DblArr = ct.c_double * size
dll00 = ct.PyDLL(DLL_NAME)
dll00Func00 = dll00.dll00Func00
dll00Func00.argtypes = (ct.c_double, ct.c_double, ct.c_uint, DblPtr, DblPtr)
dll00Func00.restype = ct.c_int
in_arr = DblArr(*range(size))
out_arr = DblArr()
print("Output array:")
for i in range(size):
print("{:.3f}".format(out_arr[i]), end=" ")
print("\n")
res = dll00Func00(2, 2.5, size, in_arr, out_arr)
print("Output array:")
for i in range(size):
print("{:.3f}".format(out_arr[i]), end=" ")
print()
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main(*sys.argv[1:])
print("\nDone.")
Notes:
In Python, use ctypes.PyDLL as you're (indirectly) calling Python API functions
In the .dll, use [Python 3.Docs]: Initialization, Finalization, and Threads - int Py_IsInitialized()
As a side note, the if test is not needed in Py_Initialize's case as Py_Initialize simply returns if the interpreter is already initialized (so I left it there just for consistency), but it is needed for Py_Finalize as one wouldn't want to uninitialize the interpreter while still in Python. So Py_Initialize / Py_Finalize pair doesn't work on "reference count" (every Py_Initialize call requires an Py_Finalize one)
Calling Py_Initialize / Py_Finalize in the function, seems like an overkill (if the function is being called multiple times). I'd do 2 wrapper functions in the .dll and call:
one at the beginning
the other at the end
of the (C) program
Output:
e:\Work\Dev\StackOverflow\q059937552>sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2017\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.19
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
[prompt]> dir /b
code00.py
dll00.c
dll00.h
main00.c
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.07.06\include" dll00.c /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.07.06\libs"
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]> cl /nologo /MD /W0 main00.c /link /NOLOGO /OUT:main00_064.exe dll00.lib
main00.c
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.h
dll00.lib
dll00.obj
main00.c
main00.obj
main00_064.exe
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32
Output array:
0.000 0.000 0.000 0.000 0.000
From C .dll: in function
From Python (within C .dll): test
Output array:
2.500 4.500 6.500 8.500 10.500
Done.
[prompt]> set PATH=%PATH%;c:\Install\pc064\Python\Python\03.07.06
[prompt]> main00_064.exe
From C .dll: in function
From C .dll: initializing Python interpreter
From Python (within C .dll): test
From C .dll: uninitializing Python interpreter
Output array:
20.500 22.500 24.500 26.500
Related
I have a dll function in c++:
void get_DLLVersion(CAxClass* obj, char ** pVal);
In pVal get_DLLVersion write c string like "1.0.0.1"
In c++ its like:
char *strdll = (char*)malloc(50);
get_DLLVersion(tst, &strdll);
cout << "strdll = "<<strdll<<endl;
I need to use this function in python.
The main problem is how to create char** and put as 2nd argument of dll function.
I use next code:
import ctypes
libc = ctypes.CDLL("AxECR.so")
ecr = libc.create_object() #return CAxClass* obj
print (libc.get_DLLVersion)
libc.get_DLLVersion.argtypes = [c_void_p, ctypes.POINTER(ctypes.POINTER(c_char))]
dll = ctypes.POINTER(ctypes.POINTER(c_char))
libc.get_DLLVersion(ecr,dll) #don`t work Segmentation fault (core dumped)
Listing [Python.Docs]: ctypes - A foreign function library for Python.
Notes:
To fix this, a buffer (array) can be created via create_string_buffer, then its address passed (via byref) to the function
An explicit cast (from char array to char pointer) is required
For the 1st argument, I create singleton CAxClass object that is returned by every createObject call. I could also have the function creating the new instance, but another one would be then required to destroy it, in order to prevent memory leaks (1)
Looking at the way the function is called from C++, it just populates the memory at the address given as an argument (if not NULL, hopefully).
In this case, using a double pointer doesn't make much sense, as the same goal could be achieved using a simple one (I added another function in the example below to prove this)
Example:
dll00.cpp:
#include <cstring>
#include <iostream>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#define BUF_LEN 50
class CAxClass {};
static CAxClass *gObj = new CAxClass(); // nullptr;
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API void* createObject();
DLL00_EXPORT_API void dllVersion(CAxClass *pObj, char **ppVer);
DLL00_EXPORT_API void dllVersionSinglePtr(CAxClass *pObj, char *pVer);
#if defined(__cplusplus)
}
#endif
void* createObject() {
return gObj;
}
void dllVersion(CAxClass *pObj, char **ppVer)
{
if ((ppVer) && (*ppVer)) {
strncpy(*ppVer, "1.22.333.4444", BUF_LEN);
} else {
std::cout << "C - NULL pointer\n";
}
}
void dllVersionSinglePtr(CAxClass *pObj, char *pVer)
{
if (pVer) {
strncpy(pVer, "55555.666666.7777777.88888888", BUF_LEN);
} else {
std::cout << "C - NULL pointer\n";
}
}
code00.py:
#!/usr/bin/env python
import ctypes as cts
import sys
CharPtr = cts.c_char_p # More generic: cts.POINTER(cts.c_char) ?
CharPtrPtr = cts.POINTER(CharPtr)
BUF_LEN = 50
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
def main(*argv):
dll = cts.CDLL(DLL_NAME)
createObject = dll.createObject
createObject.argtypes = ()
createObject.restype = cts.c_void_p
dllVersion = dll.dllVersion
dllVersion.argtypes = (cts.c_void_p, CharPtrPtr)
dllVersion.restype = None
# #TODO - cfati: Testing purposes
dllVersionSinglePtr = dll.dllVersionSinglePtr
dllVersionSinglePtr.argtypes = (cts.c_void_p, CharPtr)
dllVersionSinglePtr.restype = None
obj = createObject()
print("Object: {:}".format(obj))
buf = cts.create_string_buffer(BUF_LEN)
dllVersion(obj, cts.byref(cts.cast(buf, CharPtr)))
print("Version: {:}".format(buf.value))
dllVersionSinglePtr(obj, cts.cast(buf, CharPtr))
print("Version: {:}".format(buf.value))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.\n")
sys.exit(rc)
output:
(qaic-env) [cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q075446745]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> ls
code00.py dll00.cpp
[064bit prompt]>
[064bit prompt]> g++ -fPIC -shared -o dll00.so dll00.cpp
[064bit prompt]>
[064bit prompt]> ls
code00.py dll00.cpp dll00.so
[064bit prompt]>
[064bit prompt]> python ./code00.py
Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] 064bit on linux
Object: 34716928
Version: b'1.22.333.4444'
Version: b'55555.666666.7777777.88888888'
Done.
Might also check:
[SO]: C++ & Python: Pass and return a 2D double pointer array from python to c++ (#CristiFati's answer) for the use of a double pointer when passing a 2D array
[SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) for a common pitfall when working with CTypes (calling functions)
[SO]: Python ctypes cdll.LoadLibrary, instantiate an object, execute its method, private variable address truncated (#CristiFati's answer) for an example of footnote #1
I just created a DLL in C with Python in it.
When I export my function with python functions in it, I can't call it in my Python code
But when I export a classic C function without Python code inside, it works perfectly. I don't understand
C DLL
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
__declspec(dllexport) PyObject* getList()
{
PyObject *PList = PyList_New(0);
PyList_Append(PList, Py_BuildValue("i", 1));
return PList;
}
Python Code
import ctypes
lib = ctypes.cdll.LoadLibrary("EasyProtect.dll")
getList = lib.getList
getList.argtypes = None
getList.restype = ctypes.py_object
print(getList())
My Error
print(getList())
OSError: exception: access violation reading 0x0000000000000010
According to [Python.Docs]: class ctypes.PyDLL(name, mode=DEFAULT_MODE, handle=None) (emphasis is mine):
Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.
Thus, this is only useful to call Python C api functions directly.
So, you should replace cdll (CDLL) by pydll (PyDLL). I enhanced your example a bit.
dll00.c:
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API PyObject* createList(unsigned int size, int *pValues);
#if defined(__cplusplus)
}
#endif
PyObject* createList(unsigned int size, int *pValues)
{
PyObject *pList = PyList_New(size);
for (unsigned int i = 0; i < size; ++i) {
PyList_SetItem(pList, i, Py_BuildValue("i", pValues[i]));
}
return pList;
}
code00.py:
#!/usr/bin/env python
import ctypes as ct
import sys
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
IntPtr = ct.POINTER(ct.c_int)
def main(*argv):
dll00 = ct.PyDLL(DLL_NAME)
create_list = dll00.createList
create_list.argtypes = (ct.c_uint, IntPtr)
create_list.restype = ct.py_object
dim = 5
int_arr = (ct.c_int * dim)(*range(dim, 0, -1)) # Intermediary step: create an array
res = create_list(dim, ct.cast(int_arr, IntPtr))
print("\n{:s} returned: {:}".format(create_list.__name__, res))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.")
sys.exit(rc)
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q072231434]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul
[prompt]> dir /b
code00.py
dll00.c
[prompt]>
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.09\include" dll00.c /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.09\libs"
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]>
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj
[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe" code00.py
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 064bit on win32
createList returned: [5, 4, 3, 2, 1]
Done.
When you use the Python C API, the GIL (global interpreter lock) must be held. Use PyDLL for that. Load your DLL with:
lib = ctypes.PyDLL("EasyProtect.dll")
As an aside, your DLL has a reference leak. Py_BuildValue returns a new object, and PyList_Append increments the reference when adding it to the list. Py_DECREF should be called on the object returned by Py_BuildValue.
In this case, though, create the list with the size you want and use PyList_SET_ITEM which steals the new object's reference to initialize the list and is optimized for initializing new list items:
#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>
__declspec(dllexport) PyObject* getList()
{
PyObject *PList = PyList_New(1); # size 1 list
PyList_SET_ITEM(PList, 0, PyLong_FromLong(1)); # assign offset 0
return PList;
}
I'm working on a program that supports Python extensions and noticed that it does not open if the user does not have Python in their machines or uses the x64 version instead of the x32 one. (I cannot change this last part as it does not depend on me).
So I've been reading about delay loading to later check if the library is available and did this:
// linker: /DELAYLOAD:python3.dll
#include <delayimp.h>
#include <Python.h>
#pragma comment(lib, "delayimp")
#pragma comment(lib, "python3")
...and everything worked fine until studio gave me this problem:
LNK1194 cannot delay-load 'python3.dll' due to import of data symbol
'__imp__PyType_Type'; link without /DELAYLOAD:python3.dll
So my question is: Is there a workaround for this issue?
I've been thinking of editing the includes and directly defining PyType_Type in my program from their GitHub page but I'm afraid of breaking something...
Thank you.
Here's a small demo (as I thought in one of my comments)
dll00.h:
#pragma once
#if defined(_WIN32)
# if defined DLL00_EXPORTS
# define DLL00_EXPORT_API __declspec(dllexport)
# else
# define DLL00_EXPORT_API __declspec(dllexport)
# endif
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API int dllPyFunc(const char *pyStr);
#if defined(__cplusplus)
}
#endif
dll00.c:
#define DLL00_EXPORTS
#include "dll00.h"
#include <Python.h>
#include <stdio.h>
int dllPyFunc(const char *pyStr)
{
if (!pyStr) {
printf("NULL PY test!\n");
return -1;
}
int res = 0;
if (!Py_IsInitialized())
Py_InitializeEx(0);
res = PyRun_SimpleString(pyStr);
res |= Py_FinalizeEx();
return res;
}
main00.c:
#if defined(DELAYLOAD)
# include "dll00.h"
# pragma comment(lib, "delayimp")
# pragma comment(lib, "dll00")
#endif
#include <Windows.h>
#include <stdio.h>
#if !defined(DELAYLOAD)
typedef int (*DllPyFuncFuncPtr)(const char *pyStr);
DllPyFuncFuncPtr dllPyFunc = NULL;
#endif
int main(int argc, char *argv[])
{
int ret = 0;
printf("Arg count: %d\n", argc);
if (argc == 1) {
printf("NO PYTHON WHATSOEVER!!!\n");
ret = 0;
} else {
printf("Attempt to run [%s] from Python\n", argv[1]);
#if !defined(DELAYLOAD)
HMODULE hDll00 = LoadLibrary("dll00.dll");
if (hDll00 == NULL) {
printf("Error loading dll: %d\n", GetLastError());
return -1;
}
dllPyFunc = (DllPyFuncFuncPtr)GetProcAddress(hDll00, "dllPyFunc");
if (dllPyFunc == NULL) {
printf("Error getting function: %d\n", GetLastError());
FreeLibrary(hDll00);
hDll00 = NULL;
return -2;
}
#endif
ret = dllPyFunc(argv[1]);
#if !defined(DELAYLOAD)
FreeLibrary(hDll00);
hDll00 = NULL;
#endif
}
printf("\nDone.\n");
return ret;
}
Output:
[cfati#CFATI-W10PC064:e:\Work\Dev\StackOverflow\q069418904]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\pc032\MS\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 >nul
[prompt]> dir /b
dll00.c
dll00.h
main00.c
[prompt]> :: Build .dll
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.08\include" dll00.c /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.08\libs"
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]> :: Build .exe (dynamic .dll load)
[prompt]> cl /nologo /MD /W0 main00.c /link /NOLOGO /OUT:main00_pc064.exe
main00.c
[prompt]> :: Build .exe (delayed .dll load)
[prompt]> cl /nologo /MD /W0 /DDELAYLOAD main00.c /link /NOLOGO /OUT:main00_dl_pc064.exe /DELAYLOAD:dll00.dll
main00.c
[prompt]> dir /b
dll00.c
dll00.dll
dll00.exp
dll00.h
dll00.lib
dll00.obj
main00.c
main00.obj
main00_dl_pc064.exe
main00_pc064.exe
[prompt]> :: Save current path (which doesn't have python38.dll's parent)
[prompt]> set _PATH=%PATH%
[prompt]> :: Add python38.dll's parent to PATH
[prompt]> set PATH=%_PATH%;c:\Install\pc064\Python\Python\03.08
[prompt]>
[prompt]> main00_pc064.exe
Arg count: 1
NO PYTHON WHATSOEVER!!!
Done.
[prompt]> main00_pc064.exe "import os;print(os.getcwd())"
Arg count: 2
Attempt to run [import os;print(os.getcwd())] from Python
e:\Work\Dev\StackOverflow\q069418904
Done.
[prompt]> main00_dl_pc064.exe
Arg count: 1
NO PYTHON WHATSOEVER!!!
Done.
[prompt]> main00_dl_pc064.exe "import os;print(os.getcwd())"
Arg count: 2
Attempt to run [import os;print(os.getcwd())] from Python
e:\Work\Dev\StackOverflow\q069418904
Done.
[prompt]> :: NO python38.dll
[prompt]> set PATH=%_PATH%
[prompt]> main00_pc064.exe
Arg count: 1
NO PYTHON WHATSOEVER!!!
Done.
[prompt]> main00_pc064.exe "import os;print(os.getcwd())"
Arg count: 2
Attempt to run [import os;print(os.getcwd())] from Python
Error loading dll: 126
[prompt]> main00_dl_pc064.exe
Arg count: 1
NO PYTHON WHATSOEVER!!!
Done.
[prompt]> main00_dl_pc064.exe "import os;print(os.getcwd())"
Arg count: 2
Attempt to run [import os;print(os.getcwd())] from Python
<<<<<<<< CRASH HERE >>>>>>>>
Notes:
Although this example is too trivial to evidentiate it, dynamic loading the library requires more code (on the .exe side)
On the other hand, delay load approach crashes if Python is required but not present in %PATH%
So I found a post by Raymond Chen about dll forwarding. It may be a solution so I will post it here for now and update later if it works.
Raymond Chen post: https://devblogs.microsoft.com/oldnewthing/20080204-00/?p=23593
Edit: It seems like a good option for other libs but not for Python. They are already doing this (the python3 dll is a forwarding to python3X) and the __imp__PyType_Type is still a problem.
In the end I just created a copy of the lib and put it inside a folder called PyLibrary that compiles with Python.
Edit2: In the end I may just take python off and into another module and just delayload that module.
Say you have the following C code:
typedef void (*PythonCallbackFunc)(void* userData);
void cb(PythonCallbackFunc pcf, void* userData)
{
pcf(userData);
}
and the following Python 3 code:
import ctypes
class PythonClass():
def foo():
print("bar")
CALLBACK_TYPE = ctypes.CFUNCTYPE(None, ctypes.c_void_p)
def callback(userData):
instanceOfPythonClass = ???(userData) # <-- this part right here
instanceOfPythonClass.foo()
lib = ctypes.cdll.LoadLibrary("path/to/lib.dll")
pc = PythonClass()
lib.cb(ctypes.byref(pc), CALLBACK_TYPE(callback))
Where "path/to/lib.dll" is a compiled binary of the C code up top.
How would one go about casting the userData parameter in "callback" back to an instance of PythonClass, so one could call the function "foo()"?
Based on [Python.Docs]: ctypes - A foreign function library for Python, I did some changes to your code in order to make it work.
dll00.c:
#include <stdio.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#define C_TAG "From C"
#define PRINT_MSG_0() printf("%s - [%s] (%d) - [%s]\n", C_TAG, __FILE__, __LINE__, __FUNCTION__)
typedef void (*PythonCallbackFuncPtr)(void* userData);
DLL00_EXPORT_API void callPython(PythonCallbackFuncPtr callbackFunc, void* userData)
{
PRINT_MSG_0();
callbackFunc(userData);
}
code00.py:
#!/usr/bin/env python
import ctypes as ct
import sys
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
CallbackFuncType = ct.CFUNCTYPE(None, ct.py_object)
class PythonClass():
def foo(self):
print("Dummy Python method")
def callback(userData):
print("From Python: {:s}".format(callback.__name__))
userData.foo()
def main(*argv):
dll = ct.CDLL(DLL_NAME)
callPython = dll.callPython
callPython.argtypes = [CallbackFuncType, ct.py_object]
callPython.rettype = None
instance = PythonClass()
callPython(CallbackFuncType(callback), instance)
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.")
sys.exit(rc)
Notes:
When dealing with Python types, use ctypes.py_object (which is a wrapper over PyObject) rather than ctypes.c_void_p
Always define argtypes (and restype) for C functions that you call from Python (e.g. call_python_func (which wraps callPython)). Check [SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) for more details
PythonClass.foo was missing the 1st (self) argument and thus being just a function defined inside PythonClass instead of a method
Did other non critical changes (mostly renames)
Output:
(py35x64_test) e:\Work\Dev\StackOverflow\q052053434>sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:\Install\pc032\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64
[prompt]> dir /b
code00.py
dll00.c
[prompt]> cl /nologo /DDLL dll00.c /link /DLL /OUT:dll00.dll
dll.c
Creating library dll00.lib and object dll00.exp
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj
[prompt]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] 064bit on win32
From C - [dll00.c] (18) - [callPython]
From Python: callback
Dummy Python method
I have structs in a C library that are like this.
The function pointers in DataFn point to static functions.
.h
struct Data {
int i;
int *array;
};
typedef struct {
bool (* const fn1) (struct Data*, const char *source);
....
} DataFn;
extern DataFn const DATAFUNC
Using objdump, the table only contains DATAFUNC and a few other things from gcc.
This is fine in C where calling fn1 would go like DATAFUNC.fn1(..., ...), but how would something like this be wrapped around so fn1 can be called in python w/ ctypes?
Example python
libc = ctypes.cdll.LoadLibrary("./data.so")
print(libc.DATAFUNC)
results in
<_FuncPtr object at 0x6ffffcd7430>
This is similar, but there isn't a factory function.
[Python.Docs]: ctypes - A foreign function library for Python contains everything required to solve this problem.
I believe that the main piece missing, was the in_dll method of a CTypes type (Accessing values exported from dll section).
Other than that, in order to work with C data, you need to let Python know of the data format. That applies to:
structs. Define Python counterparts by subclassing
ctypes.Structure
Function pointers (applies to your case). Define them using ctypes.CFUNCTYPE
I prepared a simplified example that illustrates the above. Note that I didn't do any error handling (checking for NULLs (which you should)), to keep things simple.
dll00.h:
struct Data {
int i;
};
typedef struct {
int (* const Func00Ptr) (struct Data*, const char*);
} DataFunc;
extern DataFunc const dataFunc;
dll00.c:
#include <stdio.h>
#include "dll00.h"
static int func00(struct Data *pData, const char *source)
{
printf("From C - Data.i: [%d], source: [%s]\n", pData->i, source);
return -255;
}
DataFunc const dataFunc = { &func00 };
code00.py:
#!/usr/bin/env python
import ctypes as ct
import sys
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
class Data(ct.Structure):
_fields_ = (
("i", ct.c_int),
)
Func00Type = ct.CFUNCTYPE(ct.c_int, ct.POINTER(Data), ct.c_char_p)
class DataFunc(ct.Structure):
_fields_ = (
("func00", Func00Type),
)
def main(*argv):
data = Data(127)
dll = ct.CDLL(DLL_NAME)
data_func = DataFunc.in_dll(dll, "dataFunc")
ret = data_func.func00(ct.byref(data), "abcd".encode())
print("Function returned: {:d}".format(ret))
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.")
sys.exit(rc)
Output:
[cfati#cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q049962265]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> ls
dll00.c dll00.h code00.py
[064bit prompt]> gcc -shared -fPIC -o dll00.so dll00.c
[064bit prompt]> ls
dll00.c dll00.h code.py dll00.so
[064bit prompt]> objdump -t dll00.so | grep dataFunc
0000000000200e10 g O .data.rel.ro 0000000000000008 dataFunc
[064bit prompt]> python code00.py
Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] 064bit on linux
From C - Data.i: [127], source: [abcd]
Function returned: -255
Done.