How does module loading work in CPython under the hood? Especially, how does the dynamic loading of extensions written in C work? Where can I learn about this?
I find the source code itself rather overwhelming. I can see that trusty ol' dlopen() and friends is used on systems that support it but without any sense of the bigger picture it would take a long time to figure this out from the source code.
An enormous amount could be written on this topic but as far as I can tell, almost nothing has been — the abundance of webpages describing the Python language itself makes this difficult to search for. A great answer would provide a reasonably brief overview and references to resources where I can learn more.
I'm mostly concerned with how this works on Unix-like systems simply because that's what I know but I am interested in if the process is similar elsewhere.
To be more specific (but also risk assuming too much), how does CPython use the module methods table and initialization function to "make sense" of dynamically loaded C?
TLDR short version bolded.
References to the Python source code are based on version 2.7.6.
Python imports most extensions written in C through dynamic loading. Dynamic loading is an esoteric topic that isn't well documented but it's an absolute prerequisite. Before explaining how Python uses it, I must briefly explain what it is and why Python uses it.
Historically C extensions to Python were statically linked against the Python interpreter itself. This required Python users to recompile the interpreter every time they wanted to use a new module written in C. As you can imagine, and as Guido van Rossum describes, this became impractical as the community grew. Today, most Python users never compile the interpreter once. We simply "pip install module" and then "import module" even if that module contains compiled C code.
Linking is what allows us to make function calls across compiled units of code. Dynamic loading solves the problem of linking code when the decision for what to link is made at runtime. That is, it allows a running program to interface with the linker and tell the linker what it wants to link with. For the Python interpreter to import modules with C code, this is what's called for. Writing code that makes this decision at runtime is quite uncommon and most programmers would be surprised that it's possible. Simply put, a C function has an address, it expects you to put certain data in certain places, and it promises to have put certain data in certain places upon return. If you know the secret handshake, you can call it.
The challenge with dynamic loading is that it's incumbent upon the programmer to get the handshake right and there are no safety checks. At least, they're not provided for us. Normally, if we try to call a function name with an incorrect signature we get a compile or linker error. With dynamic loading we ask the linker for a function by name (a "symbol") at runtime. The linker can tell us if that name was found but it can’t tell us how to call that function. It just gives us an address - a void pointer. We can try to cast to a function pointer of some sort but it is solely up to the programmer to get the cast correct. If we get the function signature wrong in our cast, it’s too late for the compiler or linker to warn us. We’ll likely get a segfault after the program careens out of control and ends up accessing memory inappropriately. Programs using dynamic loading must rely on pre-arranged conventions and information gathered at runtime to make proper function calls. Here's a small example before we tackle the Python interpreter.
File 1: main.c
/* gcc-4.8 -o main main -ldl */
#include <dlfcn.h> /* key include, also in Python/dynload_shlib.c */
/* used for cast to pointer to function that takes no args and returns nothing */
typedef void (say_hi_type)(void);
int main(void) {
/* get a handle to the shared library dyload1.so */
void* handle1 = dlopen("./dyload1.so", RTLD_LAZY);
/* acquire function ptr through string with name, cast to function ptr */
say_hi_type* say_hi1_ptr = (say_hi_type*)dlsym(handle1, "say_hi1");
/* dereference pointer and call function */
(*say_hi1_ptr)();
return 0;
}
/* error checking normally follows both dlopen() and dlsym() */
File 2: dyload1.c
/* gcc-4.8 -o dyload1.so dyload1.c -shared -fpic */
/* compile as C, C++ does name mangling -- changes function names */
#include <stdio.h>
void say_hi1() {
puts("dy1: hi");
}
These files are compiled and linked separately but main.c knows to go looking for ./dyload1.so at runtime. The code in main assumes that dyload1.so will have a symbol "say_hi1". It gets a handle to dyload1.so's symbols with dlopen(), gets the address of a symbol using dlsym(), assumes it is a function that takes no arguments and returns nothing, and calls it. It has no way to know for sure what "say_hi1" is -- a prior agreement is all that keeps us from segfaulting.
What I've shown above is the dlopen() family of functions. Python is deployed on many platforms, not all of which provide dlopen() but most have similar dynamic loading mechanisms. Python achieves portable dynamic loading by wrapping the dynamic loading mechanisms of several operating systems in a common interface.
This comment in Python/importdl.c summarizes the strategy.
/* ./configure sets HAVE_DYNAMIC_LOADING if dynamic loading of modules is
supported on this platform. configure will then compile and link in one
of the dynload_*.c files, as appropriate. We will call a function in
those modules to get a function pointer to the module's init function.
*/
As referenced, in Python 2.7.6 we have these dynload*.c files:
Python/dynload_aix.c Python/dynload_beos.c Python/dynload_hpux.c
Python/dynload_os2.c Python/dynload_stub.c Python/dynload_atheos.c
Python/dynload_dl.c Python/dynload_next.c Python/dynload_shlib.c
Python/dynload_win.c
They each define a function with this signature:
dl_funcptr _PyImport_GetDynLoadFunc(const char *fqname, const char *shortname,
const char *pathname, FILE *fp)
These functions contain the different dynamic loading mechanisms for different operating systems. The mechanism for dynamic loading on Mac OS newer than 10.2 and most Unix(-like) systems is dlopen(), which is called in Python/dynload_shlib.c.
Skimming over dynload_win.c, the analagous function for Windows is LoadLibraryEx(). Its use looks very similar.
At the bottom of Python/dynload_shlib.c you can see the actual call to dlopen() and to dlsym().
handle = dlopen(pathname, dlopenflags);
/* error handling */
p = (dl_funcptr) dlsym(handle, funcname);
return p;
Right before this, Python composes the string with the function name it will look for. The module name is in the shortname variable.
PyOS_snprintf(funcname, sizeof(funcname),
LEAD_UNDERSCORE "init%.200s", shortname);
Python simply hopes there's a function called init{modulename} and asks the linker for it. Starting here, Python relies on a small set of conventions to make dynamic loading of C code possible and reliable.
Let's look at what C extensions must do to fulfill the contract that makes the above call to dlsym() work. For compiled C Python modules, the first convention that allows Python to access the compiled C code is the init{shared_library_filename}() function. For a module named spam compiled as shared library named “spam.so”, we might provide this initspam() function:
PyMODINIT_FUNC
initspam(void)
{
PyObject *m;
m = Py_InitModule("spam", SpamMethods);
if (m == NULL)
return;
}
If the name of the init function does not match the filename, the Python interpreter cannot know how to find it. For example, renaming spam.so to notspam.so and trying to import gives the following.
>>> import spam
ImportError: No module named spam
>>> import notspam
ImportError: dynamic module does not define init function (initnotspam)
If the naming convention is violated there's simply no telling if the shared library even contains an initialization function.
The second key convention is that once called, the init function is responsible for initializing itself by calling Py_InitModule. This call adds the module to a "dictionary"/hash table kept by the interpreter that maps module name to module data. It also registers the C functions in the method table. After calling Py_InitModule, modules may initialize themselves in other ways such as adding objects. (Ex: the SpamError object in the Python C API tutorial). (Py_InitModule is actually a macro that creates the real init call but with some info baked in like what version of Python our compiled C extension used.)
If the init function has the proper name but does not call Py_InitModule(), we get this:
SystemError: dynamic module not initialized properly
Our methods table happens to be called SpamMethods and looks like this.
static PyMethodDef SpamMethods[] = {
{"system", spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL, NULL, 0, NULL}
};
The method table itself and the function signature contracts it entails is the third and final key convention necessary for Python to make sense of dynamically loaded C. The method table is an array of struct PyMethodDef with a final sentinel entry. A PyMethodDef is defined in Include/methodobject.h as follows.
struct PyMethodDef {
const char *ml_name; /* The name of the built-in function/method */
PyCFunction ml_meth; /* The C function that implements it */
int ml_flags; /* Combination of METH_xxx flags, which mostly
describe the args expected by the C func */
const char *ml_doc; /* The __doc__ attribute, or NULL */
};
The crucial part here is that the second member is a PyCFunction. We passed in the address of a function, so what is a PyCFunction? It's a typedef, also in Include/methodobject.h
typedef PyObject *(*PyCFunction)(PyObject *, PyObject *);
PyCFunction is a typedef for a pointer to a function that returns a pointer to a PyObject and that takes for arguments two pointers to PyObjects. As a lemma to convention three, C functions registered with the method table all have the same signature.
Python circumvents much of the difficulty in dynamic loading by using a limited set of C function signatures. One signature in particular is used for most C functions. Pointers to C functions that take additional arguments can be "snuck in" by casting to PyCFunction. (See the keywdarg_parrot example in the Python C API tutorial.) Even C functions that backup Python functions that take no arguments in Python will take two arguments in C (shown below). All functions are also expected to return something (which may just be the None object). Functions that take multiple positional arguments in Python have to unpack those arguments from a single object in C.
That's how the data for interfacing with dynamically loaded C functions is acquired and stored. Finally, here's an example of how that data is used.
The context here is that we're evaluating Python "opcodes", instruction by instruction, and we've hit a function call opcode. (see https://docs.python.org/2/library/dis.html. It's worth a skim.) We've determined that the Python function object is backed by a C function. In the code below we check if the function in Python takes no arguments (in Python) and if so, call it (with two arguments in C).
Python/ceval.c.
if (flags & (METH_NOARGS | METH_O)) {
PyCFunction meth = PyCFunction_GET_FUNCTION(func);
PyObject *self = PyCFunction_GET_SELF(func);
if (flags & METH_NOARGS && na == 0) {
C_TRACE(x, (*meth)(self,NULL));
}
It does of course take arguments in C - exactly two. Since everything is an object in Python, it gets a self argument. At the bottom you can see that meth is assigned a function pointer which is then dereferenced and called. The return value ends up in x.
Related
New to cpp (Java guy).
I have 3rd party library that has method sendMail(txt).
I don't want to test the library. i want to test my own method, so in order to do this , i need to mock the library calls .
My own method is looking like this:
#include "mailsender.h"
int run(txt){
analysis(txt);
...
...
int status = sendMail(txt);//sendMail is a 3rd party library call. i need to mock it.its not part of the unit test
return status;
}
In Java the mailsender was interface and it was injected to my class, so in case of test i inject mock.
What is a good practice in cpp to mock library calls?
I can wrap the 3rd party library call in a class and inject this class, but i am looking for something simpler and for the common practice (maybe ifndf).
I am familiar with googlemock.
googlemock allow me to mock classes . i am not aware to option how to mock a call in my tested method.
So I assume you have a 'global' function that is implemented in a library that you both include a header file for (to get the definition) and link (to get the implementation).
You obviously need to replace the implementation of the library with your own - one that does "nothing", so you can do this in 2 ways:
you replace the .dll (or .so) with your own implementation that has all the methods the 3rd party library exposes. This is easy once you've written a new version of all the 3rd party lib functions, but writing them all out can be a pain.
you remove the library temporarily, and replace the calls you make to that in a .cpp source file that implements those functions. So you'd create your own sendMail() function in a .cpp file and include this into the program instead of the mailsender.h include.
The latter is easier, but you might also have to modify your program to not link with the 3rd party lib. This can also require changing the #include as well, as some compilers (eg VC++) allow you to embed linker directives in the source. If your does this, then you won't be able to stop the linker from including the 3rd party lib.
The other option is to modify your code to use a different call to the sendMail call, eg test__sendMail() that you implement yourself. Wrap this is a macro to conditionally include your, or the real, function call depending on your build options.
If this was a c++ library then you'd probably be able to use a mocking framework like you're used to, but it sounds like its a C library, and they simply provide a list of functions that you use directly in your code. You could wrap the library in your own class and use that instead of calling the 3rd party lib functions directly.
There is a list of C mocking frameworks.
This is an old question, with an already choosen response, but maybe the following contribution can help someone else.
First solution
You still have to create a custom library to redefine the functions, but you do not need to change Makefiles to link to your "fake-library", just use LD_PRELOAD with the path to the fake-library and that will be the first that the linker will find and then use.
example
Second solution
ld (GNU) linker has an option --wrap that let you wrap only one function with another provided by the user. This way you do not have to create a new library/class just to mock the behavior
Here is the example from the man page
--wrap=symbol
Use a wrapper function for symbol. Any undefined reference to symbol will be resolved to "__wrap_ symbol ". Any undefined reference
to "__real_ symbol " will be resolved to symbol.
This can be used to provide a wrapper for a system function. The wrapper function should be called "__wrap_ symbol ". If it wishes to
call the system function, it should call "__real_ symbol ".
Here is a trivial example:
void *
__wrap_malloc (size_t c)
{
printf ("malloc called with %zu\n", c);
return __real_malloc (c);
}
If you link other code with this file using --wrap malloc, then all calls to "malloc" will call the function "__wrap_malloc" instead.
The call to "__real_malloc" in "__wrap_malloc" will call the real
"malloc" function.
You may wish to provide a "__real_malloc" function as well, so that links without the --wrap option will succeed. If you do this, you
should not put the definition of "__real_malloc" in the same file as
"__wrap_malloc"; if you do, the assembler may resolve the call before
the linker has a chance to wrap it to "malloc".
Disclaimer: I wrote ELFspy.
Using ELFspy, the following code will allow you to fake/mock the sendMail function by replacing it with an alternative implementation.
void yourSendMail(const char* txt) // ensure same signature as sendMail
{
// your mocking code
}
int main(int argc, char** argv)
{
spy::initialise(argc, argv);
auto sendMail_hook = SPY(&sendMail); // grab a hook to sendMail
// use hook to reroute all program calls to sendMail to yourSendMail
auto sendMail_fake = spy::fake(sendMail_hook, &yourSendMail);
// call run here..
}
Your program must be compiled with position independent code (built with shared libraries) to achieve this.
Further examples are here:
https://github.com/mollismerx/elfspy/wiki
Though there is no interface keyword, you can use Abstract Base Classes for similar things in C++.
If the library you are using doesn't come with such abstractions, you can wrap it behind your own "interface". If your code separates construction of objects from usage (e.g. by IoC), you can either use this to inject a fake or use Mocks:
https://stackoverflow.com/questions/38493/are-there-any-good-c-mock-object-frameworks
While exploring Cython compile steps, I found I need to link C libraries like math explicitly in setup.py. However, such step was not needed for numpy. Why so? Is numpy being imported through usual python import mechanism? If that is the case, we need not explicitly link any extension module in Cython?
I tried to rummage through the official documentation, but unfortunately there was no explanation as to when an explicit linking is required and when it will be dealt automatically.
Call of a cdef-function corresponds more or less just to a jump to an address in the memory - the one from which the command should be read/executed. The question is how this address is provided. There are some cases we need to consider:
A. inline functions
The code of those functions is either inlined or the definition of the function is in the same translation unit, thus the address is known to the linker at the link time (or even compiler at compile-time) - no need for additional libraries.
An example are header-only libraries.
Consequences: Only include path(s) should be provided in setup.py.
B. static linking
The definition/functionality we need is in another translation unit/library - the target-address of the jump is calculated at the link-time and cannot be changed anymore afterwards.
An example are additional c/cpp-files or static libraries which are added to extension-definition.
Consequences: Static library should be added to setup.py, i.e. library-path and library name along with include paths.
C. dynamic linking
The necessary functionality is provided in a shared object/dll. The address to jump to is calculated during the runtime from loader and can be replaced at program start by exchanging the loaded shared objects.
An example are stdlibc++ (usually added automatically by g++) or libm, which is not automatically added to linker command by gcc.
Consequences: Dynamic library should be added to setup.py, i.e. library-path and library name, maybe r-path + include paths. Shared object/dll must be provided at the run time. More (than one probably would like to know) information about Cython/Python using dynamic libraries can be found in this SO-post.
D. Calling via a pointer
Linker is needed only when we call a function via its name. If we call it via a function-pointer, we don't need a linker/loader because the address of the function is already known - the value in the function pointer.
Example: Cython-generated modules uses this machinery to enable access to its cdef-functions exported through pxd-file. It creates a data structure (which is stored as variable __pyx_capi__ in the module itself) of function-pointers, which is filled by the loader once the so/dll is loaded via ldopen (or whatever Windows' equivalent). The lookup in the dictionary happens only once when the module is loaded and the addresses of functions are cached, so the calls during the run time have almost no overhead.
We can inspect it, for example via
#foo.pyx:
cdef void doit():
print("doit")
#foo.pxd
cdef void doit()
>>> cythonize -3 -i foo.pyx
>>> python -c "import foo; print(foo.__pyx_capi__)"
{'doit': <capsule object "void (void)" at 0x7f7b10bb16c0>}
Now, calling a cdef function from another module is just jumping to the corresponding address.
Consequences: We need to cimport the needed funcionality.
Numpy is a little bit more complicated as it uses a sophisticated combination of A and D in order to postpone the resolution of symbols until the run time, thus not needing shared-object/dlls at link time (but at run time!).
Some functionality in numpy-pxd file can be directly used because they are inlined (or even just defines), for example PyArray_NDIM, basically everything from ndarraytypes.h. This is the reason one can use cython's ndarrays without much ado.
Other functionality (basically everything from ndarrayobject.h) cannot be accessed without calling np.import_array() in an initialization step, for example PyArray_FromAny. Why?
The answer is in the header __multiarray_api.h which is included in ndarrayobject.h, but cannot be found in the git-repository as it is generated during the installation, where the definition of PyArray_FromAny can be looked up:
...
static void **PyArray_API=NULL; //usually...
...
#define PyArray_CheckFromAny \
(*(PyObject * (*)(PyObject *, PyArray_Descr *, int, int, int, PyObject *)) \
PyArray_API[108])
...
PyArray_CheckFromAny isn't a name of a function, but a define fo a function pointer saved in PyArray_API, which is not initialized (i.e. is NULL), when module is first loaded! Btw, there is also a (private) function called PyArray_CheckFromAny, which is what the function pointer actually points to - and because the public version is a define there is no name collision when linked...
The last piece of the puzzle - the function _import_array (more or less the working horse behind np.import_array) is an inline function (case A), so only include path is needed, to be able to use it.
_import_array uses a similar approach to Cython's __pyx_capi__ to get the function pointers: The field is called _ARRAY_API and can be inspected via:
>>> import numpy.core._multiarray_umath as macore
>>> macore._ARRAY_API
<capsule object NULL at 0x7f17d85f3810>
More info about how PyArray_API can be initialized can be found in this SO-answer of mine.
However, when using functionality from numpy/math.pxd, one has to staticly link numpy's math-library (see for example this SO-question).
Please read this post before answering: Pass a closure from Cython to C++
In the accepted answers, it is neatly shown how a python function is converted into a std::function using Boost Python.
Following this example I'm able to wrap functions taking an std::function as an argument and call them using a python function as input. However, this only works when the std::function parameters are primitives like int, double, string etc.
Any guidance on how to make this work for custom types as well will be highly appreciated.
This won't be a complete answer - it assumes you can fill in the gaps from my previous answer that the question was based on. Unfortunately it is a little bit more complicated than that case.
Just to define the problem - assume you have a parameter of a custom C++ class, like:
class cpp_class {
// some non-trivial contents
};
and thus your C++ interface looks like this:
void call_some_std_func(std::function<void(cpp_class&)> callback) {
callback(5,std::string("hello"));
}
The first thing to do is to write a Cython wrapper for your C++ class (in principle you could make a Boost Python wrapper instead). Here you need to make a choice of about "ownership" of the C++ object. The first choice is to make a copy:
cdef extern from "cpp_file.hpp":
cppclass cpp_class:
pass # details
cdef class CyWrapper:
cdef cpp_class* ptr
def __dealloc__(self):
del self.ptr
# other details following standard wrapper pattern
cdef public make_CyWrapper(cpp_class& x):
obj = CyWrapper()
obj.ptr = new cpp_class(x)
return obj
I've created a wrapper class with a destructor that handles the memory and a publicly accessible constructor function that can be called from external code. This version is safe because your wrapper owns the object it holds and so there can be no writes to invalid memory. However, because it makes a copy, you can't make changes to the original C++ object.
A second option is to hold a pointer to an object you don't own. The code is basically identical except you remove the __dealloc__ and avoid making a copy in make_CyWrapper:
obj.ptr = &x // instead of new cpp_class(x)
This is unsafe - you need to ensure your C++ object outlives the Cython wrapper - but allows you to modify the object.
You could also imagine a few other options: you could take ownership of an existing object with your Cython wrapper (Such a scheme would have to pass by pointer rather than reference, or it could use move constructors); you could deconstruct your C++ class into a representation expressed in basic types and pass those to Python; you could use shared pointers to split the ownership; or you have a more elaborate way of marking your Cython wrapper as "invalid" once your held C++ instance is destructed.
What you do next depends on whether you're using Boost Python (for it's convenient, callable wrapping of Python objects) or if you're making your own version. (I showed both possibilities in the previous answer).
Assuming Boost Python, you need to do two things - tell it about the conversion and make sure that it imports the module that your wrapper is defined in (if you don't do this you get exciting segmentation faults)
struct convert_to_PyWrapper {
static PyObject* convert(const cpp_class& rhs) {
// the const_cast here is a bit dodgy, but was needed to make it work
return make_CyWrapper(const_cast<cpp_class&>(rhs));
}
};
inline void setup_boost_python() {
PyInit_your_module_name(); // named inityour_module_name in Python 2
boost::python::to_python_converter<
cpp_class,
convert_to_PyWrapper>();
}
You need to make sure that your Python/Cython code calls "setup_boost_python" before attempting to use the callback (if you put it at module level it's done on import, which is ideal).
If you're following my "manual" scheme (avoiding the dependency on Boost Python) then you need to modify the call_obj Cython function that does the C++ to Cython type conversion.
cdef public void call_obj(obj, cpp_class& c):
obj(make_CyWrapper(c))
You also need to ensure the wrapper Cython module is imported before use (otherwise you get segmentation faults). I did this in "py_object_wrapper.hpp" but providing it's done once somewhere you can place it where you like.
void operator()(cpp_class& a) {
PyInit_your_module_name();
if (held) {
call_obj(held,a);
}
}
The method mapping table is an array of PyMethodDef structures,
struct PyMethodDef {
char *ml_name;
PyCFunction ml_meth;
int ml_flags;
char *ml_doc;
};
where the ml_name is the name of the functions accessed in the interpreter (i.e. gc.collect()), ml_meth is the address of the function described in the previous section, ml_flags indicates which of the signatures ml_meth is using and ml_doc is the docstring for the function.
Now, say I want to access from within the gcmodule.c gc_list_append which is not exposed by the gc interface.
I've tried the following:
pythonapi.PyImport_ImportModule("gcmodule")
or
pythonapi.PyImport_ImportModule(ctypes.c_wchar("gcmodule"))
Hoping that the reference returned can be used to access unexposed methods, but in both cases I get errors:
ImportError: No module named 'g'
or
TypeError: one character unicode string expected
Any idea how an unexposed function or data structures can be accessed via ctypes/pythonapi?
PyImport_ImportModule imports a module as named by the const char * argument. It calls PyImport_Import, which in turn calls __builtin__.__import__ in PY2. Calling it with the C string "gc" returns a reference to sys.modules['gc'] if it exists, else the interpreter creates the module by calling initgc , which it finds in _PyImport_Inittab. The gc module's init function calls Py_InitModule4, which loops over the GcMethods table to add methods to the module dict.
Every global definition you see in gcmodule.c that's marked static is only visible within that compilation unit at the system level, i.e. it doesn't have external linkage. Commonly an extension module only exports the name of the PyMODINIT_FUNC, but several of the built-in modules in CPython also export public APIs such as PyObject_GC_Track.
On Windows, exporting a symbol from a DLL by name or ordinal requires the extra step of declaring it in a .def file or with the modifier __declspec(dllexport). In CPython that gets applied via the macros PyAPI_FUNC (pyport.h), PyAPI_DATA, and PyMODINIT_FUNC. For example, objimpl.h declares PyAPI_FUNC(void) PyObject_GC_Track(void *).
Declaring a symbol static is namespace management. However, if you can get the address of a function somehow, you can certainly call it with a ctypes function pointer. Finding the address is the issue. ctypes uses the standard POSIX dlsym and Windows GetProcAddress functions to look up exported symbols, which won't help you here. You may be able to access function pointers referenced directly or indirectly by CPython objects, since id returns the object base address.
The following isn't directly related to your problem, but I'm including it for general interest.
On a POSIX system that loads extension modules with dlopen, you can specify RTLD_GLOBAL to merge the dynamic symbols into the global namespace. For example, the _ctypes extension module is linked from multiple object files and needs several extern symbols, such as PyCData_set. You can tweak the import to have PyCData_set accessible globally (e.g. via ctypes.pythonapi):
>>> import sys, DLFCN
>>> sys.setdlopenflags(DLFCN.RTLD_GLOBAL | DLFCN.RTLD_NOW)
>>> import ctypes
>>> hasattr(ctypes.pythonapi, 'PyCData_set')
True
Of course, that won't work on Windows. The proper way for an extension module to make a C API available is by using a PyCapsule, such as datetime.datetime_CAPI (datetimemodule.c).
I have dll, that builded in Delphi, and I try to call function from it. Declaration of function looks like this:
function GetUid(UID:Pointer):Integer; stdcall;
This is equivalent to this C function signature:
int GetUID(void *pointer);
Library handled using ctypes:
from ctypes import *
lib = cdll.LoadLibrary("mylib.dll")
But i stuck here:
res = lib.GetUid(?)
What I need to pass in this function?
Pointer is void *, but how make this rightly in python?
Python is a high level language. You do not typically import a DLL from a C or Pascal native library and invoke it and pass variables from Python into a C or Pascal function taking a void * type raw pointer and then manipulate raw memory this way.
In short if you knew what you were doing you would know better than to try to do what you're doing here.
Let's suppose that your implementation is like this:
function GetUid(UID:Pointer):Integer; stdcall;
var
P2:^Integer;
begin
P2 := UID;
P2^ := 0;
end;
Then, what you would want to do is pass in an address to a 32 bit integer. Of course my example above is absurd, because what would have made sense above is to just declare the parameter as an "int *pointer" (in C terms) rather than as a "void *pointer".
Whatever it is you're doing, the next thing that will likely happen is that you will corrupt your python interpreter's heap, and cause lots of fun crashes and errors.
A far more sensible approach is to read the Python documentation on writing C extensions that can manipulate native Python types (PyObject), and doing the same thing but in pascal, if you like.
p4d appears to be a workable way of writing extension DLLs in delphi:
https://code.google.com/p/python4delphi/source/list