I wrote a test dll in C++ to make sure things work before I start using a more important dll that I need. Basically it takes two doubles and adds them, then returns the result. I've been playing around and with other test functions I've gotten returns to work, I just can't pass an argument due to errors.
My code is:
import ctypes
import string
nDLL = ctypes.WinDLL('test.dll')
func = nDLL['haloshg_add']
func.restype = ctypes.c_double
func.argtypes = (ctypes.c_double,ctypes.c_double)
print(func(5.0,5.0))
It returns the error for the line that called "func":
ValueError: Procedure probably called with too many arguments (8 bytes in excess)
What am I doing wrong? Thanks.
You probably got the calling conventions mixed up. I'm guessing you have a C function declared something like this:
double haloshg_add(double d1, double s2)
{
return d1+d2;
}
This will use the C calling convention by default. The simplest approach would be to change the calling convention in your ctypes code:
nDLL = ctypes.CDLL('test.dll')
If you wanted to change the calling convention in the C code to stdcall (to match ctypes.WinDLL) then you would do this:
double __stdcall haloshg_add(double d1, double s2)
Whatever you do, only do one of these changes. If you do both you'll have the reverse failure!
If it were me, I'd just change the Python code to use C calling convention (use CDLL). That change has the least impact.
Related
I have a module called test.go that contains two simple Go functions which accept string types:
package main
import (
"fmt"
"C"
)
//export TestConcat
func TestConcat(testArg string, testArg2 string) (string) {
retval := testArg + testArg2
return retval
}
//export TestHello
func TestHello(testArg string) {
fmt.Println("%v\n", testArg)
}
func main(){}
I compile it as a shared library with go build -o test.so -buildmode=c-shared test.go
Then I have a Python module called test.py
import ctypes
from ctypes import cdll
test_strings = [
"teststring1",
"teststring2"
]
if __name__ == '__main__':
lib = cdll.LoadLibrary("./test.so")
lib.TestConcat.argtypes = [ctypes.c_wchar_p, ctypes.c_wchar_p]
lib.TestHello.argtypes = [ctypes.c_wchar_p]
for test_string in test_strings:
print(
lib.TestConcat("hello", test_string)
)
lib.TestHello(test_string)
Then I run test.py and get a nasty segfault
runtime: out of memory: cannot allocate 279362762964992-byte block (66781184 in use)
fatal error: out of memory
I've tried wrapping the arguments in ctypes.c_wchar_p to no avail.
What am I doing wrong here? And specifically, how does one interact with Go functions that accept string arguments in Python?
The Go's string type is actually something like
type string {
ptr *byte
size int
}
so that is what the Test{Hello|Concat} actually expect—not a pair of pointers but a pair of struct-typed values.
In other words, cgo performs just enough magic to gateway calls from Go to C and back, but it does not perform automatic conversions of values.
You have two options:
Explicitly work with this from your ctypes bindings, if possible.
When compiling your package, cgo generates a header file which contains a C definition for the struct representing a Go string; you could use it right away.
Make the functions exported to C compatible with the C's "type system".
For this, cgo offers helper functions C.CString and C.GoString.
Basically, you can define your API like this:
func TestHello(a, b *C.char) *C.char {
testArg1, testArg2 := C.GoString(a), C.GoString(b)
return C.CString(testArg + TestArg2)
}
Note few caveats here:
Both of these helpers copy the memory of their argument, so the silly example above would work just fine but it would first duplicate the memory blocks pointed to by a and b, then eat up twice as much memory to produce the concatenated string and then copy the memory of the resulting string once again to produce the returned pointer.
IOW, this approach is fine if you're trying to export to C some big chunk of Go code so that these allocations are dwarfed by whatever that chunk does.
Using *C.char is the same as *char in C, so the string is expected to be NUL-terminated; if it's not, use C.GoStringN.
Every memory block allocated by C.CString have to be freed by a call to C.free. And here's a twist: C.free is basically a thin shim to call free() from the linked in libc, so if you can guarantee the complete product (the code fully loaded into memory and (inter)linked using the dymanic linker) has only a single copy of libc linked in, you can call free() from the non-Go code on the memory blocks produced by calls to C.Cstring in the Go code.
A few more random pointers:
I'm not well-versed in Python's ctypes but I'd speculate using ctypes.c_wchar_p is not correct: in C (and C++, FWIW) wchar_t is a type to denote a single fixed-sized "wide character", which is usually a UCS-2/UTF-16 code point, and Go's strings are not composed of these—they may contain arbitrary bytes, and when they are used to contain Unicode text, they are encoded using UTF-8 which is a multi-byte ecoding (a single Unicode code point may be represented by 1 to 4 bytes in the string).
In either case, wchar_t cannot be used for UTF-8 (and actually many seasoned devs beleive it's an abomination).
Please read the docs on cmd/cgo completely before embarking on this project. Really, please do!
I receive this error when I try to call this function written in Delphi. But another code works fine. Maybe I'm not declaring the args ans result types? I am using 32Bit python 3.7). Related code snippets:
Delphi:
Test(deposit, MarginCall: double; CallBack: TProgrCallBackProc); stdcall;
Python:
self.FTCore = ctypes.WinDLL(self.FTCore_library_path)
self.FTCore.Test.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double)]
self.FTCore.Test.restype = ctypes.POINTER(ctypes.c_char)
deposit = ctypes.c_double(100)
callback = ctypes.c_double(1)
self.FTCore.Test(deposit, callback)
Error:
violation reading 0x00000004
Three errors that I can see:
The Delphi function accepts three arguments, you define only two in argtypes. You will need to define the third argument, TProgrCallBackProc defined somewhere in the Delphi code.
The two double parameters are passed by value, but you define them as pointers to double in your argtypes definition. They should be defined as plain ctypes.c_double.
The Delphi function has no return value, but your restype contradicts that. You need to set restype to None.
I am creating Python bindings for a C library.
In C the code to use the functions would look like this:
Ihandle *foo;
foo = MethFunc();
SetArribute(foo, 's');
I am trying to get this into Python. Where I have MethFunc() and SetAttribute() functions that could be used in my Python code:
import mymodule
foo = mymodule.MethFunc()
mymodule.SetAttribute(foo)
So far my C code to return the function looks like this:
static PyObject * _MethFunc(PyObject *self, PyObject *args) {
return Py_BuildValue("O", MethFunc());
}
But that fails by crashing (no errors)
I have also tried return MethFunc(); but that failed.
How can I return the function foo (or if what I am trying to achieve is completely wrong, how should I go about passing MethFunc() to SetAttribute())?
The problem here is that MethFunc() returns an IHandle *, but you're telling Python to treat it as a PyObject *. Presumably those are completely unrelated types.
A PyObject * (or any struct you or Python defines that starts with an appropriate HEAD macro) begins with pointers to a refcount and a type, and the first thing Python is going to do with any object you hand it is deal with those pointers. So, if you give it an object that instead starts with, say, two ints, Python is going to end up trying to access a type at 0x00020001 or similar, which is almost certain to segfault.
If you need to pass around a pointer to some C object, you have to wrap it up in a Python object. There are three ways to do this, from hackiest to most solid.
First, you can just cast the IHandle * to a size_t, then PyLong_FromSize_t it.
This is dead simple to implement. But it means these objects are going to look exactly like numbers from the Python side, because that's all they are.
Obviously you can't attach a method to this number; instead, your API has to be a free function that takes a number, then casts that number back to an IHandle* and calls a method.
It's more like, e.g., C's stdio, where you have to keep passing stdin or f as an argument to fread, instead of Python's io, where you call methods on sys.stdin or f.
But even worse, because there's no type checking, static or dynamic, to protect you from some Python code accidentally passing you the number 42. Which you'll then cast to an IHandle * and try to dereference, leading to a segfault…
And if you were hoping Python's garbage collector would help you know when the object is still referenced, you're out of luck. You need to make your users manually keep track of the number and call some CloseHandle function when they're done with it.
Really, this isn't that much better than accessing your code from ctypes, so hopefully that inspires you to keep reading.
A better solution is to cast the IHandle * to a void *, then PyCapsule_New it.
If you haven't read about capsules, you need to at least skim the main chapter. But the basic idea is that it wraps up a void* as a Python object.
So, it's almost as simple as passing around numbers, but solves most of the problems. Capsules are opaque values which your Python users can't accidentally do arithmetic on; they can't send you 42 in place of a capsule; you can attach a function that gets called when the last reference to a capsule goes away; you can even give it a nice name to show up in the repr.
But you still can't attach any behavior to capsules.
So, your API will still have to be a MethSetAttribute(mymodule, foo) instead of mymeth.SetAttribute(foo) if mymodule is a capsule, just as if it's an int. (Except now it's type-safe.)
Finally, you can build a new Python extension type for a struct that contains an IHandle *.
This is a lot more work. And if you haven't read the tutorial on Defining Extension Types, you need to go thoroughly read through that whole chapter.
But it means that you have an actual Python type, with everything that goes with it.
You can give it a SetAttribute method, and Python code can just call that method. You can give it whatever __str__ and __repr__ you want. You can give it a __doc__. Python code can do isinstance(mymodule, MyMeth). And so on.
If you're willing to use C++, or D, or Rust instead of C, there are some great libraries (PyCxx, boost::python, Pyd, rust-python, etc.) that can do most of the boilerplate for you. You just declare that you want a Python class and how you want its attributes and methods bound to your C attributes and methods and you get something you can use like a C++ class, except that it's actually a PyObject * under the covers. (And it'll even takes care of all the refcounting cruft for you via RAII, which will save you endless weekends debugging segfaults and memory leaks…)
Or you can use Cython, which lets you write C extension modules in a language that's basically Python, but extended to interface with C code. So your wrapper class is just a class, but with a special private cdef attribute that holds the IHandle *, and your SetAttribute(self, s) can just call the C SetAttribute function with that private attribute.
Or, as suggested by user, you can also use SWIG to generate the C bindings for you. For simple cases, it's pretty trivial—just feed it your C API, and it gives you back the code to build your Python .so. For less simple cases, I personally find it a lot more painful than something like PyCxx, but it definitely has a lower learning curve if you don't already know C++.
I am trying to recreate a DLL function call in Python. The code uses a DLL file to set the laser parameters for various material setting. Here is the original C# code:
public bool ApplyLasFile(string strLasFile)
{
if (!File.Exists(strLasFile))
{
//MessageBox.Show("File not found", "Error", MessageBoxButtons.OK);
return false;
}
Char[] arr = strLasFile.ToCharArray();
ApplyLAS(arr, strLasFile.Length);
return true;
}
Here is my python code:
def setLaserSettings(input):
#laserSettings(power (0-1000), speed (0-1000), height (0-4000), ppi (0-1000))
input=input.upper()
if(input=="ABS"):
settings=laserSettings(825, 1000)
elif(input=="STAINLESS"):
settings=laserSettings(1000, 84)
elif(input=="TITANIUM"):
settings=laserSettings(1000, 84)
else:
return False
charList=[]
for x in range (0, len(settings)): #convert string into c_char[]
charList.append(c_char(settings[x]))
#print charList
print ULSLib.ApplyLAS(charList, len(settings))
The DLL call ULSLib.ApplyLAS(charList, len(settings)) is returning the error ArgumentError: argument 1: <type 'exceptions.TypeError'>: Don't know how to convert parameter 1. I started out with just using list(settings) to stand in for ToCharArray(), and when that didn't work I built a c_char array per the Python ctypes man page. However, I am still getting that error. Does anybody see what I am missing? Thanks for any help you can offer.
Edit: I have also tried list(string) and list(charList) and both return the same error.
So this is weird. The function call requires a Char array, but putting the string strait into the function call within python seems to be getting the job done for some reason. I am new to python, so hopefully someday this will make sense to me.
I suspect the reason that your code works unexpectedly is that the C function you are calling in the DLL takes a char * for its first parameter. I'm guessing its declaration looks something like:
bool ApplyLAS(char *filename, int length);
I'm guessing the return type here, and it's also quite possible that the first parameter is actually a wchar_t * or the second is an unsigned int. I'm also assuming your DLL has been compiled from C code. It's possible it may have been compiled from another language such as Fortran instead.
A char * parameter, such as the one I suspect is in your C function, is typically how you pass a string to C. Python's ctypes module will pass a Python string to the C function as a char *, thereby making it easy to pass a string to C.
I'm experimenting with ctypes, and I found something that baffled me. I have a .so file that exports this function:
void hello(int a) {
printf("hello, a=%d", a);
}
I call it from Python, but I put a parameter of the wrong type to see what will happen:
mydll = ctypes.CDLL("libhello.so")
mydll.hello('this is string')
This doesn't throw an error, and just prints hello, a=-1219868708. But if I pass a double instead – mydll.hello(1.0), then it throws an exception:
ctypes.ArgumentError: argument 1: : Don't know how to convert parameter 1
I find it strange because exported C functions usually don't have information about their signature (unlike C++ functions), just the name of the function and maybe the total number of bytes of the parameters.
So thinking that maybe ctypes was counting the number of bytes to ensure some measure of type safety, I tried passing 2 strings – mydll.hello('string1', 'string2'), and there was no error again; it just printed hello, a=-1223698284.
So now I'm stumped. What does ctypes check, exactly, so that it knows that a double is wrong, but at the same time doesn't know that 2 strings are not?
The ctypes documentation states:
None, integers, longs, byte strings and unicode strings are the only native Python objects that can directly be used as parameters in these function calls.
To pass another type such as a Python float, you have to wrap it in the corresponding ctypes type (c_double or c_float). Otherwise, you get the ArgumentError exception.
So this error wasn't about passing an incorrect type: You'd get the error even if double or float was the type expected by the C function. As you correctly said, ctypes does not know the actual types expected by the C function and cannot check that the correct type was passed.