As part of larger cython project I am getting an error that seems to indicate that maybe the automatic type merging is failing. It is possible I am doing something silly or that the fact that I am structuring things in this way is a bad smell. I think I can get around the issue by using void* and then casts everywhere but this seems quite messy for what should be a simple thing.
It seems similar to this issue, but that involved function pointers.
Wrong signature error using function types in multiple modules
I have put together a minimal example that demonstrates the issue. The files are as follows.
a.pxd
cdef struct A:
int a
cdef A makeA(int a)
a.pyx
cdef struct A:
pass
cdef A makeA(int a):
cdef A sA
sA.a = a
return sA
b.pyx
from cfr.a cimport A, makeA
cdef int get_A_value():
cdef A sA = makeA(5)
return sA.a
If I compile and then try to import b in a jupyter-lab notebook I get the following error:
b.pyx in init b()
TypeError: C function cfr.a.makeA has wrong signature (expected struct __pyx_t_3cfr_1a_A (int), got struct __pyx_t_1a_A (int))
EDIT: It looks like it might be something to do with my directory structure. Note that I do from cfr.a cimport A (as this happens to be the layout of my project). I think I need to do from a import A and get the import to work through the use of my pythonpath.
So it turned out the issue was in the from cfr.a cimport A line. This was somehow meaning that the A type was not being seen as the same type as A in the a.pyx file.
To fix use:
from a cimport A
(You may then get a not found error, but we can fix this with our pythonpath).
In the comments adding a __init__.pxd file was suggested, but adding a blank one did not work for me.
Related
Although I few similar questions already being asked, but I couldn't get head around on how to fix.
Basically I have this function:
Module one.pyx:
cdef char *isInData(data, dataLength):
cdef char *ret = <char *>malloc(200)
memset(ret, 0x00, 200)
if (b"Hello" in data and b"World" in data):
strcat(ret, b"Hello World!")
return ret
Module two.pyx:
import one
from libc.stdlib cimport malloc,realloc, free
cdef char *tempo():
cdef char *str
str = one.isInData(b"Hello what is up World", 23)
# do some stuff
free(str)
Issue occurs on line str = one.isInData("Hello what is up World", 23), I am assuming that as soon as isInData->ret is assigned to str. isInData->ret is deleted which causes this issue. But annyone help me on how to fix this?
import one
This line does a Python import of one. It doesn't know about any cdef functions defined in one (or even that one is a Cython module...). Therefore it assumes that isInData is a Python object that it can look up and that it'll be a callable returning another Python object.
cdf char* str = some_python_function() is unsafe because str points into the Python object. However the Python object is only a temporary and is most likely freed almost immediately.
What you mean is:
cimport one
which tells Cython that it's a Cython module that you're importing, and it should expect to find the declarations at compile-time. You'll probably need to write a pxd file to give the declarations.
I haven't looked in detail at the rest of your code so don't know if you are handling C strings right. But in general you may find it easier just to use Python strings rather than messing around with C string handling.
I am using C-contiguous memoryviews in my Python code and I would like to use dgemm which needs Fortran-contiguous memoryviews.
I would like to use the function PyMemoryView_GetContiguous found here but I don't know how to access it.
Does someone know which import I have to do ?
I don't want to use the function copy_fortran() as it really slows down my code.
PyMemoryView_GetContiguous doesn't look to be exposed as part of the Cython standard includes unfortunately. It should be reasonably easy to wrap though:
from cpython.buffer cimport PyBUF_READ # we'll need this later
cdef extern from "Python.h":
# copy the signature replacing PyObject* with object to let Cython
# handle the reference counting
object PyMemoryView_GetContiguous(object, int, char)
def test(double[:,::1] c_contig):
f_contig = PyMemoryView_GetContiguous(c_contig, PyBuf_READ,'F')
# .... do something useful
Be aware that this will still involve copying all the memory (this is absolutely unavoidable!) so is unlikely to be significantly faster than copy_fortran.
There's a problem though - PyMemoryView_GetContiguous won't return a writeable memoryview unless it does not have to make a copy, and Cython requires things assigned to a typed memoryview be writeable, so you can only use it as a Python object.
You can get a pointer to the first element though - the underlying object that created is a bytes/str object, so you can get a char* then cast that to whatever pointer you need. This should be enough to call your Fortran functions:
cdef char* as_bytes = f_contig.obj
some_function(<double*>as_bytes)
I'm trying to talk to this DLL using python's ctypes. Many of the functions take or return an HGRABBER type:
typedef struct HGRABBER_t__ { int unused; } HGRABBER_t;
#define HGRABBER HGRABBER_t*
(the full header file can be viewed here). Here's an example of a function prototype that returns an HGRABBER type:
HGRABBER __stdcall IC_CreateGrabber();
Here's my attempt at implementing this struct in python, and using it to call that function from the DLL:
import ctypes as C
class GrabberHandle(C.Structure):
_fields_ = [('unused', C.c_int)]
dll = C.windll.LoadLibrary('tisgrabber_x64.dll')
dll.create_grabber = dll.IC_CreateGrabber
dll.create_grabber.argtypes = []
dll.create_grabber.restype = GrabberHandle
my_handle = dll.create_grabber()
This seems to work, but I'm worried that I'm doing this wrong. I'm not experienced with C, and I don't think I understand the typedef and #define statements which define the HGRABBER type. Am I calling IC_CreateGrabber correctly? Should I have defined GrabberHandle to be a pointer to a struct, instead of a struct?
Thanks for reading, please let me know if I can clarify my question somehow.
You're right that you actually want a POINTER to the Structure, not the Structure itself.
Translating the C into English, being very loose (in a way that would be dangerous if you were trying to learn C but is good enough for using ctypes):
The struct defines a type named struct HGRABBER_t__, as a structure with one int in it.
The typedef defines a type named HGRABBER_t, as a synonym for struct HGRABBER_t__.
The #define defines a type named HGRABBER as a pointer to HGRABBER_t.
So, your GrabberHandle is the equivalent of HGRABBER_t; the equivalent of HGRABBER is:
GrabberHandlePtr = C.POINTER(GrabberHandle)
So you want this:
dll.create_grabber.restype = GrabberHandlePtr
It may be hard to debug the difference. A C struct with nothing but an int in it looks identical to an int in memory. And on Win32, an int and a pointer are both 32-bit values. And an int named unused is likely to be filled with meaningless garbage, making it hard to distinguish it from a pointer you've accidentally treated as an int. So everything will look fine, until you segfault 30 lines later in your code and have no idea what's wrong. :)
This library does what you are trying to do: https://github.com/morefigs/py-ic-imaging-control :)
But to answer your question, the library uses the code:
from ctypes import *
import os
class GrabberHandle(Structure):
pass
GrabberHandle._fields_ = [('unused', c_int)]
# set and check path
dll_path = os.path.join(os.path.expanduser('~'),
'Documents\\The Imaging Source Europe GmbH\\TIS Grabber DLL\\bin\\win32\\tisgrabber.dll')
with open(dll_path) as thefile:
pass
# open DLL
_ic_grabber_dll = windll.LoadLibrary(dll_path)
# create grabber
create_grabber = _ic_grabber_dll.IC_CreateGrabber
create_grabber.restype = POINTER(GrabberHandle)
create_grabber.argtypes = None
# get handle
handle = create_grabber()
Edit: changed code to use a pointer to GrabberHandle as per abarnert's answer as this is correct. However, in this particular case I have found no practical difference (with the 32-bit DLL), probably because the GrabberHandle structure is so simple.
Let us have script foo.pyx with function in it:
def hello():
cdef int* i = <int *> malloc(sizeof(int))
i[0] = 1
trol(i)
print i
and script with function noo.pyx:
cdef trol(int * i):
i[0] = 42
the question is, how do I now import the trol function from file noo.pyx to foo.pyx, so I can use it in hello function.
This is only model example, but I think, that it illustrate the problem fair enough.
I tried simple
from noo import trol
but that throws "Cannot convert 'int *' to Python object"
Edit: I would better add, that this example will work just fine if I put both functions to same file.
This seems like something obvious to try, but did you try:
from noo cimport trol
If you use import instead of cimport, I think it will try to cast trol as a python function and generate the error you're getting.
The solution eventually was to create additional .pxd file, which something very similar to classic header .h file in C. It stores functions declarations and when cimport is called, it is in this file where it looks for functions and structures.
So to be specific, all I needed to do was to create file noo.pxd containing:
cdef trol(int * i)
and than we can simply cimport this function from foo.pyx by calling
from noo cimport trol
I'm trying to port some python code to cython and I'm encountering some minor problems.
Below you see a code snippet (simplified example) of the code.
cimport numpy as np
cimport cython
#cython.boundscheck(False) # turn of bounds-checking for entire function
#cython.wraparound(False)
#cython.nonecheck(False)
def Interpolation(cells, int nmbcellsx):
cdef np.ndarray[float,ndim=1] celle
cdef int cellnonzero
cdef int i,l
for i in range(nmbcellsx):
celle = cells[i].e
cellnonzero = cells[i].nonzero
for l in range(cellnonzero):
celle[l] = celle[l] * celle[l]
I don't understand why the inner-most loop does not fully translate to C code (i.e. the last line, celle[l] = ...), see output from cython -a feedback:
What am I missing here?
Thanks a lot.
I finally realized that a simple "return 0" at the very end of the function solves this problem. However, this behaviour seems quite strange to me. Is this actually a bug?