Related
I'm trying to initialize two vectors in memory using gsl_vector_set(). In the main code it is initialized to zero on default, but I wanted to initialize them to some non-zero value. I made a test code based on a working function that uses the gsl_vector_set() function.
from ctypes import *;
gsl = cdll.LoadLibrary('libgsl-0.dll');
gsl.gsl_vector_get.restype = c_double;
gsl.gsl_matrix_get.restype = c_double;
gsl.gsl_vector_set.restype = c_double;
foo = dict(
x_ht = [0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,
0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],
x_ht_m = [0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,
0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0]
);
for f in range(0,18):
gsl.gsl_vector_set(foo['x_ht_m'],f,c_double(1.0));
gsl.gsl_vector_set(foo['x_ht'],f,c_double(1.0));
When I run the code I get this error.
ArgumentError: argument 1: <type 'exceptions.TypeError'>: Don't know how to convert parameter 1
I'm new to using ctypes and gsl functions so I'm not sure what the issue is or what the error message means. I an also not sure if there is a better way that I should be trying to save a vector to memory
Thank you #CristiFati for pointing out that I needed gsl_vector_calloc in my test code. I noticed that in the main code I was working in that the vector I needed to set was
NAV.KF_dictnry['x_hat_m']
instead of
NAV.KF_dictnry['x_ht_m']
So I fixed the test code to mirror the real code a bit better by creating a class holding the dictionary, and added the ability to change each value in the vector to an arbitrary value.
from ctypes import *;
gsl = cdll.LoadLibrary('libgsl-0.dll');
gsl.gsl_vector_get.restype = c_double;
gsl.gsl_matrix_get.restype = c_double;
gsl.gsl_vector_set.restype = c_double;
class foo(object):
fu = dict(
x_hat = gsl.gsl_vector_calloc(c_size_t(18)),
x_hat_m = gsl.gsl_vector_calloc(c_size_t(18)),
);
x_ht = [1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]
x_ht_m = [1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]
for f in range(0,18):
gsl.gsl_vector_set(foo.fu['x_hat_m'],f,c_double(x_ht_m[f]));
gsl.gsl_vector_set(foo.fu['x_hat'],f,c_double(x_ht[f]));
After running I checked with:
gsl.gsl_vector_get(foo.fu['x_hat_m'],0)
and got out a 1.0 (worked for the entire vector).
Turned out to just be some stupid mistakes on my end.
Thanks again!
I am a Python n00b and at the risk of asking an elementary question, here I go.
I am porting some code from C to Python for various reasons that I don't want to go into.
In the C code, I have some code that I reproduce below.
float table[3][101][4];
int kx[6] = {0,1,0,2,1,0};
int kz[6] = {0,0,1,0,1,2};
I want an equivalent Python expression for the C code below:
float *px, *pz;
int lx = LX; /* constant defined somewhere else */
int lz = LZ; /* constant defined somewhere else */
px = &(table[kx[i]][0][0])+lx;
pz = &(table[kz[i]][0][0])+lz;
Can someone please help me by giving me the equivalent expression in Python?
Here's the thing... you can't do pointers in python, so what you're showing here is not "portable" in the sense that:
float *px, *pz; <-- this doesn't exist
int lx = LX; /* constant defined somewhere else */
int lz = LZ; /* constant defined somewhere else */
px = &(table[kx[i]][0][0])+lx;
pz = &(table[kz[i]][0][0])+lz;
^ ^ ^
| | |
+----+----------------------+---- Therefore none of this makes any sense...
What you're trying to do is have a pointer to some offset in your multidimensional array table, because you can't do that in python, you don't want to "port" this code verbatim.
Follow the logic beyond this, what are you doing with px and pz? That is the code you need to understand to try and port.
There is no direct equivalent for your C code, since Python has no pointers or pointer arithmetic. Instead, refactor your code to index into the table with bracket notation.
table[kx[i]][0][lx] = 3
would be a rough equivalent of the C
px = &(table[kx[i]][0][0])+lx;
*px = 3;
Note that in Python, your table would not be contiguous. In particular, while this might work in C:
px[10] = 3; // Bounds violation!
This will IndexError in Python:
table[kx[i]][0][lx + 10] = 3
This might be a bit foolish but I'm trying to use ctypes to call a function that receives a complex vector as a paramter. But in ctypes there isn't a class c_complex.
Does anybody know how to solve this?
edit: I'm refering to python's ctypes, in case there are others....
I don't really like the #Biggsy answer above, because it requires additional wrapper written in c/fortran. For a simple case it does not matter, but if you want to use external libraries (like lapack) that would require you to make some proxy dll for every function/library you want to call and it probably won't be fun and many things can go wrong (like linkage, reusability e.t.c). My approach relies only on python with the aid of ctypes and numpy. Hope this will be helpful.
In the example below I show how to pass complex numbers/arrays between python, numpy (both are essentially C) and pure C or fortran (i use fortran, but for C it is exactly the same on python side).
First lets make a dummy fortran (f90) library, say f.f90, with C interface:
! print complex number
subroutine print_c(c) bind(c)
use iso_c_binding
implicit none
complex(c_double_complex), intent(in) :: c
print "(A, 2F20.16)", "#print_c, c=", c
end subroutine print_c
! multiply real numbers
real(c_double) function mtp_rr(r1,r2) bind(c)
use iso_c_binding
implicit none
real(c_double), intent(in) :: r1,r2
print "(A)", "#mpt_rr" ! make sure its fortran
mtp_rr = r1 * r2
end function mtp_rr
! multiply complex numbers
complex(c_double_complex) function mtp_cc(c1,c2) bind(c)
use iso_c_binding
implicit none
complex(c_double_complex), intent(in) :: c1,c2
print "(A)", "#mpt_cc" ! make sure its fortran
mtp_cc = c1 * c2
return
end function mtp_cc
! print array of complex numbers
subroutine print_carr(n, carr) bind(c)
use iso_c_binding
implicit none
integer(c_long), intent(in) :: n
integer(c_long) :: i
complex(c_double_complex), intent(in) :: carr(n)
print "(A)", "#print_carr"
do i=1,n
print "(I5,2F20.16)", i, carr(i)
end do
end subroutine print_carr
The library can be simply compiled as usual (in this example into libf.so). Notice that each subroutine contains its own "use" statement, this can be avoided if declare "use iso_c_binding" on module level. (I don't know why gfortran understands type of functions without global use of iso_c_binding, but it works and its fine for me.)
gfortran -shared -fPIC f.f90 -o libf.so
Then i created a python script, say p.py, with the following content:
#!/usr/bin/env python3
import ctypes as ct
import numpy as np
## ctypes 1.1.0
## numpy 1.19.5
# print("ctypes", ct.__version__)
# print("numpy", np.__version__)
from numpy.ctypeslib import ndpointer
## first we prepare some datatypes
c_double = ct.c_double
class c_double_complex(ct.Structure):
"""complex is a c structure
https://docs.python.org/3/library/ctypes.html#module-ctypes suggests
to use ctypes.Structure to pass structures (and, therefore, complex)
"""
_fields_ = [("real", ct.c_double),("imag", ct.c_double)]
#property
def value(self):
return self.real+1j*self.imag # fields declared above
c_double_complex_p = ct.POINTER(c_double_complex) # pointer to our complex
## pointers
c_long_p = ct.POINTER(ct.c_long) # pointer to long (python `int`)
c_double_p = ct.POINTER(ct.c_double) # similar to ctypes.c_char_p, i guess?
# ndpointers work well with unknown dimension and shape
c_double_complex_ndp = ndpointer(np.complex128, ndim=None)
### ct.c_double_complex
### > AttributeError: module 'ctypes' has no attribute 'c_double_complex'
## then we prepare some dummy functions to simplify argument passing
b_long = lambda i: ct.byref(ct.c_long(i))
b_double = lambda d: ct.byref(ct.c_double(d))
b_double_complex = lambda c: ct.byref(c_double_complex(c.real, c.imag))
## now we load the library
libf = ct.CDLL("libf.so")
## and define IO types of its functions/routines
print_c = libf.print_c
print_c.argtypes = [c_double_complex_p] # fortran accepes only references
print_c.restype = None # fortran subroutine (c void)
mtp_rr = libf.mtp_rr
mtp_rr.argtypes = [c_double_p, c_double_p]
mtp_rr.restype = c_double # ctypes.c_double
mtp_cc = libf.mtp_cc
mtp_cc.argtypes = [c_double_complex_p, c_double_complex_p]
mtp_cc.restype = c_double_complex # custom c_double_complex
print_carr = libf.print_carr
print_carr.argtypes = [c_long_p, c_double_complex_ndp]
print_carr.restype = None
## finally we can prepare some data and test what we got
print("test print_c")
c = 5.99j+3.1234567890123456789
print_c(b_double_complex(c)) # directly call c/fortran function
print(c)
print("\ntest mtp_rr")
d1 = 2.2
d2 = 3.3
res_d = mtp_rr(b_double(d1),b_double(d2))
print("d1*d2 =", res_d)
print("\ntest mtp_cc")
c1 = 2j+3
c2 = 3j
res = mtp_cc(b_double_complex(c1), b_double_complex(c2))
print(res.value)
print("\ntest print_carr")
n = 10
arr = np.arange(n) + 10j * np.arange(10)
print("arr.shape =",arr.shape)
print_carr(b_long(n), arr)
arr = arr.reshape((5,2)) # still contiguous chunk of memory
print("arr.shape =",arr.shape)
print_carr(b_long(n), arr) # same result
print("done")
and everything works. I have no idea why something like this is not implemented yet in ctypes.
Also related to other suggestions here: you can pass complex numbers by making new ctypes type
__c_double_complex_p = ct.POINTER(2*ct.c_double)
dll.some_function.argtypes = [__c_double_complex_p, ...]
but you can not retrieve results in that way! To set proper restype only method via class worked for me.
As noted by the OP in their comment on #Armin Rigo's answer, the correct way to do this is with wrapper functions. Also, as noted in the comments (by me) on the original question, this is also possible for C++ and Fortran. However, the method for getting this to work in all three languages is not necessarily obvious. Therefore, this answer presents a working example for each language.
Let's say you have a C/C++/Fortran procedure that takes a scalar complex argument. Then you would need to write a wrapper procedure that takes two floats / doubles (the real and imaginary parts), combines them into a complex number and then calls the original procedure. Obviously, this can be extended to arrays of complex numbers but let's keep it simple with a single complex number for now.
C
For example, let's say you have a C function to print a formatted complex number. So we have my_complex.c:
#include <stdio.h>
#include <complex.h>
void print_complex(double complex z)
{
printf("%.1f + %.1fi\n", creal(z), cimag(z));
}
Then we would have to add a wrapper function (at the end of the same file) like this:
void print_complex_wrapper(double z_real, double z_imag)
{
double complex z = z_real + z_imag * I;
print_complex(z);
}
Compile that into a library in the usual way:
gcc -shared -fPIC -o my_complex_c.so my_complex.c
Then call the wrapper function from Python:
from ctypes import CDLL, c_double
c = CDLL('./my_complex_c.so')
c.print_complex_wrapper.argtypes = [c_double, c_double]
z = complex(1.0 + 1j * 2.0)
c.print_complex_wrapper(c_double(z.real), c_double(z.imag))
C++
The same thing in C++ is a bit more fiddly because an extern "C" interface needs to be defined to avoid name-mangling and we need to deal with the class in Python (both as per this SO Q&A).
So, now we have my_complex.cpp (to which I have already added the wrapper function):
#include <stdio.h>
#include <complex>
class ComplexPrinter
{
public:
void printComplex(std::complex<double> z)
{
printf("%.1f + %.1fi\n", real(z), imag(z));
}
void printComplexWrapper(double z_real, double z_imag)
{
std::complex<double> z(z_real, z_imag);
printComplex(z);
}
};
We also need to add an extern "C" interface (at the end of the same file) like this:
extern "C"
{
ComplexPrinter* ComplexPrinter_new()
{
return new ComplexPrinter();
}
void ComplexPrinter_printComplexWrapper(ComplexPrinter* printer, double z_real, double z_imag)
{
printer->printComplexWrapper(z_real, z_imag);
}
}
Compile into a library in the usual way:
g++ -shared -fPIC -o my_complex_cpp.so my_complex.cpp
And call the wrapper from Python:
from ctypes import CDLL, c_double
cpp = CDLL('./my_complex_cpp.so')
cpp.ComplexPrinter_printComplexWrapper.argtypes = [c_double, c_double]
class ComplexPrinter:
def __init__(self):
self.obj = cpp.ComplexPrinter_new()
def printComplex(self, z):
cpp.ComplexPrinter_printComplexWrapper(c_double(z.real), c_double(z.imag))
printer = ComplexPrinter()
z = complex(1.0 + 1j * 2.0)
printer.printComplex(z)
Fortran
And finally in Fortran we have the original subroutine in my_complex.f90:
subroutine print_complex(z)
use iso_c_binding, only: c_double_complex
implicit none
complex(c_double_complex), intent(in) :: z
character(len=16) :: my_format = "(f4.1,a3,f4.1,a)"
print my_format, real(z), " + ", aimag(z), "i"
end subroutine print_complex
To which we add the wrapper function (at the end of the same file):
subroutine print_complex_wrapper(z_real, z_imag) bind(c, name="print_complex_wrapper")
use iso_c_binding, only: c_double, c_double_complex
implicit none
real(c_double), intent(in) :: z_real, z_imag
complex(c_double_complex) :: z
z = cmplx(z_real, z_imag)
call print_complex(z)
end subroutine print_complex_wrapper
Then compile into a library in the usual way:
gfortran -shared -fPIC -o my_complex_f90.so my_complex.f90
And call from Python (note the use of POINTER):
from ctypes import CDLL, c_double, POINTER
f90 = CDLL('./my_complex_f90.so')
f90.print_complex_wrapper.argtypes = [POINTER(c_double), POINTER(c_double)]
z = complex(1.0 + 1j * 2.0)
f90.print_complex_wrapper(c_double(z.real), c_double(z.imag))
I think that you mean the C99 complex types, e.g. "_Complex double". These types are not natively supported by ctypes. See for example the discussion there: https://bugs.python.org/issue16899
My problem was to pass a numpy array of complex values from Python to C++.
The steps followed where:
Creating a numpy array in Python as
data=numpy.ascontiguousarray(data,dtype=complex)
Creating a double pointer to the data.
c_double_p = ctypes.POINTER(ctypes.c_double)
data_p = data.ctypes.data_as(c_double_p)
Obtaining the value in C++ by defining the parameter as
extern "C" void sample(std::complex < double > *data_p)
If c_complex is a C struct and you have the definition of in documentation or a header file you could utilize ctypes to compose a compatible type. It is also possible, although less likely, that c_complex is a typdef for a type that ctypes already supports.
More information would be needed to provide a better answer...
Use c_double or c_float twice, once for real and once for imaginary.
For example:
from ctypes import c_double, c_int
dimType = c_int * 1
m = dimType(2)
n = dimType(10)
complexArrayType = (2 * c_double) * ( n * m )
complexArray = complexArrayType()
status = yourLib.libcall(m,n,complexArray)
on the library side (fortran example):
subroutine libcall(m,n,complexArrayF) bind(c, name='libcall')
use iso_c_binding, only: c_double_complex,c_int
implicit none
integer(c_int) :: m, n
complex(c_double_complex), dimension(m,n) :: complexArrayF
integer :: ii, jj
do ii = 1,m
do jj = 1,n
complexArrayF(ii,jj) = (1.0, 0.0)
enddo
enddo
end subroutine
Coming from MATLAB, I am looking for some way to create functions in Python which are derived from wrapping C functions. I came across Cython, ctypes, SWIG. My intent is not to improve speed by any factor (it would certainly help though).
Could someone recommend a decent solution for such a purpose.
Edit: What's the most popular/adopted way of doing this job?
Thanks.
I've found that weave works pretty well for shorter functions and has a very simple interface.
To give you an idea of just how easy the interface is, here's an example (taken from the PerformancePython website). Notice how multi-dimensional array conversion is handled for you by the converter (in this case Blitz).
from scipy.weave import converters
def inlineTimeStep(self, dt=0.0):
"""Takes a time step using inlined C code -- this version uses
blitz arrays."""
g = self.grid
nx, ny = g.u.shape
dx2, dy2 = g.dx**2, g.dy**2
dnr_inv = 0.5/(dx2 + dy2)
u = g.u
code = """
#line 120 "laplace.py" (This is only useful for debugging)
double tmp, err, diff;
err = 0.0;
for (int i=1; i<nx-1; ++i) {
for (int j=1; j<ny-1; ++j) {
tmp = u(i,j);
u(i,j) = ((u(i-1,j) + u(i+1,j))*dy2 +
(u(i,j-1) + u(i,j+1))*dx2)*dnr_inv;
diff = u(i,j) - tmp;
err += diff*diff;
}
}
return_val = sqrt(err);
"""
# compiler keyword only needed on windows with MSVC installed
err = weave.inline(code,
['u', 'dx2', 'dy2', 'dnr_inv', 'nx', 'ny'],
type_converters=converters.blitz,
compiler = 'gcc')
return err
after having no success with my question on How to use float ** in Python with Swig?, I started thinking that swig might not be the weapon of choice. I need bindings for some c functions. One of these functions takes a float**. What would you recomend? Ctypes?
Interface file:
extern int read_data(const char *file,int *n_,int *m_,float **data_,int **classes_);
I've used ctypes for several projects now and have been quite happy with the results. I don't think I've personally needed a pointer-to-pointer wrapper yet but, in theory, you should be able to do the following:
from ctypes import *
your_dll = cdll.LoadLibrary("your_dll.dll")
PFloat = POINTER(c_float)
PInt = POINTER(c_int)
p_data = PFloat()
p_classes = PInt()
buff = create_string_buffer(1024)
n1 = c_int( 0 )
n2 = c_int( 0 )
ret = your_dll.read_data( buff, byref(n1), byref(n2), byref(p_data), byref(p_classes) )
print('Data: ', p_data.contents)
print('Classes: ', p_classes.contents)