I have an numpy-array (arr) of type integer which is passed to a C-function with ctypes. I narrowed it down to a simple example. I have this three functions:
int fun(void *arr, void *arr_out){
fun2(arr,arr_out);
return(1);
}
void fun2(int *arr, double *arr_out){
for(int i = 0; i<10; i++){
*(arr_out+i) = calc((arr+i));
}
}
double calc(int *in){
return (double)((log(*in)*-0.15)/1.76);
}
Do I use the pointers in right way? My outputs looks kind of random. As if the pointer itself is in someway part of the calculation and not the value?
Related
How to convert a C++ function that return pointer? Just like:
int * getRandom( )
{
static int r[10];
srand( (unsigned)time( NULL ) );
for (int i = 0; i < 10; ++i)
{
r[i] = rand();
cout << r[i] << endl;
}
return r;
}
How should I bind to convert this C++ function to python?
m.def("getRandom", &getRandom, py::return_value_policy::reference);
The above method can only return the first number of the array. In python side, How should I do to get the array value with length of 10 that function getRandom generate in C++ code.
Like this
#include <pybind11/numpy.h>
// ...
pybind11::array_t<int> getRandomWrapper(void)
{
int* values = getRandom();
pybind11::capsule cleanup(values, [](void *f) {});
return pybind11::array_t<int>(
{10}, // shape
{sizeof(int)}, // stride
values, // pointer to data
cleanup // garbage collection callback
);
}
and then just
m.def("getRandom", &getRandomWrapper);
(return_value_policy::reference is not appropriate here as you are not directly referencing r.)
The fourth argument to the array_t<int> constructor is a wrapper or "capsule" around a function that describes how to free the array's underlying memory. In this case you are returning a pointer to something managed by "automatic storage" so there is nothing to cleanup, hence it does nothing.
If, on the other hand, you were dynamically allocating the returned array you would need something like
pybind11::capsule cleanup(values, [](void *f) {
delete[] static_cast<int*>(values);
});
I've created a simple encryption program in D, and I had the idea to make a DLL from it and try to import it to, for example, Python.
I've could simply call my main function, becouse it dosn't need any params. But when I get to my encrytion method, it uses dynamic-lenght ubyte[] arrays, but as far as I know, they don't exist in other C/C++ based langs.
For example, there's the first line of one of my funcs:
ubyte[] encode(ubyte[] data, ubyte[] key){
But I can't use an array without fixed lenght in other languages!
How can I import that function, for example, in Python?
EDIT:
I know that I can create a wrapper that takes a pointer and the lenght of the array, but isn't there a more elegant solution?
(Where I don't need to use D to use a lib written in D)
Well tbh. there's no real elegant way other than wrapping a pointer with a length or wrapping to C arrays and then to D.
However you can make a somewhat elegant purpose with the first way using a struct that has a pointer, a length and a property that converts it to a D array.
Then the function you export takes your struct, all that function should do is call an internal function that takes an actual D array and you'd simply pass the array to it and the conversion would happen at that moment through alias this and the conversion property.
An example usage is here:
module main;
import core.stdc.stdlib : malloc;
import std.stdio;
struct DArray(T) {
T* data;
size_t length;
/// This field can be removed, only used for testing purpose
size_t offset;
#property T[] array() {
T[] arr;
foreach(i; 0 .. length) {
arr ~= data[i];
}
return arr;
}
alias array this;
/// This function can be removed, only used for testing purpose
void init(size_t size) {
data = cast(T*)malloc(size * T.sizeof);
length = size;
}
/// This function can be removed, only used for testing purpose
void append(T value) {
data[offset] = value;
offset++;
}
}
// This function is the one exported
void externalFoo(DArray!int intArray) {
writeln("Calling extern foo");
internalFoo(intArray);
}
// This function is the one you use
private void internalFoo(int[] intArray) {
writeln("Calling internal foo");
writeln(intArray);
}
void main() {
// Constructing our test array
DArray!int arrayTest;
arrayTest.init(10);
foreach (int i; 0 .. 10) {
arrayTest.append(i);
}
// Testing the exported function
externalFoo(arrayTest);
}
Here is an absolute minimum version of how to do it
struct DArray(T) {
T* data;
size_t length;
#property T[] array() {
T[] arr;
foreach(i; 0 .. length) {
arr ~= data[i];
}
return arr;
}
alias array this;
}
// This function is the one exported
void externalFoo(DArray!int intArray) {
writeln("Calling extern foo");
internalFoo(intArray);
}
// This function is the one you use
private void internalFoo(int[] intArray) {
writeln("Calling internal foo");
writeln(intArray);
}
my C program needs a char** input which I store in python as a numpy object array of strings.
a = np.empty(2, dtype=object)
a[0] = 'hi you'
a[1] = 'goodbye'
What is the correct way to pass this to my C program considering that numpy.i only defines typemaps for char* arrays?
That's impossibru AFAIK, and as far as the docs go:
Some data types are not yet supported, like boolean arrays and string arrays.
You'll either have to write an intermediary function that takes the strings as separate arguments, put them in an array and pass that to your C function, or work out another way of doing things
So it is doable, but you need to convert the numpy object array to a list of python strings with a.tolist(). Then you can pass it to the C code with the following tutorial code as a char **
http://www.swig.org/Doc1.3/Python.html#Python_nn59
Edit: Turned out to be a real pain in the *** since the example above is for Python 2 but gives useless error messages in Python 3. Python 3 moved to unicode strings and I had to do some doc reading to make it work. Here is the python 3 equivalent of the above example.
// This tells SWIG to treat char ** as a special case
%typemap(in) char ** {
/* Check if is a list */
if (PyList_Check($input)) {
int size = PyList_Size($input);
Py_ssize_t i = 0;
$1 = (char **) malloc((size+1)*sizeof(char *));
for (i = 0; i < size; i++) {
PyObject *o = PyList_GetItem($input,i);
if (PyUnicode_Check(o))
$1[i] = PyUnicode_AsUTF8(PyList_GetItem($input,i));
else {
//PyErr_SetString(PyExc_TypeError,"list must contain strings");
PyErr_Format(PyExc_TypeError, "list must contain strings. %d/%d element was not string.", i, size);
free($1);
return NULL;
}
}
$1[i] = 0;
} else {
PyErr_SetString(PyExc_TypeError,"not a list");
return NULL;
}
}
// This cleans up the char ** array we malloc'd before the function call
%typemap(freearg) char ** {
free((char *) $1);
}
Essentially just had to replace PyString_Check with PyUnicode_Check and PyString_AsString with PyUnicode_AsUTF8 (introduced in python 3.3 and later)
I'm new to swig and I have the following function which i cant fix:
int get_list(IN const char * string, OUT struct entry ** results);
where struct entry is defined:
struct flux_entry
{
char * addr_str;
char cc[2];
};
the entry struct is properly converted to a python class.
I googled but couldn't find any explanation i could use.
I want to make it return a tuple of: (original get_list int return value, python list of entry python objects, based on the results buffer), but don't know how to convert the C entry to a python object in the argout code snippet.
I've managed to get thus far:
%typemap(argout) struct entry **
{
PyObject *o = PyList_New(0);
int i;
for(i=0; $1[i] ; i++)
{
PyList_Append(o, SWIG_HOW_TO_CONVERT_TO_PYOBJECT($1[i]));
}
$result = o;
}
what should i replace SWIG_HOW_TO_CONVERT_TO_PYOBJECT with?
passed results is supposed to be a pointer to a (struct entry *) type, set to NULL before calling get_list and should be set to an allocated array of struct entry * pointers. maybe a small wrapper function could make that easier?
the struct entry array is allocated within the C function using malloc, after calculating (inside get_list) how many elements are needed, and ends with a NULL pointer to indicate the end of the array.
i'd also like to make sure it's freed somewhere :)
thanks!
This should at least give you a starting point that works. I still wasn't sure how the data was returned, since to return an array of pointers so that the final one was NULL I'd think you'd need a struct entry ***, so I just set addr_str = NULL on the last one as a sentinel, and just put some dummy data partially based on the input string into the fields. Modify as needed to suit your needs:
%module example
// Insert the structure definition and function to wrap into the wrapper code.
%{
struct entry {
char* addr_str;
char cc[2];
};
int get_list(const char* string, struct entry** results)
{
*results = malloc(3 * sizeof(struct entry));
(*results)[0].addr_str = malloc(10);
strcpy((*results)[0].addr_str,"hello");
(*results)[0].cc[0] = string[0];
(*results)[0].cc[1] = string[1];
(*results)[1].addr_str = malloc(10);
strcpy((*results)[1].addr_str,"there");
(*results)[1].cc[0] = string[2];
(*results)[1].cc[1] = string[3];
(*results)[2].addr_str = NULL;
return 0;
}
%}
#include <typemaps.i>
// Define the structure for SWIG
struct entry {
char* addr_str;
char cc[2];
};
// Define a set of typemaps to be used for an output parameter.
// This typemap suppresses requiring the parameter as an input.
// A temp variable is created and passed instead.
%typemap(in,numinputs=0) struct entry **OUTPUT (struct entry* temp) %{
$1 = &temp;
%}
// Build a list of tuples containing the two entries from the struct.
// Append the new Python list object to the existing "int" result.
%typemap(argout) struct entry **OUTPUT {
int i = 0;
PyObject* out = PyList_New(0);
while((*$1)[i].addr_str != NULL)
{
//PyObject* t = PyTuple_New(2);
//PyTuple_SET_ITEM(t,0,PyBytes_FromString((*$1)[i].addr_str));
//PyTuple_SET_ITEM(t,1,PyBytes_FromStringAndSize((*$1)[i].cc,2));
//PyList_Append(out,t);
//Py_DECREF(t);
PyObject* s = SWIG_NewPointerObj(*$1+i,$descriptor(struct entry*),0);
PyList_Append(out,s);
Py_DECREF(s);
++i;
}
$result = SWIG_AppendOutput($result,out);
}
// Since a Python object was created and the data copied for each entry struct,
// free the memory returned in the structure.
//%typemap(freearg) struct entry **OUTPUT {
// int i=0;
// while((*$1)[i].addr_str != NULL) {
// free((*$1)[i].addr_str);
// ++i;
// }
// free(*$1);
//}
// Apply the OUTPUT typemap set to the "results" parameter.
%apply struct entry **OUTPUT {struct entry** results};
// Finally, define the function for SWIG
int get_list(const char* string, struct entry** results);
Demo (Python 3.3):
>>> import example
>>> example.get_list('abcd')
[0, [(b'hello', b'ab'), (b'there', b'cd')]]
Hope that helps.
Edit:
I commented out the tuple creation and just save the entry* proxy instead. This doesn't leak Python objects, but the memory malloced for use by an entry* is not freed. I'm not sure where to put that, although I'm experimenting with %extend.
While attempting to read a Python list filled with float numbers and to populate real channels[7] with their values (I'm using F2C, so real is just a typedef for float), all I am able to retrieve from it are zero values. Can you point out the error in the code below?
static PyObject *orbital_spectra(PyObject *self, PyObject *args) {
PyListObject *input = (PyListObject*)PyList_New(0);
real channels[7], coefficients[7], values[240];
int i;
if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &input)) {
return NULL;
}
for (i = 0; i < PyList_Size(input); i++) {
printf("%f\n", PyList_GetItem(input, (Py_ssize_t)i)); // <--- Prints zeros
}
//....
}
PyList_GetItem will return a PyObject*. You need to convert that to a number C understands. Try changing your code to this:
printf("%f\n", PyFloat_AsDouble(PyList_GetItem(input, (Py_ssize_t)i)));
Few things I see in this code.
You leak a reference, don't create that empty list at the beginning, it's not needed.
You don't need to cast to PyListObject.
PyList_GetItem returns a PyObject, not a float. Use PyFloat_AsDouble to extract the value.
If PyList_GetItem returns NULL, then an exception has been thrown, and you should check for it.