I've created a simple encryption program in D, and I had the idea to make a DLL from it and try to import it to, for example, Python.
I've could simply call my main function, becouse it dosn't need any params. But when I get to my encrytion method, it uses dynamic-lenght ubyte[] arrays, but as far as I know, they don't exist in other C/C++ based langs.
For example, there's the first line of one of my funcs:
ubyte[] encode(ubyte[] data, ubyte[] key){
But I can't use an array without fixed lenght in other languages!
How can I import that function, for example, in Python?
EDIT:
I know that I can create a wrapper that takes a pointer and the lenght of the array, but isn't there a more elegant solution?
(Where I don't need to use D to use a lib written in D)
Well tbh. there's no real elegant way other than wrapping a pointer with a length or wrapping to C arrays and then to D.
However you can make a somewhat elegant purpose with the first way using a struct that has a pointer, a length and a property that converts it to a D array.
Then the function you export takes your struct, all that function should do is call an internal function that takes an actual D array and you'd simply pass the array to it and the conversion would happen at that moment through alias this and the conversion property.
An example usage is here:
module main;
import core.stdc.stdlib : malloc;
import std.stdio;
struct DArray(T) {
T* data;
size_t length;
/// This field can be removed, only used for testing purpose
size_t offset;
#property T[] array() {
T[] arr;
foreach(i; 0 .. length) {
arr ~= data[i];
}
return arr;
}
alias array this;
/// This function can be removed, only used for testing purpose
void init(size_t size) {
data = cast(T*)malloc(size * T.sizeof);
length = size;
}
/// This function can be removed, only used for testing purpose
void append(T value) {
data[offset] = value;
offset++;
}
}
// This function is the one exported
void externalFoo(DArray!int intArray) {
writeln("Calling extern foo");
internalFoo(intArray);
}
// This function is the one you use
private void internalFoo(int[] intArray) {
writeln("Calling internal foo");
writeln(intArray);
}
void main() {
// Constructing our test array
DArray!int arrayTest;
arrayTest.init(10);
foreach (int i; 0 .. 10) {
arrayTest.append(i);
}
// Testing the exported function
externalFoo(arrayTest);
}
Here is an absolute minimum version of how to do it
struct DArray(T) {
T* data;
size_t length;
#property T[] array() {
T[] arr;
foreach(i; 0 .. length) {
arr ~= data[i];
}
return arr;
}
alias array this;
}
// This function is the one exported
void externalFoo(DArray!int intArray) {
writeln("Calling extern foo");
internalFoo(intArray);
}
// This function is the one you use
private void internalFoo(int[] intArray) {
writeln("Calling internal foo");
writeln(intArray);
}
Related
How to convert a C++ function that return pointer? Just like:
int * getRandom( )
{
static int r[10];
srand( (unsigned)time( NULL ) );
for (int i = 0; i < 10; ++i)
{
r[i] = rand();
cout << r[i] << endl;
}
return r;
}
How should I bind to convert this C++ function to python?
m.def("getRandom", &getRandom, py::return_value_policy::reference);
The above method can only return the first number of the array. In python side, How should I do to get the array value with length of 10 that function getRandom generate in C++ code.
Like this
#include <pybind11/numpy.h>
// ...
pybind11::array_t<int> getRandomWrapper(void)
{
int* values = getRandom();
pybind11::capsule cleanup(values, [](void *f) {});
return pybind11::array_t<int>(
{10}, // shape
{sizeof(int)}, // stride
values, // pointer to data
cleanup // garbage collection callback
);
}
and then just
m.def("getRandom", &getRandomWrapper);
(return_value_policy::reference is not appropriate here as you are not directly referencing r.)
The fourth argument to the array_t<int> constructor is a wrapper or "capsule" around a function that describes how to free the array's underlying memory. In this case you are returning a pointer to something managed by "automatic storage" so there is nothing to cleanup, hence it does nothing.
If, on the other hand, you were dynamically allocating the returned array you would need something like
pybind11::capsule cleanup(values, [](void *f) {
delete[] static_cast<int*>(values);
});
Let's say we have a following struct in C:
typedef struct buffer
{
uint8_t* const data;
const size_t maxSize;
size_t size;
} buffer_t;
How can I make a SWIG wrapper for this so that when buffer_t is created in Python it allocates given number of bytes to data pointer and sets maxSize accordingly?
Details
So basically the problem here is related to constant struct members. By default SWIG initializes structs with default values. This leads into problems:
StructWithConsts_t struct;
struct.constant = 5; // Error. This should've been set when creating the struct.
The accepted answer with constructors and destructors provide solution for this problem.
You can treat C structs as C++ classes in SWIG and extend them with creators and destroyers methods.
In your case your .i should have this
typedef struct
{
uint8_t* const data;
const size_t maxSize;
size_t size;
} buffer_t;
%extend buffer_t {
buffer_t(size_t size)
{
buffer_t* pBuffer = malloc(sizeof *pBuffer);
pBuffer->data = malloc(size * sizeof (*pBuffer->data));
pBuffer->maxSize = size;
return pBuffer;
}
void ~buffer_t()
{
buffer_t *pBuffer = ($self);
free(pBuffer->data);
free(pBuffer);
return;
}
};
In python it will be used like this
from yourmodule import buffer_t
buffer = buffer_t(10)
The garbage collector will take care of freeing the memory.
There are other ways of doing this but you don't have a reproducing example to test it out.
I found one solution, but it is rather hackish due to pointer typecasts. I'm not convinced that this is the proper way to do this, but it works:
In the .i file I created a wrapper using %inline:
%inline %{
buffer_t* new_buffer(size_t size)
{
buffer_t* pBuffer = (buffer_t*)malloc(sizeof(buffer_t));
*(uint8_t**)&pBuffer->data = (uint8_t*)malloc(size * sizeof(uint8_t));
*(size_t*)&pBuffer->maxSize = size;
return (buffer_t*)pBuffer;
}
void free_buffer(buffer_t* pBuffer)
{
free(pBuffer->data);
free(pBuffer);
return;
}
%}
I have an numpy-array (arr) of type integer which is passed to a C-function with ctypes. I narrowed it down to a simple example. I have this three functions:
int fun(void *arr, void *arr_out){
fun2(arr,arr_out);
return(1);
}
void fun2(int *arr, double *arr_out){
for(int i = 0; i<10; i++){
*(arr_out+i) = calc((arr+i));
}
}
double calc(int *in){
return (double)((log(*in)*-0.15)/1.76);
}
Do I use the pointers in right way? My outputs looks kind of random. As if the pointer itself is in someway part of the calculation and not the value?
So this question is sort of one of translation. I am new to C++, and was looking through the class documentation. However, it looks like finding the answer to my question is a bit hard via the documentation.
I have code for generating a random number between 0 and 1 in C++: (obtained from here, since the rand() function solution for floats is integer based)
#include <random>
#include <iostream>
int main()
{
std::random_device rd;
std::mt19937 gen(rd());
std::uniform_real_distribution<> dis(0, 1); //corrected from 1,2
for (int n = 0; n < 10; ++n) {
std::cout << dis(gen) << ' ';
}
std::cout << '\n';
}
Next, I would like to create a class or struct or something (not really an OOP guy) that has an API like:
float x = my_RandomNumberGenerator.next();
In python, I might write something like:
class my_RNG():
def __init__(self):
self.rd = (the random device object I initialize in c code)
self.gen = (the mersenne_twister engine object)(rd)
self.distribution = (the uniform real distribution object)
def next():
return self.distribution(self.gen)
my_randomNumberGenerator = my_RNG()
print(my_randomNumberGenerator.next())
How would I implement this in C++?
update Here is what I have so far (it does not work... or compile...but there seems to be some strangeness in the way things are initialized in my template code that I got from the reference site that I don't understand):
#include <iostream>
#include <random>
class MyRNG
{
public:
float next(void);
private:
std::random_device randomDevice;
std::mt19937_64 randomGenerator;
std::uniform_real_distribution distribution;
MyRNG(float range_lower,float range_upper);
};
MyRNG::MyRNG(float range_lower, float range_upper)
{
randomGenerator = std::mersenne_twister_engine(randomDevice);
distribution = std::uniform_real_distribution<> distribution(range_lower,range_upper);
}
MyRNG::next(void)
{
return distribution(randomGenerator);
}
int main() {
MyRNG my_rng = MyRNG(0,1);
std::cout << my_rng.next() << std::endl;
return 0;
}
Seems like you just need some form of probability generation class, see below for a basic implementation which meets your question requirements:
template<class Ty = double,
class = std::enable_if_t<std::is_floating_point<Ty>::value>
> class random_probability_generator {
public:
// default constructor uses single random_device for seeding
random_probability_generator()
: mt_eng{std::random_device{}()}, prob_dist(0.0, 1.0) {}
// ... other constructors with custom seeds if necessary
Ty next() { return prob_dist(mt_eng); }
// ... other methods if necessary
private:
std::mt19937 mt_eng;
std::uniform_real_distribution<Ty> prob_dist;
};
Then you can use this simply via:
random_probability_generator<> pgen;
double p = pgen.next(); // double in range [0.0, 1.0]
Or if you want random floats instead (as part of your question seems to imply):
random_probability_generator<float> pgen;
float p = pgen.next(); // float in range [0.0f, 1.0f]
Also, to address why the class you posted isn't compiling, the error in your class is that you try to initialise a std::mt19937_64 type object (randomGenerator) with a std::mersenne_twister_engine instance but they are fundamentally different types. Instead you would need to do
randomGenerator = std::mt19937_64(randomDevice());
in MyRNG constructor, or construct via initialisation list as I have done in the example above.
As pointed out in the comments, a more suitable c++-esque implementation of this is to overload operator() instead of creating a next() method. See below for a better implementation of the above class,
template<class FloatType = double,
class Generator = std::mt19937,
class = std::enable_if_t<std::is_floating_point<FloatType>::value>
> class uniform_random_probability_generator {
public:
typedef FloatType result_type;
typedef Generator generator_type;
typedef std::uniform_real_distribution<FloatType> distribution_type;
// default constructor
explicit uniform_random_probability_generator(Generator&& _eng
= Generator{std::random_device{}()}) : eng(std::move(_eng)), dist() {}
// construct from existing pre-defined engine
explicit uniform_random_probability_generator(const Generator& _eng)
: eng(_eng), dist() {}
// generate next random value in distribution (equivalent to next() in above code)
result_type operator()() { return dist(eng); }
// will always yield 0.0 for this class type
constexpr result_type min() const { return dist.min(); }
// will always yield 1.0 for this class type
constexpr result_type max() const { return dist.max(); }
// resets internal state such that next call to operator()
// does not rely on previous call
void reset_distribution_state() { dist.reset(); }
private:
generator_type eng;
distribution_type dist;
};
Then you can use this similarly to the first class in this answer,
uniform_random_probability_generator<> urpg;
double next_prob = urpg();
Additionally, uniform_random_probability_generator can use a different Generator type as a template parameter so long as this type meets the requirements of UniformRandomBitGenerator. For example, if for any reason you needed to use std::knuth_b instead of std::mt19937 then you can do so as follows:
uniform_random_probability_generator<double, std::knuth_b> urpg_kb;
double next_prob = urpg_kb();
You can create a class that holds a random number generator as a private member variable (like std::mt19937) and seeds it in the constructor. Your next function could just invoke the stored generator to get the next value (applying whatever distribution you want (if any) of course).
This is not very complicated, so I'm afraid I'm missing the real point of your question..
I'm new to swig and I have the following function which i cant fix:
int get_list(IN const char * string, OUT struct entry ** results);
where struct entry is defined:
struct flux_entry
{
char * addr_str;
char cc[2];
};
the entry struct is properly converted to a python class.
I googled but couldn't find any explanation i could use.
I want to make it return a tuple of: (original get_list int return value, python list of entry python objects, based on the results buffer), but don't know how to convert the C entry to a python object in the argout code snippet.
I've managed to get thus far:
%typemap(argout) struct entry **
{
PyObject *o = PyList_New(0);
int i;
for(i=0; $1[i] ; i++)
{
PyList_Append(o, SWIG_HOW_TO_CONVERT_TO_PYOBJECT($1[i]));
}
$result = o;
}
what should i replace SWIG_HOW_TO_CONVERT_TO_PYOBJECT with?
passed results is supposed to be a pointer to a (struct entry *) type, set to NULL before calling get_list and should be set to an allocated array of struct entry * pointers. maybe a small wrapper function could make that easier?
the struct entry array is allocated within the C function using malloc, after calculating (inside get_list) how many elements are needed, and ends with a NULL pointer to indicate the end of the array.
i'd also like to make sure it's freed somewhere :)
thanks!
This should at least give you a starting point that works. I still wasn't sure how the data was returned, since to return an array of pointers so that the final one was NULL I'd think you'd need a struct entry ***, so I just set addr_str = NULL on the last one as a sentinel, and just put some dummy data partially based on the input string into the fields. Modify as needed to suit your needs:
%module example
// Insert the structure definition and function to wrap into the wrapper code.
%{
struct entry {
char* addr_str;
char cc[2];
};
int get_list(const char* string, struct entry** results)
{
*results = malloc(3 * sizeof(struct entry));
(*results)[0].addr_str = malloc(10);
strcpy((*results)[0].addr_str,"hello");
(*results)[0].cc[0] = string[0];
(*results)[0].cc[1] = string[1];
(*results)[1].addr_str = malloc(10);
strcpy((*results)[1].addr_str,"there");
(*results)[1].cc[0] = string[2];
(*results)[1].cc[1] = string[3];
(*results)[2].addr_str = NULL;
return 0;
}
%}
#include <typemaps.i>
// Define the structure for SWIG
struct entry {
char* addr_str;
char cc[2];
};
// Define a set of typemaps to be used for an output parameter.
// This typemap suppresses requiring the parameter as an input.
// A temp variable is created and passed instead.
%typemap(in,numinputs=0) struct entry **OUTPUT (struct entry* temp) %{
$1 = &temp;
%}
// Build a list of tuples containing the two entries from the struct.
// Append the new Python list object to the existing "int" result.
%typemap(argout) struct entry **OUTPUT {
int i = 0;
PyObject* out = PyList_New(0);
while((*$1)[i].addr_str != NULL)
{
//PyObject* t = PyTuple_New(2);
//PyTuple_SET_ITEM(t,0,PyBytes_FromString((*$1)[i].addr_str));
//PyTuple_SET_ITEM(t,1,PyBytes_FromStringAndSize((*$1)[i].cc,2));
//PyList_Append(out,t);
//Py_DECREF(t);
PyObject* s = SWIG_NewPointerObj(*$1+i,$descriptor(struct entry*),0);
PyList_Append(out,s);
Py_DECREF(s);
++i;
}
$result = SWIG_AppendOutput($result,out);
}
// Since a Python object was created and the data copied for each entry struct,
// free the memory returned in the structure.
//%typemap(freearg) struct entry **OUTPUT {
// int i=0;
// while((*$1)[i].addr_str != NULL) {
// free((*$1)[i].addr_str);
// ++i;
// }
// free(*$1);
//}
// Apply the OUTPUT typemap set to the "results" parameter.
%apply struct entry **OUTPUT {struct entry** results};
// Finally, define the function for SWIG
int get_list(const char* string, struct entry** results);
Demo (Python 3.3):
>>> import example
>>> example.get_list('abcd')
[0, [(b'hello', b'ab'), (b'there', b'cd')]]
Hope that helps.
Edit:
I commented out the tuple creation and just save the entry* proxy instead. This doesn't leak Python objects, but the memory malloced for use by an entry* is not freed. I'm not sure where to put that, although I'm experimenting with %extend.