We Keep Coding

How to use Shared Library in PyBind11

I am trying to use PyBind11 to create a C++ Wrapper to access my USB device on Ubuntu 22.04. I have working c++ program that does this .
#include <iostream>
#include "FUTEK_USB_DLL.h"
#include <string.h>
#include <stdio.h>
#include <ctime>
#include <ratio>
#include <chrono>
#include <unistd.h>
#include <fstream>
#include <cmath>
using namespace std;
int main()
{
_FUTEK_USB_DLL::FUTEK_USB_DLL dll;
PVOID deviceHandle ;
BYTE channelNumber = 0;
string sn = dll.Get_Device_Serial_Number(0);
char *serialNumber = new char[sn.length() + 1];
strcpy(serialNumber, sn.c_str());
dll.Open_Device_Connection(serialNumber);
deviceHandle = dll.DeviceHandle;
string boardType = dll.Get_Type_of_Board(deviceHandle);
string firmVersion = dll.Get_Firmware_Version(deviceHandle);
string samplingRate = dll.Get_ADC_Sampling_Rate(deviceHandle, channelNumber);
double offsetValue = stod(dll.Get_Offset_Value(deviceHandle, channelNumber));
double fullScaleLoad = stod(dll.Get_Fullscale_Load(deviceHandle, channelNumber));
double offsetLoad = stod(dll.Get_Offset_Load(deviceHandle, channelNumber));
double fullScaleValue = stod(dll.Get_Fullscale_Value(deviceHandle, channelNumber));
double decimalPoint = stod(dll.Get_Decimal_Point(deviceHandle, channelNumber));
cout << "Get_Offset_Value: " << offsetValue << endl;
cout << "Get_Fullscale_Load: " << fullScaleLoad << endl;
cout << "Get_Offset_Load: " << offsetLoad << endl;
cout << "Get_Fullscale_Value: " << fullScaleValue << endl;
cout << "Get_Decimal_Point: " << decimalPoint << endl;
while(true)
{
double normalDataRequest = stod(dll.Normal_Data_Request(deviceHandle, channelNumber));
//cout << "Normal_Data_Request: " << normalDataRequest << endl;
float torque = (normalDataRequest - offsetValue)*((fullScaleLoad - offsetLoad)/(fullScaleValue-offsetValue)) / pow(10, decimalPoint);
cout << "torque: " << torque << endl;
usleep((unsigned int)500000);
}
dll.Close_Device_Connection(serialNumber);
}
This code uses two external shared libraries that I link when compiling.
all: main.cpp
g++ -D _GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -Wall main.cpp -o main -lFUTEK_USB-x86_64 -lftd2xx
I am now trying to use PyBind11 to be able to call this code from Python. I am new to PyBind11, but I did get the "hello world" example to compile and work from being called from python. My issue is I am not sure how to link these two dependencies (FUTEK_USB and lftd2xx) in my module using CMake. Here is my CMakeLists.txt
cmake_minimum_required(VERSION 3.4)
project(pybind_test)
find_library(futekLib FUTEK_USB-x86_64)
if (${futekLib} STREQUAL futekLib-NOTFOUND)
message(FATAL_ERROR "No FutekLib, sad day")
else()
message(STATUS "futeklib Found! as ${futekLib}")
endif()
find_library(ftd2xxLib ftd2xx)
if (${ftd2xxLib} STREQUAL ftd2xxLib-NOTFOUND)
message(FATAL_ERROR "No ftd2xxLib, sad day")
else()
message(STATUS "ftd2xxLib Found! as ${ftd2xxLib}")
endif()
add_subdirectory(pybind11)
include_directories(thirdparty)
pybind11_add_module(bind_test main.cpp)
target_link_libraries(bind_test PUBLIC ${futekLib} ${ftd2xxLib})
By adding those message statements to my CMakeLists.txt I can confirm that it does find those two libraries. When i build this python module and then try to import it, I keep getting this undefined reference
<ipython-input-1-a9d1c7c9c951> in <module>
----> 1 import bind_test
ImportError: /home/dyno/Downloads/futek_usb_linux_x86_64/FUTEK_USB-x86_64/bind_test.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN14_FUTEK_USB_DLL13FUTEK_USB_DLL21Get_ADC_Sampling_RateB5cxx11EPvh
Here is my PyBind11 file im trying to run
#include <pybind11/pybind11.h>
#include <iostream>
#include "FUTEK_USB_DLL.h"
#include <string.h>
#include <stdio.h>
#include <ctime>
#include <ratio>
#include <chrono>
#include <unistd.h>
#include <fstream>
namespace py = pybind11;
using namespace std;
std::string get_torque(){
_FUTEK_USB_DLL::FUTEK_USB_DLL dll;
PVOID deviceHandle ;
BYTE channelNumber = 0;
string sn = dll.Get_Device_Serial_Number(0);
char *serialNumber = new char[sn.length() + 1];
strcpy(serialNumber, sn.c_str());
dll.Open_Device_Connection(serialNumber);
deviceHandle = dll.DeviceHandle;
string boardType = dll.Get_Type_of_Board(deviceHandle);
string firmVersion = dll.Get_Firmware_Version(deviceHandle);
string samplingRate = dll.Get_ADC_Sampling_Rate(deviceHandle, channelNumber);
string adc = dll.Normal_Data_Request(deviceHandle, channelNumber);
return adc;
}
PYBIND11_MODULE(bind_test, handle){
handle.doc()="TEST";
handle.def("python_name", &get_torque);
}
Any help would be greatly appreciated as I am very stuck on this issue. Here is the repo with the full code https://github.com/RavenLabsNH/FutekTorqueTransducer

How can I implement closures in C?

I want to implement the following function in c. I have written three different programming language's implementations of the function:
Python
def get_add(x):
def add(y):
return x + y
return add
add5 = get_add(5)
add5(10) # 15
add10 = get_add(10)
add10(10) # 20
JS
function get_add(x) {
return (y)=>{
return x + y
}
}
add5 = get_add(5)
add5(10) // 15
add10 = get_add(10)
add10(10) // 20
lua
function get_add(num)
return function(a) return a+num end
end
add5 = get_add(5)
add5(10) -- 15
add10 = get_add(10)
add10(10) -- 20
I cannot think of a way to implement this. Maybe this can be implemented using hash table in some manner? Or function pointers?

I would be fairly confident in the claim that there is no way to do this portably in strictly conforming C. That said, if you're willing to make some generous assumptions about how the specific implementation you're using works, you could do this by allocating memory for new code (or at least the data the closure would capture), marking it as executable, and then engaging in some standards-violating pointer casting. An example that at least works on my machine (x86-64 Linux) would be:
#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
uint8_t (*add(uint8_t x))(uint8_t) {
// lea eax, [rdi + x]
// ret
char code[] = { 0x8D, 0x47, x, 0xC3 };
char *p = mmap(0, sizeof code, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0);
memcpy(p, code, sizeof code);
return (uint8_t(*)(uint8_t))p;
}
int main(void) {
uint8_t (*add5)(uint8_t) = add(5);
printf(" 5 + 10 = %" PRIu8 "\n", add5(10));
printf("10 + 10 = %" PRIu8 "\n", add(10)(10));
return 0;
}
But as noted, this is non-portable at best, and is definitely not anywhere close to idiomatic C.
There are ways to do equivalent things to this in a standards-conforming manner, like storing the data you would capture in a struct and passing that to a different function, but in terms of doing it transparently with functions, I think this is about the best you can do.
So is there any other way to implement it?
Yes, but without external dependencies it can begin to get a little unwieldy.
One option is something like this, where for each different function you create a struct with all the variables you would want to capture and pass that as an argument:
#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
struct add {
uint8_t x;
};
struct add get_add(uint8_t x) {
return (struct add) {
.x = x,
};
}
uint8_t add(struct add info, uint8_t y) {
return info.x + y;
}
int main(void) {
struct add add5 = get_add(5);
printf(" 5 + 10 = %" PRIu8 "\n", add(add5, 10));
printf("10 + 10 = %" PRIu8 "\n", add(get_add(10), 10));
return 0;
}
This becomes a bit verbose if you want a function that has more than a couple curried arguments.
#SteveSummit's suggestion in the comments about libffi is also good. With their closures API, here's an example of what you might want:
#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <ffi.h>
void _add(ffi_cif *cif, void *ret, void *args[], void *x) {
*(ffi_arg*)ret = *(uint64_t*)x + *(uint64_t*)args[0];
}
int main(void) {
ffi_cif cif;
ffi_type *args[1];
void *add_code;
ffi_closure *add_closure = ffi_closure_alloc(sizeof *add_closure, &add_code);
uint64_t x = 5;
if (add_closure) {
args[0] = &ffi_type_uint64;
if (ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 1, &ffi_type_uint64, args) == FFI_OK) {
if (ffi_prep_closure_loc(add_closure, &cif, _add, &x, add_code) == FFI_OK) {
printf(" 5 + 10 = %" PRIu64 "\n", ((uint64_t (*)(uint64_t))add_code)(10));
printf(" 5 + 15 = %" PRIu64 "\n", ((uint64_t (*)(uint64_t))add_code)(15));
}
}
}
ffi_closure_free(add_closure);
return 0;
}
I haven't checked, but my guess is their implementation is probably just a more robust wrapper around the first example I give (with additional platform support).

Error in method argv[] SWIG C++ / Python, what happens?

I'm using swig to connect a C ++ function with my Python libraries. I managed to compile everything and create the .so file. But after having done the import of my C ++ function in python I have some errors on the argv ... maybe I'm wrong to pass the values to it from the outside or maybe when I compiled and created the .i swig file I got something wrong. The C ++ function creates a TCP socket and sends a Hex code to a device that replies with another Hex code, from a terminal with the C ++ code i write:
connect_PE_func 192.168.1.170 600000060
and it works perfectly. and it works perfectly. I would expect a similar syntax once I set my c ++ function via swig in python, type:
answer= connect_PE_func.connect_PE_func("192.168.1.170", 600000060,2)
but I get this error:
test=connect_PE_func.connect_pe_func(["192.168.2.170"],["2600000026"])
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-12-f58f79b27fe5> in <module>
----> 1 test=connect_PE_func.connect_pe_func(["192.168.2.170"],["2600000026"])
~/mqtt_atenapy/C_connect_PE_dev/test_cpython/connect_PE_func.py in connect_pe_func(argv, argc)
64
65 def connect_pe_func(argv, argc):
---> 66 return _connect_PE_func.connect_pe_func(argv, argc)
67
68
TypeError: in method 'connect_pe_func', argument 1 of type 'char *[]'
I attach the code to the files .c .h e .i which I used with swig to get the .so.
thanks.
connect_PE.func.c :
#include <stdio.h>
#include <errno.h>
#include <string>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <cstring>
#include <unistd.h>
#include <iostream>
#include <unistd.h>
#include <sstream>
#include "connect_PE_func.h"
using namespace std;
// to compile gcc connect_PE_func.cpp -lstdc++ -c
char* connect_pe_func(char *argv[],int argc)
{
int sockfd, n;
int connected = 0;
struct sockaddr_in servaddr;
std::string serveraddr = argv[1];
sockfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = inet_addr(serveraddr.c_str());
servaddr.sin_port = htons(9761);
connect(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr));
std::string pref_hex;
std::string hex("0x");
std::string test = argv[2];
size_t numbytes = test.size() / 2;
uint8_t command[numbytes];
for (size_t w = 0, x = 0; w < numbytes; ++w, x += 2)
{
pref_hex = hex + test.substr(x, 2);
cout << pref_hex;
command[w] = stoi(pref_hex, nullptr, 16);
//cout << test.substr(x, 2);
//cout << "\n";
//cout << command[w];
//cout << "\n";
}
//uint8_t command[] = {0x26, 0x00, 0x00, 0x00, 0x26};
int bytes_to_send = sizeof(command);
send(sockfd, command, bytes_to_send, 0);
uint8_t output_command[numbytes];
recv(sockfd, output_command, bytes_to_send, 0);
char test_out[10];
for (size_t w = 0, x = 0; w < numbytes; ++w, x += 2)
{
test_out[x] = (char)output_command[w];
//cout << unsigned(test_out[x]);
}
return test_out;
}
connect_PE_func.h:
// file: connect_PE_func.h
char* connect_pe_func(char *argv[], int argc);
connect_PE_func.i:
/* file: connect_PE_func.i */
%module connect_PE_func
%{
/* Everything in this block will be copied in the wrapper file. We include the C header file necessary to compile the interface
*/
#include "connect_PE_func.h"
// extern char *connect_pe_func(int argc, char *argv[]);
%}
/* list functions to be interfaced: */
char* connect_pe_func(char *argv[], int argc);

You've misplaced a comma func(["192.168.2.170"] , ["2600000026"])
My guess is that you need to pass func(["192.168.2.170", "2600000026"], 2)

Can C++ program string search as fast as and/or faster than python?

I'm not sure why I'm having easier time string searching in program I wrote in python faster than a program I wrote in C++. Is there a trick I'm missing?
Generating Use Case
This is for a single line use case, however in the real use case I care about multiple lines.
#include "tchar.h"
#include "stdio.h"
#include "stdlib.h"
#include <string>
#include <sstream>
#include <iostream>
#include <fstream>
#include <ctime>
using namespace std;
void main(void){
ofstream testfile;
unsigned int line_idx = 0;
testfile.open("testfile.txt");
for(line_idx = 0; line_idx < 50000u; line_idx++)
{
if(line_idx != 43268u )
{
testfile << line_idx << " dontcare" << std::endl;
}
else
{
testfile << line_idx << " care" << std::endl;
}
}
testfile.close();
}
The regular expression
Using regular expression ^(\d*)\s(care)$
The C++ Program takes 13.954 seconds
#include "tchar.h"
#include "stdio.h"
#include "stdlib.h"
#include <string>
#include <sstream>
#include <iostream>
#include <fstream>
#include <ctime>
using namespace std;
void main(void){
double duration;
std::clock_t start;
ifstream testfile("testfile.txt", ios_base::in);
unsigned int line_idx = 0;
bool found = false;
string line;
regex ptrn("^(\\d*)\\s(care)$");
start = std::clock(); /* Debug time */
while (getline(testfile, line))
{
std::smatch matches;
if(regex_search(line, matches, ptrn))
{
found = true;
}
}
testfile.close();
duration = ( std::clock() - start ) / (double) CLOCKS_PER_SEC;
std::cout << "Found? " << (found ? "yes" : "no") << std::endl;
std::cout << " Total time: " << duration << std::endl;
}
Python Program takes 0.02200 seconds
import sys, os # to navigate and open files
import re # to search file
import time # to benchmark
ptrn = re.compile(r'^(\d*)\s(care)$', re.MULTILINE)
start = time.time()
with open('testfile.txt','r') as testfile:
filetext = testfile.read()
matches = re.findall(ptrn, filetext)
print("Found? " + "Yes" if len(matches) == 1 else "No")
end = time.time()
print("Total time", end - start)

Implemented Ratah's recommendation to 8.923
about 5 seconds improvement, by reading file to single string
double duration;
std::clock_t start;
ifstream testfile("testfile.txt", ios_base::in);
unsigned int line_idx = 0;
bool found = false;
string line;
regex ptrn("^(\\d*)\\s(care)$");
std::smatch matches;
start = std::clock(); /* Debug time */
std::string test_str((std::istreambuf_iterator<char>(testfile)),
std::istreambuf_iterator<char>());
if(regex_search(test_str, matches, ptrn))
{
found = true;
}
testfile.close();
duration = ( std::clock() - start ) / (double) CLOCKS_PER_SEC;
std::cout << "Found? " << (found ? "yes" : "no") << std::endl;
std::cout << " Total time: " << duration << std::endl;
After UKMonkey's note, reconfigured project to release which also includes \O2 and brought it down to 0.086 seconds
Thanks to Jean-Francois Fabre, Ratah, UKMonkey

How to return output from pyrun_simplefile in c code

The code is
{
char name[MAX_JSON_FIELD];
FILE *fp;
copy_cJSON(name,objs[0]);
if ( (fp= fopen(name, "r")) != 0 )
{
Py_Initialize();
PyRun_SimpleFile(fp, name);
Py_Finalize();
fclose(fp);
}
return(clonestr("return string"));
}
How can I get it to return the output of the python file instead of printing it?

I achieved this using a huge workaround. I made both C and Python read and write into a file. I didn't find a better option yet.
I found an actual solution. It consists of 2 files: "main.c" that opens the script-file "script.py" which compares two strings (here: "Hello" and "Mars") and returns the longer one. I still find it strange that it takes ~20 commands to achieve this, maybe there's a better solution out there.
[main.c]
//compile me with "gcc main.c -I/usr/include/python2.7 -lpython2.7"
//original source: "http://python.haas.homelinux.net/python_kapitel_26_003.htm"
//owner is Peter Kaiser and Johannes Ernesti who published the Book "Python" under Galileo Computing
//Translation from german with many additional (and very unprofessional) comments and slight adaption by Cupacoffee, 17.02.2015.
//bugs, grammar mistakes and wrong explainations are my contribution
#include <Python.h>
int main (int argc, char *argv[])
{
char *result;//This char will receive the return value.
PyObject *module, *func, *prm, *ret;//These are some helping variables i don't understand.
Py_Initialize();
PySys_SetPath(".");//Sets the working path to the current path
module = PyImport_ImportModule("script");//Import of the script-file, note that the actual script name is "script.py"!
if (module != 0)//Asks if the script was loaded at all.
{
func = PyObject_GetAttrString(module, "compare_function");//Opens a function within the python script. Notice that you must use a function within the python script, because otherwise you can't return anything.
prm = Py_BuildValue("(ss)", "Hello", "Mars");//The "(ss)" means two strings are passed (replace with "i" for integer for instance), the "Hello" and "Mars" are the strings i pass to the script.
ret = PyObject_CallObject(func, prm);//Returns some python object i have literally no idea about ...
result = PyString_AsString(ret);// ... but luckily there's a function to cast it back to a c-compatible char*!
printf("The Script decdided that '%s' is longer!",result);
Py_DECREF(module);//cleanup?
Py_DECREF(func);//cleanup?
Py_DECREF(prm);//cleanup?
Py_DECREF(ret);//cleanup?
}
else//No script found
{
printf("Error: No script file named \"script.py\" was found!\n");
}
Py_Finalize();
return 0;
}
[script.py]
def compare_function(a, b):#this function takes 2 parameters, they are strings
return (a if min(a) < min(b) else b)#they get compared and returned to the c-program
Good luck.
*Grumble, took me over 2 hours to format this text so I could post it.*

A friend of mine gave me some code snippets which answer the problem. I didn't want to edit the old post because these two programs are entirely new approaches; One in C and one in C++. Both use the same Python Script.
He also pointed me to the manual page of "system" [Unix: "man system"] and "popen" [Unix: "man popen"]. The second one allows interactive communication, which might be useful later.
[C-File:]
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
int callScript()
{
char *cmd = "./script.py hello world";
return WEXITSTATUS(system(cmd));
}
int main(int argc, char *argv[])
{
printf("main - argc: %d, arguments:\n", argc);
for (int i = 0; i < argc; i++)
printf("\targv[%d]: %s\n", i, argv[i]);
int ret = callScript();
printf("exit code of script %d\n", ret);
return 0;
}
[C++-File:]
#include <string>
#include <sstream>
#include <iostream>
#include <stdlib.h>
#include <sys/wait.h>
int callScript(std::string args)
{
std::string cmd = "./script.py" + args;
int ret = system(cmd.c_str());
return WEXITSTATUS(ret);
}
int main(int argc, char *argv[])
{
std::cout << "main - argc: " << argc << ", arguments:" << std::endl;
std::stringstream args;
for (int i = 0; i < argc; i++)
{
std::cout << "\targv[" << i << "]: " << argv[i] << std::endl;
if (i)
args << " " << argv[i];
}
int ret = callScript(args.str());
std::cout << "exit code of script " << ret << std::endl;
return 0;
}
[Python-Script:]
#!/usr/bin/env python
import sys
def Hello(person):
print "Hello " + person
def PrintArgs(argv):
for arg in argv:
print arg
if __name__ == "__main__":
Hello("World!")
PrintArgs(sys.argv[1:])
sys.exit(2)