I am developing a Python module using SWIG and linking with a certain library.
I have a typemap like the following:
%typemap(in) MyStringType {
char* buffer = 0;
size_t size = 0;
int alloc = SWIG_OLDOBJ;
int res = SWIG_AsCharPtrAndSize( $input, &buffer, &size, &alloc );
if (!SWIG_IsOK(res)) {
SWIG_exception_fail(SWIG_TypeError, "in method '$symname', expecting string");
}
$1.p = buffer;
$1.n = size - 1;
}
I have some function that takes as parameter MyStringType as the parameter:
void myFunc(MyStringType type);
Then, when I create the python module I do the following in python 2.7:
myFunc("Test string")
However, when I try with Python 3.4 I get the following:
TypeError: in method 'myFunc', expecting string
I could solve that by adding this to the swig interface file:
%begin %{
#define SWIG_PYTHON_STRICT_UNICODE_WCHAR
%}
That solved the problem. However I still need to add one more macro because some other functions require it. So, now it is like this:
%begin %{
#define SWIG_PYTHON_STRICT_BYTE_CHAR
#define SWIG_PYTHON_STRICT_UNICODE_WCHAR
%}
This would make me call myFunc like this:
myFunc(b"Test string")
And everything works fine when using tox in my pc, for python 2.7, 3.4 and 3.5, but it fails in travis-ci.
After removing the b before "Test string" it succeeds in travis-ci but fails in my pc.
In travis-ci I have: python: 2.7.6, 3.4.3
In my pc : python: 2.7.13, 3.4.8
Any idea why?
Solved. Thanks for all the hints.
It was different swig versions.
Related
I'm very new to the whole CMake. Following this and this posts, now I want to call a MAXON function inside Python, using pybind11. What I have done so far:
The library can be downloaded from this page (direct download link).
wget https://www.maxongroup.com/medias/sys_master/root/8837358518302/EPOS-Linux-Library-En.zip
unzip:
unzip EPOS-Linux-Library-En.zip
make the install shell script executable and run it:
chmod +x ./install.sh
sudo ./install.sh
Then going to the example folder:
cd /opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd/
Now combining the CMakeLists.txt files from here:
# CMakeLists.txt
cmake_minimum_required(VERSION 2.8.12)
project (HelloEposCmd)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -Wall")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -Wall")
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
find_package(pybind11 REQUIRED)
pybind11_add_module(${PROJECT_NAME} HelloEposCmd.cpp)
add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} -lEposCmd)
and the HelloEposCmd.cpp this line is added right after other header files:
#include <pybind11/pybind11.h>
the main function is renamed to:
int run(int argc, char** argv)
and the pybind11 syntax to add the module is written at the end:
PYBIND11_MODULE(HelloEposCmd, m) {
m.def("run", &run, "runs the HelloEposCmd");
}
However, When I run the cmake . I get the error:
CMake Error at CMakeLists.txt:13 (add_executable):
add_executable can not create target "HelloEposCmd" because another target with the same name already exists. The existing target is a module library created in source directory "/opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd" See documentation for policy CMP0002 for more details.
...
I was wondering if you could be kind to help me get the right CMakeList.txt file. Ideally, I should be able to call the compiled module in python:
# HelloEposCmd.py
import HelloEposCmd
HelloEposCmd.run()
Thanks for your support in advance.
pybind11_add_module already creates a target for you. So you don't need add_executable anymore. Just remove that line and when you will build you will get a library with the name HelloEposCmd
add_executable is needed if you are building an executable (.exe), which I believe is not what you want.
Documenation of pybind11 says.
This function behaves very much like CMake’s builtin add_library (in fact, it’s a wrapper function around that command).
Thanks to abhilb post and his kind followup in the comments I was able to figure the problem out. well, at least find a temporary workaround:
According to this post, the last two lines of the CMakeLists.txt file should change to
# this line can be removed
# add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} PRIVATE -lEposCmd)
and then because according to this post pybind11 doesn't support double pointers we change the run function to:
int run() {
int argc = 1;
char* argv[] = {"./HelloEposCmd"};
...
}
which I suppose to be a horrible workaround (inspired by information from this page). Now running cmake ., make and python3 HelloEposCmd.py should work properly (except a small c++ warning!).
P.S.1. Maybe someone could use std::vector<std::string> as suggested here. This idea was proposed here and there are already some answers worth investigating.
P.S.2. Following this discussion, another workaround could be something like:
#include <stdio.h>
#include <stdlib.h>
void myFunc(int argc, char* argv[]) {
for (int i = 0; i < argc; ++i) {
printf("%s\n", argv[i]);
}
}
int run(int argc, long* argv_) {
char** argv = (char**)malloc(argc * sizeof(char*));
for (int i = 0; i < argc; ++i) {
argv[i] = (char*)(argv_[i]);
}
myFunc(argc, argv);
free(argv);
return 0;
}
I'm trying to figure out why I can't simply get and set the python path through its C API. I am using Python3.6, on Ubuntu 17.10 with gcc version 7.2.0. Compiling with:
gcc pytest.c `python3-config --libs` `python3-config --includes`
#include <Python.h>
int main()
{
Py_Initialize(); // removes error if put after Py_SetPath
printf("setting path\n"); // prints
Py_SetPath(L"/usr/lib/python3.6"); // Error in `./a.out': free(): invalid size: 0x00007fd5a8365030 ***
printf("success\n"); // doesn't print
return 0;
}
Setting the path works fine, unless I also try to get the path prior to doing so. If I get the path at all, even just to print without modifying the returned value or anything, I get a "double free or corruption" error.
Very confused. Am I doing something wrong or is this a bug? Anyone know a workaround if so?
Edit: Also errors after calling Py_Initialize();. Updated code. Now errors even if I don't call Py_GetPath() first.
From alk it seems related to this bug: https://bugs.python.org/issue31532
Here is the workaround I am using. Since you can't call Py_GetPath() before Py_Initialize(), and also seemingly you can't call Py_SetPath() after Py_Initialize(), you can add to or get the path like this after calling Py_Initialize():
#include <Python.h>
int main()
{
Py_Initialize();
// get handle to python sys.path object
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
// make a list of paths to add to sys.path
PyObject *newPaths = PyUnicode_Split(PyUnicode_FromWideChar(L"a:b:c", -1), PyUnicode_FromWideChar(L":", 1), -1);
// iterate through list and add all paths
for(int i=0; i<PyList_Size(newPaths); i++) {
PyList_Append(path, PyList_GetItem(newPaths, i));
}
// print out sys.path after appends
PyObject *newlist = PyUnicode_Join(PyUnicode_FromWideChar(L":", -1), path);
printf("newlist = %ls\n", PyUnicode_AsWideCharString(newlist, NULL));
return 0;
}
[the below answer refers to this version of the question.]
From the docs:
void Py_Initialize()
Initialize the Python interpreter. In an application embedding Python, this should be called before using any other Python/C API functions; with the exception of Py_SetProgramName(), Py_SetPythonHome() and Py_SetPath().
But the code you show does call Py_GetPath() before it calls Py_Initialize();, which it per the above paragraph implicitly should not.
I am totally new to boost.python.
I reviewed a lot of recommending of using boost.python to apply with python, however still not easy to understand and find a solution for me.
What I want is to import a function or class that directly from a python "SourceFile"
Example File:
Main.cpp
MyPythonClass.py
Let's says if there is a "Dog" class in "MyPythonClass.py" with "bark()" function, how do I get callback and send argument in cpp?
I have no idea what I should do!
Please help me!
When one needs to call Python from C++, and C++ owns the main function, then one must embed the Python interrupter within the C++ program. The Boost.Python API is not a complete wrapper around the Python/C API, so one may find the need to directly invoke parts of the Python/C API. Nevertheless, Boost.Python's API can make interoperability easier. Consider reading the official Boost.Python embedding tutorial for more information.
Here is a basic skeleton for a C++ program that embeds Python:
int main()
{
// Initialize Python.
Py_Initialize();
namespace python = boost::python;
try
{
... Boost.Python calls ...
}
catch (const python::error_already_set&)
{
PyErr_Print();
return 1;
}
// Do not call Py_Finalize() with Boost.Python.
}
When embedding Python, it may be necessary to augment the module search path via PYTHONPATH so that modules can be imported from custom locations.
// Allow Python to load modules from the current directory.
setenv("PYTHONPATH", ".", 1);
// Initialize Python.
Py_Initialize();
Often times, the Boost.Python API provides a way to write C++ code in a Python-ish manner. The following example demonstrates embedding a Python interpreter in C++, and having C++ import a MyPythonClass Python module from disk, instantiate an instance of MyPythonClass.Dog, and then invoking bark() on the Dog instance:
#include <boost/python.hpp>
#include <cstdlib> // setenv
int main()
{
// Allow Python to load modules from the current directory.
setenv("PYTHONPATH", ".", 1);
// Initialize Python.
Py_Initialize();
namespace python = boost::python;
try
{
// >>> import MyPythonClass
python::object my_python_class_module = python::import("MyPythonClass");
// >>> dog = MyPythonClass.Dog()
python::object dog = my_python_class_module.attr("Dog")();
// >>> dog.bark("woof");
dog.attr("bark")("woof");
}
catch (const python::error_already_set&)
{
PyErr_Print();
return 1;
}
// Do not call Py_Finalize() with Boost.Python.
}
Given a MyPythonClass module that contains:
class Dog():
def bark(self, message):
print "The dog barks: {}".format(message)
The above program outputs:
The dog barks: woof
Boost python is used to call cplusplus functions from a python source. Pretty much like the Perl xs module.
If you have a function say bark() in main.cpp, you can use boost python to convert this main.cpp into a python callable module.
Then from python script(assuming link output file is main.so):
import main
main.bark()
I'm trying to call some function from Kernel32.dll in my Python script running on Linux. As Johannes Weiß pointed How to call Wine dll from python on Linux? I'm loading kernel32.dll.so library via ctypes.cdll.LoadLibrary() and it loads fine. I can see kernel32 loaded and even has GetLastError() function inside. However whenever I'm trying to call the function i'm gettings segfault.
import ctypes
kernel32 = ctypes.cdll.LoadLibrary('/usr/lib/i386-linux-gnu/wine/kernel32.dll.so')
print kernel32
# <CDLL '/usr/lib/i386-linux-gnu/wine/kernel32.dll.so', handle 8843c10 at b7412e8c>
print kernel32.GetLastError
# <_FuncPtr object at 0xb740b094>
gle = kernel32.GetLastError
# OK
gle_result = gle()
# fails with
# Segmentation fault (core dumped)
print gle_result
First I was thinking about calling convention differences but it seems to be okay after all. I'm ending with testing simple function GetLastError function without any params but I'm still getting Segmentation fault anyway.
My testing system is Ubuntu 12.10, Python 2.7.3 and wine-1.4.1 (everything is 32bit)
UPD
I proceed with my testing and find several functions that I can call via ctypes without segfault. For instance I can name Beep() and GetCurrentThread() functions, many other functions still give me segfault. I created a small C application to test kernel32.dll.so library without python but i've got essentially the same results.
int main(int argc, char **argv)
{
void *lib_handle;
#define LOAD_LIBRARY_AS_DATAFILE 0x00000002
long (*GetCurrentThread)(void);
long (*beep)(long,long);
void (*sleep)(long);
long (*LoadLibraryExA)(char*, long, long);
long x;
char *error;
lib_handle = dlopen("/usr/local/lib/wine/kernel32.dll.so", RTLD_LAZY);
if (!lib_handle)
{
fprintf(stderr, "%s\n", dlerror());
exit(1);
}
// All the functions are loaded e.g. sleep != NULL
GetCurrentThread = dlsym(lib_handle, "GetCurrentThread");
beep = dlsym(lib_handle, "Beep");
LoadLibraryExA = dlsym(lib_handle, "LoadLibraryExA");
sleep = dlsym(lib_handle, "Sleep");
if ((error = dlerror()) != NULL)
{
fprintf(stderr, "%s\n", error);
exit(1);
}
// Works
x = (*GetCurrentThread)();
printf("Val x=%d\n",x);
// Works (no beeping, but no segfault too)
(*beep)(500,500);
// Segfault
(*sleep)(5000);
// Segfault
(*LoadLibraryExA)("/home/ubuntu/test.dll",0,LOAD_LIBRARY_AS_DATAFILE);
printf("The End\n");
dlclose(lib_handle);
return 0;
}
I was trying to use different calling conventions for Sleep() function but got no luck with it too. When I comparing function declarations\implementation in Wine sources they are essentially the same
Declarations
HANDLE WINAPI GetCurrentThread(void) // http://source.winehq.org/source/dlls/kernel32/thread.c#L573
BOOL WINAPI Beep( DWORD dwFreq, DWORD dwDur ) // http://source.winehq.org/source/dlls/kernel32/console.c#L354
HMODULE WINAPI DECLSPEC_HOTPATCH LoadLibraryExA(LPCSTR libname, HANDLE hfile, DWORD flags) // http://source.winehq.org/source/dlls/kernel32/module.c#L928
VOID WINAPI DECLSPEC_HOTPATCH Sleep( DWORD timeout ) // http://source.winehq.org/source/dlls/kernel32/sync.c#L95
WINAPI is defined to be __stdcall
However some of them works and some don't. As I can understand this sources are for kernel32.dll file and kernel32.dll.so file is a some kind of proxy that supposed to provide access to kernel32.dll for linux code. Probably I need to find exact sources of kernel32.dll.so file and take a look on declarations.
Is there any tool I can use to take a look inside .so file and find out what functions and what calling conventions are used?
The simplest way to examine a DLL is to use the nm command, i.e.
$ nm kernel32.dll.so | grep GetLastError
7b86aae0 T _GetLastError
As others have pointed out, the default calling convention for Windows C DLLs is stdcall. It has nothing to do with using Python. On the Windows platform, ctypes.windll is available.
However, I am not even sure what you are trying to do is at all possible. Wine is a full-blown Windows emulator and it is safe to guess that at least you would have to start it with wine_init before loading any other functions. The Windows API probably have some state (set when Windows boots).
The easiest way to continue is probably to install a Windows version of Python under Wine and run your script from there.
I am trying to embed python 2.7.3 in C++ and use Numpy library and I obtain runtime error while importing Numpy for the second time. Here is a simple code example (as smallest as possible) :
#include <Python.h>
int main() {
for(int i=0 ; i<2 ; i++) {
Py_Initialize() ;
PyImport_ImportModule("numpy");
Py_Finalize() ;
}
return 0 ;
}
What's wrong with this ?
From the Py_Finalize documentation docs you have:
Some extensions may not work properly if their initialization routine is called more than
once; this can happen if an application calls Py_Initialize() and
Py_Finalize() more than once.
I wouldn't be surprised if Numpy is one of these extensions.
Update: looks like it is, see this question.