In 2014 I have written a PyQt4 app that used a non PyQt module within and it worked as it should for nearly two years. Now, the non PyQt module had stopped working as expected within PyQt application (I have qt4 and qt5 ports, both share the problem). Below is a sample code to illustrate what is going on:
#! /usr/bin/env python3
import sys, getopt
from PyQt5.QtWidgets import QApplication # or PyQt4
# non PyQt module:
import mymodule
# demo of the mymodule use. It returns nested list
# with info about objects saved in fname
fname = "/my/file/name"
res0 = mymodule.getnames(fname)
# res0 = [[...],...,[...]] the way it should
#
app = QApplication(sys.argv)
...
# with or without actual run of the application via
# app.exec_()
...
res2 = mymodule.getnames(fname)
# this time the result is:
# res2 = None
Originally, I called the method of mymodule from within PyQt app and it returned the nested list, now it returns None. I have tried to find any explanation or clarification, why the observed behaviour has changed but failed to do so.
I have isolated the code responsible for rendering the mymodule behaviour to the call QApplication([sys.argv]) and wondered if anyone has an idea what is going on. What bugs me the most is, that the module is functional before the application start, but after that not any more.
Some further details about mymodule:
it is a wrapper of a C library written in mymodule.c and installed via the python setup.py install call.
Thank you for any comments in advance!
[Edit] Further informations about mymodule:
mymodule.c code snippet
#include <Python.h>
#include <extlib.h>
#include <math.h>
...
static PyObject * mymodule_getnames(PyObject *self, PyObject *args)
{
char *filename;
if (!PyArg_ParseTuple(args, "s", &filename))
return NULL;
FILE *fp = extlib_func(filename);
...
PyObject *names = PyList_New(0);
...
PyList_Append(names, Py_BuildValue("s", title));
...
return names;
}
...
int main(int argc, char *argv[])
{
Py_SetProgramName(
#if PY_MAJOR_VERSION >= 3
(wchar_t *)
#endif
argv[0]);
Py_Initialize();
return 0;
}
I have resolved the problem by not using anaconda as my python package manager. By installing everything in the system it worked as it should again. But the question still remains since the reason why is it not working within anaconda (but did for such a long time) is still beyond me.
I have posted this answer but will not accept it, if someone comes with an explanation of the phenomena. If after some time there will be no other reply I will mark it accepted to unclutter the questions pool.
Thank you all for your time and suggestions.
Related
I'm very new to the whole CMake. Following this and this posts, now I want to call a MAXON function inside Python, using pybind11. What I have done so far:
The library can be downloaded from this page (direct download link).
wget https://www.maxongroup.com/medias/sys_master/root/8837358518302/EPOS-Linux-Library-En.zip
unzip:
unzip EPOS-Linux-Library-En.zip
make the install shell script executable and run it:
chmod +x ./install.sh
sudo ./install.sh
Then going to the example folder:
cd /opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd/
Now combining the CMakeLists.txt files from here:
# CMakeLists.txt
cmake_minimum_required(VERSION 2.8.12)
project (HelloEposCmd)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -Wall")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -Wall")
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
find_package(pybind11 REQUIRED)
pybind11_add_module(${PROJECT_NAME} HelloEposCmd.cpp)
add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} -lEposCmd)
and the HelloEposCmd.cpp this line is added right after other header files:
#include <pybind11/pybind11.h>
the main function is renamed to:
int run(int argc, char** argv)
and the pybind11 syntax to add the module is written at the end:
PYBIND11_MODULE(HelloEposCmd, m) {
m.def("run", &run, "runs the HelloEposCmd");
}
However, When I run the cmake . I get the error:
CMake Error at CMakeLists.txt:13 (add_executable):
add_executable can not create target "HelloEposCmd" because another target with the same name already exists. The existing target is a module library created in source directory "/opt/EposCmdLib_6.6.1.0/examples/HelloEposCmd" See documentation for policy CMP0002 for more details.
...
I was wondering if you could be kind to help me get the right CMakeList.txt file. Ideally, I should be able to call the compiled module in python:
# HelloEposCmd.py
import HelloEposCmd
HelloEposCmd.run()
Thanks for your support in advance.
pybind11_add_module already creates a target for you. So you don't need add_executable anymore. Just remove that line and when you will build you will get a library with the name HelloEposCmd
add_executable is needed if you are building an executable (.exe), which I believe is not what you want.
Documenation of pybind11 says.
This function behaves very much like CMake’s builtin add_library (in fact, it’s a wrapper function around that command).
Thanks to abhilb post and his kind followup in the comments I was able to figure the problem out. well, at least find a temporary workaround:
According to this post, the last two lines of the CMakeLists.txt file should change to
# this line can be removed
# add_executable(${PROJECT_NAME} HelloEposCmd.cpp)
target_link_libraries(${PROJECT_NAME} PRIVATE -lEposCmd)
and then because according to this post pybind11 doesn't support double pointers we change the run function to:
int run() {
int argc = 1;
char* argv[] = {"./HelloEposCmd"};
...
}
which I suppose to be a horrible workaround (inspired by information from this page). Now running cmake ., make and python3 HelloEposCmd.py should work properly (except a small c++ warning!).
P.S.1. Maybe someone could use std::vector<std::string> as suggested here. This idea was proposed here and there are already some answers worth investigating.
P.S.2. Following this discussion, another workaround could be something like:
#include <stdio.h>
#include <stdlib.h>
void myFunc(int argc, char* argv[]) {
for (int i = 0; i < argc; ++i) {
printf("%s\n", argv[i]);
}
}
int run(int argc, long* argv_) {
char** argv = (char**)malloc(argc * sizeof(char*));
for (int i = 0; i < argc; ++i) {
argv[i] = (char*)(argv_[i]);
}
myFunc(argc, argv);
free(argv);
return 0;
}
I have a C++ project that I would like to add an embedded python interpreter to. I've actually succeeded in this with simple scripts, but when I try doing something with Tkinter, it opens a blank window and never draws any frame or contents. I'm pretty sure it has to do with the GIL, but have been unsuccessful in finding some combination of calls that work. I've made a simple example that illustrates this with a C++ file and a python script that the compiled program runs. If you uncomment the MyPythonThread line and comment out the two that run it in a thread, then it works as expected.
It seems python knows whether it is in the "main" thread or not, regardless of which thread I Py_Initialize from.
Other info.: I'm testing this on Mac OS X 10.13.6 with homebrew installed python 2.7.15.
//
// Compile with: g++ `python-config --cflags --libs` --std=c++11 test.cc
//
#include <cstdio>
#include <Python.h>
#include <thread>
void MyPythonThread(void)
{
PyEval_InitThreads();
Py_Initialize();
const char *fname = "test.py";
PySys_SetArgv( 1, (char**)&fname );
auto fil = std::fopen(fname, "r");
PyRun_AnyFileEx( fil, NULL, 1 );
}
int main(int narg, char * argv[])
{
// This works
// MyPythonThread();
// This does not
std::thread thr(MyPythonThread);
thr.join();
return 0;
}
Here is the python script that it runs:
#!/usr/bin/env python
import Tkinter as tk
window = tk.Tk()
top_frame = tk.Frame(window).pack()
b1 = tk.Button(top_frame, text = "Button").pack()
window.mainloop()
I'm trying to embed Python in my C++ application, somewhat akin to the method found here in section 1.4:
https://docs.python.org/3.5/extending/embedding.html
The synopsis of the problem I'm having is that I can't get the C++ application to work with .py files that import the 'emb' module, ie, the Python extension module that's written into the C++ code.
I have a Python file, testmod.py:
import emb
# define some functions
def printhello(input):
emb.numargs()
return 2
def timesfour(input):
print(input * 4)
In my C++ application, I have this code which works:
PyImport_AppendInittab("emb", &(mynamespace::PyInit_emb) );
Py_Initialize();
PyObject *globals = PyModule_GetDict(PyImport_AddModule("__main__"));
PyObject *testModule = PyImport_ImportModule("emb");
PyObject* pFunc = PyObject_GetAttrString(testModule, "numargs");
After this, pFunc is non-NULL; things look good. So I think the 'embedded module' is fine.
If I change the last two lines from above to:
PyObject* testModule = PyImport_ImportModule("testmod");
PyObject* pFunc = PyObject_GetAttrString(testModule, "printhello");
This also works fine, provided the line emb.numargs() is removed from testmod.py Once I add that line, and re-run the C++ application, testModule becomes NULL, which means something has gone wrong.
Any ideas?
Is this the way this capability is supposed to be used?
I would like to embed Python interpreter 3.4 into a Qt 5.2.1 application (64-bit).
However I'm having build issues, I mean when I include Python header in the main.cpp it compiles fine.
#include <python.h>
#include "mainwindow.h"
#include <QApplication>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
return a.exec();
}
but when I put it anywhere else (after Qt headers)
//
// embedpytest.cpp
//
#include <QLibrary>
#include <python.h>
EmbedPyTest::EmbedPyTest()
{
}
I get compile errors:
C:\Python34\include\object.h:435: error: C2059: syntax error : ';'
C:\Python34\include\object.h:435: error: C2238: unexpected token(s) preceding ';'
It's very similar problem to this one, but the solution is not working
Embedding Python in Qt 5
Anyone knows how to solve this issue ? or suggest some clean workaround so that python.h and Qt5
can live together happily ever after ?
Another way to avoid the conflict regarding 'slots', without the need for deactivating the keywords signals/slots/emit (which may be undesirable for large Qt projects), is to locally "park" the offending keyword while Python.h is included, and then reassign it. To achieve this, replace every occurrence of #include "Python.h" by the following block:
#pragma push_macro("slots")
#undef slots
#include "Python.h"
#pragma pop_macro("slots")
Or, more conveniently, put the above code in its own header, e.g. Python_wrapper.h, and replace all occurrences of #include "Python.h" by #include "Python_wrapper.h".
The offending line is this:
PyType_Slot *slots; /* terminated by slot==0. */
The problem is that with this line, "slots" is a keyword by default in Qt. In order to use that variable name in other projects, you will need to use this in your project file:
CONFIG += no_keywords
For details, see the documentation:
Using Qt with 3rd Party Signals and Slots
It is possible to use Qt with a 3rd party signal/slot mechanism. You can even use both mechanisms in the same project. Just add the following line to your qmake project (.pro) file.
CONFIG += no_keywords
It tells Qt not to define the moc keywords signals, slots, and emit, because these names will be used by a 3rd party library, e.g. Boost. Then to continue using Qt signals and slots with the no_keywords flag, simply replace all uses of the Qt moc keywords in your sources with the corresponding Qt macros Q_SIGNALS (or Q_SIGNAL), Q_SLOTS (or Q_SLOT), and Q_EMIT.
I'm trying to call some function from Kernel32.dll in my Python script running on Linux. As Johannes Weiß pointed How to call Wine dll from python on Linux? I'm loading kernel32.dll.so library via ctypes.cdll.LoadLibrary() and it loads fine. I can see kernel32 loaded and even has GetLastError() function inside. However whenever I'm trying to call the function i'm gettings segfault.
import ctypes
kernel32 = ctypes.cdll.LoadLibrary('/usr/lib/i386-linux-gnu/wine/kernel32.dll.so')
print kernel32
# <CDLL '/usr/lib/i386-linux-gnu/wine/kernel32.dll.so', handle 8843c10 at b7412e8c>
print kernel32.GetLastError
# <_FuncPtr object at 0xb740b094>
gle = kernel32.GetLastError
# OK
gle_result = gle()
# fails with
# Segmentation fault (core dumped)
print gle_result
First I was thinking about calling convention differences but it seems to be okay after all. I'm ending with testing simple function GetLastError function without any params but I'm still getting Segmentation fault anyway.
My testing system is Ubuntu 12.10, Python 2.7.3 and wine-1.4.1 (everything is 32bit)
UPD
I proceed with my testing and find several functions that I can call via ctypes without segfault. For instance I can name Beep() and GetCurrentThread() functions, many other functions still give me segfault. I created a small C application to test kernel32.dll.so library without python but i've got essentially the same results.
int main(int argc, char **argv)
{
void *lib_handle;
#define LOAD_LIBRARY_AS_DATAFILE 0x00000002
long (*GetCurrentThread)(void);
long (*beep)(long,long);
void (*sleep)(long);
long (*LoadLibraryExA)(char*, long, long);
long x;
char *error;
lib_handle = dlopen("/usr/local/lib/wine/kernel32.dll.so", RTLD_LAZY);
if (!lib_handle)
{
fprintf(stderr, "%s\n", dlerror());
exit(1);
}
// All the functions are loaded e.g. sleep != NULL
GetCurrentThread = dlsym(lib_handle, "GetCurrentThread");
beep = dlsym(lib_handle, "Beep");
LoadLibraryExA = dlsym(lib_handle, "LoadLibraryExA");
sleep = dlsym(lib_handle, "Sleep");
if ((error = dlerror()) != NULL)
{
fprintf(stderr, "%s\n", error);
exit(1);
}
// Works
x = (*GetCurrentThread)();
printf("Val x=%d\n",x);
// Works (no beeping, but no segfault too)
(*beep)(500,500);
// Segfault
(*sleep)(5000);
// Segfault
(*LoadLibraryExA)("/home/ubuntu/test.dll",0,LOAD_LIBRARY_AS_DATAFILE);
printf("The End\n");
dlclose(lib_handle);
return 0;
}
I was trying to use different calling conventions for Sleep() function but got no luck with it too. When I comparing function declarations\implementation in Wine sources they are essentially the same
Declarations
HANDLE WINAPI GetCurrentThread(void) // http://source.winehq.org/source/dlls/kernel32/thread.c#L573
BOOL WINAPI Beep( DWORD dwFreq, DWORD dwDur ) // http://source.winehq.org/source/dlls/kernel32/console.c#L354
HMODULE WINAPI DECLSPEC_HOTPATCH LoadLibraryExA(LPCSTR libname, HANDLE hfile, DWORD flags) // http://source.winehq.org/source/dlls/kernel32/module.c#L928
VOID WINAPI DECLSPEC_HOTPATCH Sleep( DWORD timeout ) // http://source.winehq.org/source/dlls/kernel32/sync.c#L95
WINAPI is defined to be __stdcall
However some of them works and some don't. As I can understand this sources are for kernel32.dll file and kernel32.dll.so file is a some kind of proxy that supposed to provide access to kernel32.dll for linux code. Probably I need to find exact sources of kernel32.dll.so file and take a look on declarations.
Is there any tool I can use to take a look inside .so file and find out what functions and what calling conventions are used?
The simplest way to examine a DLL is to use the nm command, i.e.
$ nm kernel32.dll.so | grep GetLastError
7b86aae0 T _GetLastError
As others have pointed out, the default calling convention for Windows C DLLs is stdcall. It has nothing to do with using Python. On the Windows platform, ctypes.windll is available.
However, I am not even sure what you are trying to do is at all possible. Wine is a full-blown Windows emulator and it is safe to guess that at least you would have to start it with wine_init before loading any other functions. The Windows API probably have some state (set when Windows boots).
The easiest way to continue is probably to install a Windows version of Python under Wine and run your script from there.