I have a C++ project that I would like to add an embedded python interpreter to. I've actually succeeded in this with simple scripts, but when I try doing something with Tkinter, it opens a blank window and never draws any frame or contents. I'm pretty sure it has to do with the GIL, but have been unsuccessful in finding some combination of calls that work. I've made a simple example that illustrates this with a C++ file and a python script that the compiled program runs. If you uncomment the MyPythonThread line and comment out the two that run it in a thread, then it works as expected.
It seems python knows whether it is in the "main" thread or not, regardless of which thread I Py_Initialize from.
Other info.: I'm testing this on Mac OS X 10.13.6 with homebrew installed python 2.7.15.
//
// Compile with: g++ `python-config --cflags --libs` --std=c++11 test.cc
//
#include <cstdio>
#include <Python.h>
#include <thread>
void MyPythonThread(void)
{
PyEval_InitThreads();
Py_Initialize();
const char *fname = "test.py";
PySys_SetArgv( 1, (char**)&fname );
auto fil = std::fopen(fname, "r");
PyRun_AnyFileEx( fil, NULL, 1 );
}
int main(int narg, char * argv[])
{
// This works
// MyPythonThread();
// This does not
std::thread thr(MyPythonThread);
thr.join();
return 0;
}
Here is the python script that it runs:
#!/usr/bin/env python
import Tkinter as tk
window = tk.Tk()
top_frame = tk.Frame(window).pack()
b1 = tk.Button(top_frame, text = "Button").pack()
window.mainloop()
Related
I have a c++ (VS2019) app
#include <stdio.h>
#include <Python.h>
void main(int argc, char *argv[])
{
printf( "- before\n" );
FILE* file;
Py_SetPath( L"c:\\Python37\\Lib" );
Py_Initialize();
file = fopen( "abc.py", "r" );
PyRun_SimpleFile( file, "abc.py" );
Py_Finalize();
printf( "- after\n" );
return;
}
that calls a python (3.10) script abc.py
print( "hi" );
When run (on Win11) from cmd (main.exe), the output is:
- before
hi
- after
However, when diverted into a file (main.exe > out.txt, or used in a pipe to tee), I get:
hi
- before
- after
In my real app, where I interleave python with c++, the python output is printed after the script terminates (again, only when diverting the app's output).
Adding
setbuf( stdout, NULL );
, like #retired-ninja suggested, resolves the simple example above. However, in my more complicated app, it doesn't. I need to find a new simple example.
I only needed to add auto-flush to my python script to resolve the issue:
sys.stdout.reconfigure(line_buffering=True)
This question already has answers here:
how can i include python.h in QMake
(1 answer)
Embedding python 3.4 into C++ Qt Application?
(4 answers)
Closed 2 years ago.
When I was trying to embed a Python script into my Qt C++ program, I run into multiple problems when trying to include Python.h.
The following features, I would like to provide:
Include python.h
Execute Python Strings
Execute Python Scripts
Execute Python Scripts with Arguments
It should also work when Python is not installed on the deployed machine
Therefore I searched around the Internet to try to find a solution. And found a lot of Questions and Blogs, but non have them covered all my Problems and it still took me multiple hours and a lot of frustration.
That's why I have to write down a StackOverflow entry with my full solution so it might help and might accelerate all your work :)
(This answer and all its code examples work also in a non-Qt environment. Only 2. and 4. are Qt specific)
Download and install Python https://www.python.org/downloads/release
Alter the .pro file of your project and add the following lines (edit for your correct python path):
INCLUDEPATH = "C:\Users\Public\AppData\Local\Programs\Python\Python39\include"
LIBS += -L"C:\Users\Public\AppData\Local\Programs\Python\Python39\libs" -l"python39"
Example main.cpp code:
#include <QCoreApplication>
#pragma push_macro("slots")
#undef slots
#include <Python.h>
#pragma pop_macro("slots")
/*!
* \brief runPy can execut a Python string
* \param string (Python code)
*/
static void runPy(const char* string){
Py_Initialize();
PyRun_SimpleString(string);
Py_Finalize();
}
/*!
* \brief runPyScript executs a Python script
* \param file (the path of the script)
*/
static void runPyScript(const char* file){
FILE* fp;
Py_Initialize();
fp = _Py_fopen(file, "r");
PyRun_SimpleFile(fp, file);
Py_Finalize();
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
runPy("from time import time,ctime\n"
"print('Today is', ctime(time()))\n");
//uncomment the following line to run a script
//runPyScript("test/decode.py");
return a.exec();
}
Whenever you #include <Python.h> use the following code instead. (The Slots from Python will otherwise conflict with the Qt Slots
#pragma push_macro("slots")
#undef slots
#include <Python.h>
#pragma pop_macro("slots")
After compiling, add the python3.dll, python39.dll, as well as the DLLs and Lib Python folders to your compilation folder. You can find them in the root directory of your Python installation. This will allow you to run the embedded c++ code even when python is not installed.
With these steps, I was able to get python running in Qt with the 64 bit MinGW and MSVC compiler. Only the MSVC in debug mode got still a problem.
FURTHER:
If you want to pass arguments to the python script, you need the following function (It can be easy copy-pasted into your code):
/*!
* \brief runPyScriptArgs executs a Python script and passes arguments
* \param file (the path of the script)
* \param argc amount of arguments
* \param argv array of arguments with size of argc
*/
static void runPyScriptArgs(const char* file, int argc, char *argv[]){
FILE* fp;
wchar_t** wargv = new wchar_t*[argc];
for(int i = 0; i < argc; i++)
{
wargv[i] = Py_DecodeLocale(argv[i], nullptr);
if(wargv[i] == nullptr)
{
return;
}
}
Py_SetProgramName(wargv[0]);
Py_Initialize();
PySys_SetArgv(argc, wargv);
fp = _Py_fopen(file, "r");
PyRun_SimpleFile(fp, file);
Py_Finalize();
for(int i = 0; i < argc; i++)
{
PyMem_RawFree(wargv[i]);
wargv[i] = nullptr;
}
delete[] wargv;
wargv = nullptr;
}
To use this function, call it like this (For example in your main):
int py_argc = 2;
char* py_argv[py_argc];
py_argv[0] = "Progamm";
py_argv[1] = "Hello";
runPyScriptArgs("test/test.py", py_argc, py_argv);
Together with the test.py script in the test folder:
import sys
if len(sys.argv) != 2:
sys.exit("Not enough args")
ca_one = str(sys.argv[0])
ca_two = str(sys.argv[1])
print ("My command line args are " + ca_one + " and " + ca_two)
you get the following output:
My command line args are Progamm and Hello
I have the following code in the Qt Quick Application together with Boost.
In this Cpp there is a personal module created using BOOST_PYTHON_MODULE(hello). The main goal is to be able to import hello in Python and call the methods of hello struct. My Python script only contains very simple structure as i just want to see no errors when importing hello.
import hello
print("Import was successful!")
Most of the codes below are copied from a different question in stackoverflow but not entirely so i had to repost the parts.
Main.cpp
#include <cstdlib> // setenv, atoi
#include <iostream> // cerr, cout, endl
#include <boost/python.hpp>
#include <QGuiApplication>
#include <QQmlApplicationEngine>
struct World
{
void set(std::string msg) { this->msg = msg; }
std::string greet() { return msg; }
std::string msg;
};
//---------------------------------------------------------------------------------------------------------------------
/// Staticly linking a Python extension for embedded Python.
BOOST_PYTHON_MODULE(hello)
{
namespace python = boost::python;
python::class_<World>("World")
.def("greet", &World::greet)
.def("set", &World::set)
;
}
//---------------------------------------------------------------------------------------------------------------------
int main(int argc, char *argv[])
{
QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
QGuiApplication app(argc, argv);
namespace python = boost::python;
try
{
int uploaded = PyImport_AppendInittab("hello", &PyInit_hello);
//This executes else part
if(uploaded == -1)
std::cout<< "Module table was not extended: " << uploaded << std::endl;
else
std::cout<< "Module Table was extended" << std::endl;
Py_Initialize();
} catch (...)
{
PyErr_Print();
return 1;
}
return app.exec();
}
Finally, I run my QT application and the return app.exec(); keeps it running while i try and run my python script as mentioned above from the terminal. The python script is in the same directory as the currently running application, not sure if that makes any difference.
Then the error i get is:
Traceback (most recent call last):
File "test_hilton.py", line 1, in <module>
import hello
ModuleNotFoundError: No module named 'hello'
Not sure what i am missing here. According to the Python API:
PyImport_AppendInittab - Add a single module to the existing table of
built-in modules. This is a convenience wrapper around
PyImport_ExtendInittab(), returning -1 if the table could not be
extended. The new module can be imported by the name name, and uses
the function initfunc as the initialization function called on the
first attempted import.
And the If-else part inside the try-catch block inside the main proves that the hello module is being added to the table. Out of ideas on what to do, looked in different places. But still stuck with this part of the problem.
Since the hello module is defined in that Qt program it is available only in that program. Executing the program doesn't make it available to python interpreter that expects to find hello.py or hello.so (the file extension may vary depending on the operating system) when importing hello by import hello.
You need to build a python module, answer might help.
How to compile python 3.4.3 script with the module tkinter and ttk to an self-executable exe (standalone)? (py2exe, pyinstaller, freeze doesn't work.) any suggestions? Thank You
What I do is
download Portable Python
create an file in an other language that can be compiled to exe
make that executable call portable Python with my Python file.
Structure:
application_folder # the folder where everything is in
+--my_python_folder # the folder where your python files are in
| +--my_program.py # the python file that you want to start
+--Portable Python 3 # the Python version that you use
+--program.exe # the compiled program
The C++ source code:
// based on https://msdn.microsoft.com/en-us/library/ms682425%28VS.85%29.aspx
#include <windows.h>
#include <stdio.h>
#include <tchar.h>
int _tmain( int argc, TCHAR *argv[] )
{
STARTUPINFO si;
PROCESS_INFORMATION pi;
// choose between pythonw.exe and python.exe
TCHAR command[] = "\"Portable Python 3\\App\\pythonw.exe\" \"my_program.py\"";
// the directory where you start the Python program in
TCHAR directory[] = "my_python_folder";
ZeroMemory( &si, sizeof(si) );
si.cb = sizeof(si);
ZeroMemory( &pi, sizeof(pi) );
// Start the child process.
if( !CreateProcess( NULL, // No module name (use command line)
command, // Command line
NULL, // Process handle not inheritable
NULL, // Thread handle not inheritable
FALSE, // Set handle inheritance to FALSE
0, // No creation flags
NULL, // Use parent's environment block
directory, // Use parent's starting directory
&si, // Pointer to STARTUPINFO structure
&pi ) // Pointer to PROCESS_INFORMATION structure
)
{
printf( "CreateProcess failed (%d).\n", GetLastError() );
return 1;
}
/*
// Wait until child process exits.
WaitForSingleObject( pi.hProcess, INFINITE );
// Close process and thread handles.
CloseHandle( pi.hProcess );
CloseHandle( pi.hThread );
*/
return 0;
}
You can compile the file with devc++.
Evaluation
Pros:
An other way to do it.
Cons:
Needs whole Portable Python.
No command line arguments are passed.
You can do it with a .bat file and it works, too.
The current working directory is different than the caller's one.
I have created a c program which requires an input (through scanf). Then I created the .so file and called that in a python script, so that when I run the script, input will be asked in the terminal. But when I run the python program, the terminal hangs.
Please note:
1. My c code
#include <stdio.h>
#include <string.h>
int open(void)
{
char input[20];
scanf("input = %s\n",&input);
printf("\n%s\n","input");
}
2. Command I used for compiling the code
gcc -c usb_comm.c
3.Creating .so file
gcc -shared -o libhello.so usb_comm.o
4.Relevant section of python program
Loading the .so file
from ctypes import cdll
mydll = cdll.LoadLibrary('/home/vineeshvs/Dropbox/wel/workspace/Atmel/libhello.so')
Calling the function defined in the c program
mydll.scanf("%c",mydll.open())
Thanks for listening :)
mydll.open() call scanf. Why do you call scanf in Python?
Just call mydll.open() only:
mydll.open()
UPDATE
#include <stdio.h>
void open(void)
{
char input[20];
printf("input = "); // If you want prompt
scanf("%s", input);
printf("\n%s\n", input);
}