This question already has answers here:
Interactive input/output using Python
(4 answers)
Closed 5 years ago.
I have an application(.exe) written in any language eg. C++ and want to run the application from python. I can run that simple application by using below sample Python code by following tutorial here https://docs.python.org/2/library/subprocess.html
from subprocess import Popen, PIPE
process = Popen([r"C:\Users\...\x64\Debug\Project13.exe"],stdout = PIPE,
stderr=PIPE, stdin = PIPE)
stdout = process.communicate(input = b"Bob")[0]
print(stdout)
C++ code:
#include <iostream>
#include <windows.h>
#include <string>
using namespace std;
void foo(string s) {
for (int i = 0; i < 3; ++i)
{
cout << "Welcome " << s << " ";
Sleep(2000);
}
}
int main()
{
string s;
cin>> s;
foo(s);
return 0;
}
This works fine for me. But what if i am reading input in C++ application multiple times as below:
#include <iostream>
#include <windows.h>
#include <string>
using namespace std;
int main()
{
string s;
for (int i = 0; i < 3; ++i)
{
cin >> s;
cout << "Welcome " << s << " ";
Sleep(2000);
}
return 0;
}
Here i am not able to use process.communicate() multiple times since the child has already exited by the time it returns.Basically i want to interact with the programme as a continuous session.
I want suggestion or any other approach to solve this issue in python? Thanks in advance.
Assuming you are working on windows I would suggest having a look at namedpipes, on python end you can use PyWPipe, on c++ end you will have to write your own wrappers to get the message for the process.
Related
I have a C++ code that runs in parallel with OpenMP, performing some long calculations. This part works great.
Now, I'm using Python to make a GUI around this code. So, I'd like to call my C++ code inside my python program. For that, I use Pybind11 (but I guess I could use something else if needed).
The problem is that when called from Python, my C++ code runs in serial with only one thread/CPU.
I tried (in two ways) to understand what is done in the documentation of pybind11 here but it does not seem to work at all.
My binding looks like that :
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "../cpp/include/myHeader.hpp"
namespace py = pybind11;
PYBIND11_MODULE(my_module, m) {
m.def("testFunction", &testFunction, py::call_guard<py::gil_scoped_release>());
m.def("testFunction2", [](inputType input) -> outputType {
/* Release GIL before calling into (potentially long-running) C++ code */
py::gil_scoped_release release;
outputType output = testFunction(input);
py::gil_scoped_acquire acquire;
return output;
});
}
Problem: This still does not work and uses only one thread (I verify that with a print of omp_get_num_threads() in an omp parallel region).
Question: What am I doing wrong? What do I need to do to be able to use parallel C++ code inside Python?
Disclaimer: I must admit I don't really understand the GIL thing, particularly in my case where I do not use Python inside my C++ code, which is really "independent" in theory. I just want to be able to use it in another (Python) code.
Have a great day.
EDIT :
I have solved my problem thanks to the pptaszni's answer. Indeed, the GIL things are not needed at all, I misunderstood the documentation. pptaszni's code worked and in fact it was a problem with my CMake file.
Thank you.
It's not really a good answer (too long for a comment thought), because I did not reproduce your problem, but maybe you can isolate the issue in your code by trying this example that works for me:
C++ code:
#include "OpenMpExample.hpp"
#include <algorithm>
#include <iostream>
#include <random>
#include <vector>
#include <omp.h>
constexpr int DATA_SIZE = 10000000;
std::vector<int> testFunction()
{
int nthreads = 0, tid = 0;
std::vector<std::vector<int> > data;
std::vector<int> results;
std::random_device rnd_device;
std::mt19937 mersenne_engine {rnd_device()};
std::uniform_int_distribution<int> dist {-10, 10};
auto gen = [&dist, &mersenne_engine](){ return dist(mersenne_engine); };
#pragma omp parallel private(tid)
{
tid = omp_get_thread_num();
if (tid == 0)
{
nthreads = omp_get_num_threads();
std::cout << "Num threads: " << nthreads << std::endl;
data.resize(nthreads);
results.resize(nthreads);
}
}
#pragma omp parallel private(tid) shared(data, gen)
{
tid = omp_get_thread_num();
data[tid].resize(DATA_SIZE);
std::generate(data[tid].begin(), data[tid].end(), gen);
}
#pragma omp parallel private(tid) shared(data, results)
{
tid = omp_get_thread_num();
results[tid] = std::accumulate(data[tid].begin(), data[tid].end(), 0);
}
for (auto r : results)
{
std::cout << r << ", ";
}
std::cout << std::endl;
return results;
}
I tried to keep the code short, but force the machine to actually do some computations at the same time. Each thread generates 10^7 random integers and then sums them up. Then the python binding does not even require gil_scoped_release:
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "OpenMpExample.hpp"
namespace py = pybind11;
// both versions work for me
// PYBIND11_MODULE(mylib, m) {
// m.def("testFunction", &testFunction, py::call_guard<py::gil_scoped_release>());
// }
PYBIND11_MODULE(mylib, m) {
m.def("testFunction", &testFunction);
}
Example output from python:
Python 3.6.8 (default, Jun 29 2020, 16:38:14)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mylib
>>> x = mylib.testFunction()
Num threads: 12
-10975, -22101, -11333, -28603, -471, -15505, -18141, 2887, -6813, -5328, -13975, -4321,
My environment: Ubuntu 18.04.3 LTS, gcc 8.4.0, openMP 201511, python 3.6.8;
This question already has answers here:
how can i include python.h in QMake
(1 answer)
Embedding python 3.4 into C++ Qt Application?
(4 answers)
Closed 2 years ago.
When I was trying to embed a Python script into my Qt C++ program, I run into multiple problems when trying to include Python.h.
The following features, I would like to provide:
Include python.h
Execute Python Strings
Execute Python Scripts
Execute Python Scripts with Arguments
It should also work when Python is not installed on the deployed machine
Therefore I searched around the Internet to try to find a solution. And found a lot of Questions and Blogs, but non have them covered all my Problems and it still took me multiple hours and a lot of frustration.
That's why I have to write down a StackOverflow entry with my full solution so it might help and might accelerate all your work :)
(This answer and all its code examples work also in a non-Qt environment. Only 2. and 4. are Qt specific)
Download and install Python https://www.python.org/downloads/release
Alter the .pro file of your project and add the following lines (edit for your correct python path):
INCLUDEPATH = "C:\Users\Public\AppData\Local\Programs\Python\Python39\include"
LIBS += -L"C:\Users\Public\AppData\Local\Programs\Python\Python39\libs" -l"python39"
Example main.cpp code:
#include <QCoreApplication>
#pragma push_macro("slots")
#undef slots
#include <Python.h>
#pragma pop_macro("slots")
/*!
* \brief runPy can execut a Python string
* \param string (Python code)
*/
static void runPy(const char* string){
Py_Initialize();
PyRun_SimpleString(string);
Py_Finalize();
}
/*!
* \brief runPyScript executs a Python script
* \param file (the path of the script)
*/
static void runPyScript(const char* file){
FILE* fp;
Py_Initialize();
fp = _Py_fopen(file, "r");
PyRun_SimpleFile(fp, file);
Py_Finalize();
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
runPy("from time import time,ctime\n"
"print('Today is', ctime(time()))\n");
//uncomment the following line to run a script
//runPyScript("test/decode.py");
return a.exec();
}
Whenever you #include <Python.h> use the following code instead. (The Slots from Python will otherwise conflict with the Qt Slots
#pragma push_macro("slots")
#undef slots
#include <Python.h>
#pragma pop_macro("slots")
After compiling, add the python3.dll, python39.dll, as well as the DLLs and Lib Python folders to your compilation folder. You can find them in the root directory of your Python installation. This will allow you to run the embedded c++ code even when python is not installed.
With these steps, I was able to get python running in Qt with the 64 bit MinGW and MSVC compiler. Only the MSVC in debug mode got still a problem.
FURTHER:
If you want to pass arguments to the python script, you need the following function (It can be easy copy-pasted into your code):
/*!
* \brief runPyScriptArgs executs a Python script and passes arguments
* \param file (the path of the script)
* \param argc amount of arguments
* \param argv array of arguments with size of argc
*/
static void runPyScriptArgs(const char* file, int argc, char *argv[]){
FILE* fp;
wchar_t** wargv = new wchar_t*[argc];
for(int i = 0; i < argc; i++)
{
wargv[i] = Py_DecodeLocale(argv[i], nullptr);
if(wargv[i] == nullptr)
{
return;
}
}
Py_SetProgramName(wargv[0]);
Py_Initialize();
PySys_SetArgv(argc, wargv);
fp = _Py_fopen(file, "r");
PyRun_SimpleFile(fp, file);
Py_Finalize();
for(int i = 0; i < argc; i++)
{
PyMem_RawFree(wargv[i]);
wargv[i] = nullptr;
}
delete[] wargv;
wargv = nullptr;
}
To use this function, call it like this (For example in your main):
int py_argc = 2;
char* py_argv[py_argc];
py_argv[0] = "Progamm";
py_argv[1] = "Hello";
runPyScriptArgs("test/test.py", py_argc, py_argv);
Together with the test.py script in the test folder:
import sys
if len(sys.argv) != 2:
sys.exit("Not enough args")
ca_one = str(sys.argv[0])
ca_two = str(sys.argv[1])
print ("My command line args are " + ca_one + " and " + ca_two)
you get the following output:
My command line args are Progamm and Hello
li = list(map(int,input().split()))
I am pretty new to c++ . I essentialy want the easiest verison of this code which takes in the input I pass in via terminal and push back the output to a vector.
I've tried:
#include <iostream>
#include<vector>
#include<string>
using namespace std;
int main()
{
string input;
vector<int> numbers;
while(getline(cin,input,' ')){
numbers.push_back(stoi(input));
}
for(int i : numbers){
cout << i << endl;
}
return 0;
}
I am using g++ 9.2.0.
The same code works fine on an online ide. I am not sure if it's an issue with the g++ compiler or not.
Weird Stuff!
Your example runs fine for me: https://ideone.com/bFLjB1
You could clean this up a bit by using the default type deduction and whitespace-splitting behavior of cin's operator>>:
std::vector<int> numbers;
int temp = 0;
while (std::cin >> temp) {
numbers.push_back(temp);
}
You can construct a container directly from whitespace delimited input.
#include <iostream>
#include <vector>
#include <iterator>
int main() {
std::vector<int> numbers(std::istream_iterator<int>(std::cin), {});
for(int i : numbers){
std::cout << i << std::endl;
}
return 0;
}
I checked already a lot of posts and the subprocess documentation but non of them provided a solution to my problem. At least, i can't find one.
Anyway, here is my problem description:
I would like to call a .exe from a .py file. The .exe needs a integer input argument and returns also an integer value, which i would like to use for further calculations in python.
In order to keep things simple, i would like to use a minimun working example of my "problem"-code (see below). If i run this code, then .exe crashes and i don't know why. Maybe i just missed something but i don't know what!? So here is what i did:
c++ code which i use to generate: MyExe.exe
#include <iostream>
using namespace std;
#include <stdlib.h>
#include <string>
int main(int argc, char* argv[])
{
int x = atoi(argv[1]);
return x;
}
My python code:
from subprocess import Popen, PIPE
path = 'Path to my MyExe.exe'
def callmyexe(value):
p = Popen([path], stdout=PIPE, stdin=PIPE)
p.stdin.write(bytes(value))
return p.stdout.read
a = callmyexe(5)
b = a + 1
print(b)
I use MSVC 2015 and Python 3.6.
You have to use cout for output:
#include <iostream>
using namespace std;
#include <stdlib.h>
#include <string>
int main(int argc, char* argv[])
{
int x = atoi(argv[1]);
cout << x;
}
And command line parameters for the input:
from subprocess import check_output
path = 'Path to my MyExe.exe'
def callmyexe(value):
return int(check_output([path, str(value)]))
a = callmyexe(5)
b = a + 1
print(b)
I have a simple python script that uses a c/c++ library with ctypes. My c++ library also contains a main method, so I can compile it without the -shared flag and it can be excecuted and it runs without issues.
However, when I run the same code from a python script using ctypes, a part of the c++ program is excecuted (I can tell that from the cout calls). Then the entire application, including the python script, termiantes (I can tell that from the missing cout and print calls). There is no error message, no segfault, no python stacktrace.
My question is: How can I debug this? What are possible reasons for this to happen?
Here is part of the code, however, since there is no error message, I don't know which code is relevant.
import ctypes
interface = ctypes.CDLL("apprunner.so")
interface.start()
print "complete"
.
#include "../../app/ShaderApp.cpp"
#include <iostream>
#include "TestApp.cpp"
TestApp* app = 0;
extern "C" void start() {
app = new TestApp();
cout << "Running from library" << endl;
app->run();
}
int main( int argc, const char* argv[]) {
cout << "Running from excecutable" << endl;
start();
}
Typically you begin from a small mock-up library that just lets you test the function calls from python. When this is ready (all the debug prints are ok) you proceed further. In your example, comment out #include "testapp.cpp" and get the prints to cout working.