Qt - How to write "more" data to QProcess? - python

I am facing an issue I faced a few times so far.
The issue in question just gets solved by itself every time, without me understanding what is causing it
So what happens is that I start a python virtual environment from my c++ code. That works, afterwards by using the write function I am able to write stuff in that environment. This also works perfectly fine so far. However I am unable to write my last command to the process.
I though about maybe some buffer being full but I didn't really find anything about a buffer in the Qt docs
This is the relevant piece of code:
static QStringList params;
QProcess *p = new QProcess();
params<<"-f"<<"-c"<<"python2"<< "/home/John/Desktop/python.log";
qDebug()<<"parameters: "<<params;
qDebug()<<"going to write";
p->start("script", params);
qDebug()<<"Turning on new user process...";
while(!p->waitForStarted())
{qDebug()<<"waiting for virtualenv to be ready";}
successFailWrite = p->write("import imp;\n");
while(!p->waitForBytesWritten());
successFailWrite = p->write("foo = imp.load_source('myTest', '/home/John/recognitionClass.py');\n");
while(!p->waitForBytesWritten());
successFailWrite = p->write("from myTest import recognitionClass;\n");
while(!p->waitForBytesWritten());
successFailWrite = p->write("myClassObj = recognitionClass();\n");
if(successFailWrite !=-1)
{qDebug()<<"OK written";}
while(!p->waitForBytesWritten());
successFailWrite = p->write("habelahabela\n");
if(successFailWrite !=-1)
{qDebug()<<"OK written";}
QString name = "John";
QString processNewUserParameter= "print myClassObj.addNewUser("+ name +");\n";
QByteArray processNewUserParameterByteArr= processNewUserParameter.toUtf8();
p->write(processNewUserParameterByteArr);
I keep a log file which contains what is being written to the python virtualenv and what is being printed
Script started on Son 27 Aug 2017 20:09:52 CEST
import imp;
foo = imp.load_source('myTest', '/home/John/recognitionClass.py');
from myTest import recognitionClass;
myClassObj = recognitionClass();
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp;
>>> foo = imp.load_source('myTest', '/home/John/recognit
<myTest', '/home/John/recogniti onClass.py');
/usr/local/lib/python2.7/dist-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.
DeprecationWarning)
>>> from myTest import recognitionClass;
>>> myClassObj = recognitionClass();
>>>
It does pront "OK written" twice, which on one side proves that I successfully wrote my commands to the process, yet I can't see anything.
As you can see the test sentence "habelahabela" doesn't get written neither.
Does anybody have an idea about what I may be doing wrong?
I know that I am writing my commands to quickly to the environment. Because as you can see I start by writing "import imp", it then gets buffered and a little later the buffer gets flushed and the virtualenv executes the command (this is why you see it twice).
Does anybody see why I can't see the test-sentence and -more importantly- my actual command "print myClassObj.addNewUser("+ name +");\n" being printed to the virtual environment?
Thanks

First of all, there is no sense in writing while(!p->waitForBytesWritten());. waitForBytesWritten already blocks your thread without a while loop and, as the name states, waits until bytes are written. It returns false only if there are either timeout or an error. In the first case you should give it more time to write bytes. In the second case you should fix the error and only then try again.
The same holds for waitForStarted and all other Qt functions starting with "waitFor...".
So the usage looks like:
if(!p->waitForBytesWritten(-1)) // waits forever until bytes ARE written
{
qDebug() << "Error while writing bytes";
}
Regarding the question: I believe the problem (or at least a part of it) is that you write your last 2 messages into p, but you neither wait for bytesWritten() signal, nor use waitForBytesWritten() blocking function. Although, there is probably no error occuring (because p->write(...) does not return -1 at that point), however it does not mean that your message is written yet. In a nutshell, wait for bytesWritten() signal...
QProcess inherits from QIODevice, so I recommend to look its docs and learn about it a bit more.

Related

Just started using Python 3 and no matter what I do I can't seem to get exponentials to work. All I get is a blank line when I save and run

As the title says, I've tried all kinds of things. Essentially I'm trying to write a function that accepts an integer as an argument and returns its square.
This is an example of what I've tried on my own:
number1 = "2"
def square_number(number1):
return("number1" ** 2)
This didn't work but I've tried a copy past deal from the internet to check:
number1 = 5
square1 = pow(number1, 2)
This also didn't work. All I see is this:
======================== RESTART: /Users/NK/Documents/Python training/Learning functions.py ========================
>>>
just a blank line.
You are not instructing python to print anything to the screen.
In the first case, you are simply defining a function. More on this later *
In the copy-pasted code, you are squaring number1, but then do nothing with the result. If you want to show it, you need to instruct it:
print(square1)
*
Back to your trial in your trial, there are a few errors.
First, you should put a number in the variable number1. Currently, because you've put 2 inside quotes "2", your variable is a string, not a number. Instead, say number1 = 2 (no quotes).
Second, in your function, you are using quotes around number1. Then this is also a string, so you're not referencing your variable, but you are squaring the string "number1" . This will not work and will throw an error.
Currently nothing is happening because you have only defined what the function should do. You haven't used the function yet.
Note that the parameter of the function can have any name, so you can define it as:
def square_number(a): return a ** 2
Here you've just define what the function should do when called. It is independedn to anything outside of it, like number1.
Then you can call your function, with a parameter:
sq3 = square_number(3)
sq4 = square_number(4)
sq_n1 = square_number(number1)
and then you can print the results, which here I've put in variables
print(sq3, sq4, sq_n1)
Again, note, no quotes, I'm referring to the variables.
I would suggest you start first by:
Reading about python coding convention, for the readability of your code.
Learn how to run the interpreter in your terminal.
Learn how to execute code in a .py file.
Usually, you will write your code in a .py file. But it is a good practice to always try to run every line of your code first at the terminal to know what it will change.
Terminal example
Part of your script squares a number, so you go to your terminal and experiment with the different ways of how can you do squaring in python.
ziadh#Ziads-MacBook-Air ~ % python3
Python 3.10.4 (v3.10.4:9d38120e33, Mar 23 2022, 17:29:05) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> # first method using pow
>>> num_one: int = 3
>>> pow(num_one, 2)
9
>>> pow(4, 2)
16
>>> # second method using **
>>> 4**2
16
Now you are an expert in squaring numbers in python, you go to your .py file and write the following.
num_one: int = 3
num_one_squared: int = pow(num_one, 2)
print(num_one_squared)
# you can also do
num_two: int = 4
num_two_squared: int = num_two**2
print(num_two_squared)

Issues with accessing PyObjects after writing

I am trying to do some fairly simple list manipulation, using a Jupyter notebook that calls a DLL function. I'd like my Jupyter notebook/Python code to pass in a Python list to a C++ function, which modifies the list, and then I'd like the Python code to be able to access the new list values.
I can actually read (in Jupyter) the items that were not edited by the C++ code, so there must be some issue with how I'm writing, but every example I can find looks just like my code. When I try to access the item in the list that the C++ code writes, my Jupyter kernel dies with no explanation; I've tried to run the same Python code in the terminal, and the terminal session just exits, again with no explanation.
Running on Windows 10, environment with Python 3.9.2. Here's the Python:
import os
import ctypes
import _ctypes
# Import the DLL
mydll = ctypes.cdll.LoadLibrary(*path to DLL*)
# Set up
data_in = [3,6,9]
mydll.testChange.argtypes = [ctypes.py_object]
mydll.testChange.restype = ctypes.c_float
mydll.testChange(data_in)
# Returns 0.08
After running this and closing the DLL, running data_in[1] returns 6, data_in[2] returns 9, and data_in[0] causes my kernel to die.
C code for the DLL:
float testChange(PyObject *data_out) {
Py_SetPythonHome(L"%user directory%\\anaconda3");
Py_Initialize();
PyList_SetItem(data_out, 0, PyLong_FromLong(1L));
return 0.08;
}
I can also insert a number of print statements in this code that show that I can read out all three items in the DLL both before and after the call to PyList_SetItem using calls like PyLong_AsLong(PyList_GetItem(data_out, 1)). It's not clear to me that any reference counts need changing or anything like that, but perhaps I misunderstand the idea. Any ideas you all have would be greatly appreciated.

Boost::Python Not Finding C++ Class in OSX

I'm porting an Application from Linux to OS X and the Boost::Python integration is failing at run time.
I'm exposing my C++ classes like so:
using namespace scarlet;
BOOST_PYTHON_MODULE(libscarlet) {
using namespace boost::python;
class_<VideoEngine, boost::noncopyable>("VideoEngine", no_init)
.def("play", &VideoEngine::play)
.def("pause", &VideoEngine::pause)
.def("isPaused", &VideoEngine::isPaused)
[...]
;
}
I'm importing the library like so:
try {
boost::python::import("libscarlet");
} catch (boost::python::error_already_set & e) {
PyErr_Print();
}
Then I'm inject an instance into the global Python namespace like so:
void VideoEngine::init() {
[...]
try {
auto main_module = boost::python::import("__main__");
auto main_namespace = main_module.attr("__dict__");
main_namespace["engine"] = boost::python::object(boost::python::ptr(this));
} catch (boost::python::error_already_set & e) {
PyErr_Print();
}
[...]
}
It works great in Linux but in OS X an exception is thrown and PyErr_Print() returns TypeError: No Python class registered for C++ class scarlet::VideoEngine.
As far as I can tell the module works without issue when imported via the Python interpreter. It is difficult to test since it designed to be injected as a pre-constructed instance but the class and functions are present as shown below:
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import libscarlet
>>> libscarlet.VideoEngine
<class 'libscarlet.VideoEngine'>
>>> libscarlet.VideoEngine.play
<unbound method VideoEngine.play>
Any ideas as to where the incompatibility lies?
Edit: I'm starting to think it might be related to multithreading since my OS X implementation uses a different threading structure, although all of the calls detailed here happen in the same thread. Could that be the cause of such an issue? Probably not the issue since it doesn't work in MS Windows in single-threaded mode.
I have solved this now.
It was caused entirely by Boost::Python being statically compiled and once I recompiled it as a shared library the problem went away entirely, on all platforms.
The lesson: don't compile boost statically. I'm pretty sure there are warnings against it and they should be heeded.

How do you pass a void * back and forth to a shared library in python using ctypes?

I believe the basic process is:
from ctypes import *
LEAP = ctypes.CDLL('leap.dylib')
leap_controller = LEAP.leap_controller
leap_controller.res_type = c_void_p
leap_controller_dispose = LEAP.leap_controller_dispose
leap_controller_dispose.argtype = [c_void_p]
However, when I invoke this code it mangles the void * pointer. Debugging from the associated C code, I get:
doug:test doug$ python test.py
Controller init!
returning: 0x7fcf9a0022c0
Created controller
Controller dispose!
argument: 0xffffffff9a0022c0
Segmentation fault: 11
Obviously the free() call fails because the pointer isn't right.
I've tried a couple of variations on this, for example, defining a structure, like:
class LEAP_CONTROLLER(Structure):
_fields_ = [
("data", c_void_p)
]
leap_controller = LEAP.leap_controller
leap_controller.res_type = POINTER(LEAP_CONTROLLER)
leap_controller_dispose = LEAP.leap_controller_dispose
leap_controller_dispose.argtype = [POINTER(LEAP_CONTROLLER)]
...but the result always seems to be the same.
Python reads the pointer, but only keeps 32 bits worth of data, and returns a broken pointer with the remaining bits all 1.
Both of the binaries are 64 bit:
Non-fat file: /usr/local/bin/python is architecture: x86_64
Non-fat file: ../../dll/libcleap.dylib is architecture: x86_64
I saw this ancient issue, here:
http://bugs.python.org/issue1703286
...but it's marked as resolved.
A few other random mailing this threads seem to mention ctypes 'truncating 64-bit pointers to 32-bit', but people all seem to have magically resolved their issues without doing anything ('it works now') or gotten no answer to their questions.
Anyone know what I'm doing wrong?
Answer as per the comments; this was a typo in my code.
Per the docs, it's argtypes and restype, not argtype and res_type. – eryksun Oct 4 at 2:31

PYTHON 3.3.1 - Using urllib to directly open a file, code gets stuck at a specific line

I'm trying to write a Python program to deal with RSS, however I'm having some issues downloading the files directly from the internet.
I am using urllib.request.urlopen() to get the files. Here is the bit of code that I am having trouble with:
import xml.etree.ElementTree as et
import urllib.request as urlget
self.sourceUrl = sourceUrl #sourceUrl was an argument
self.root = et.fromstring(urlget.urlopen(sourceUrl).read())
I have tracked the problem down to a single line:
urllib.request.urlopen calls urllib.request.opener.open()
which then calls self._open()
which then calls self._call_chain()
which then calls urllib.request.HTTPHandler.http_open()
which then calls urllib.request.AbstractHTTPHandler.do_open()
which then calls http.client.HTTPConnection.getresponse()
which then calls http.client.HTTTResponse.begin()
which then calls self._read_status()
Problem line (found by being the only line to appear upon pausing execution many times):
Python33\Lib\http\client.py Line 317
if len(line) > _MAXLINE:
I can continue the code, but only if I babysit it through Step Over until I get back to my code.
In my tests, this problem never occurred, so I can't think if why I am getting it now.
Thanks in advance for any help!
EDIT: Source can be found here. I lost motivation to work on this project quite some time ago, and haven't touched it since. I might redo the entire thing if I get some more motivation, but I don't expect to any time soon. If you wish to answer, I invite you to have at it, it might be beneficial to others. Be warned, however, that the code is terrible, as at the time I had relatively little experience. I can't really find my way around it, but I've figured out that you have to look at data/code/functions.py
Also note, that, as far as I can remember, it wasn't calling an error, it was just that the program was hanging for minutes at a time before I got impatient.
Without more code, it will be hard to help you. What is the URL of your feed. What does it return when you try to simply access it.
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.etree.ElementTree as et
>>> import urllib.request as urlget
>>> sourceurl = "http://www.la-grange.net/feed"
>>> root = et.fromstring(urlget.urlopen(sourceurl).read())
>>> root
<Element '{http://www.w3.org/2005/Atom}feed' at 0x1013a82b8>
>>>

Categories