We Keep Coding

Python thread is not alive/finished running but value is not returned yet

I have a class FrameThread that inherits from Thread in python
class FrameThread(Thread):
def __init__(self, threadID, customFunction, args):
threading.Thread.__init__(self)
self.threadID = threadID
self.task = customFunction
self.args = args
self.output = None
def run(self):
self.output = self.task(self.args)
The reason I have this class is so that once the thread finishes running my custom function (checking by calling is_alive()), I can retrieve the function returned value.
My program looks something like this
In the first block of code
mythread = FrameThread(threadID, myCustomFunction, args)
mythread.start()
In the second block of code
if not mythread.is_alive():
outputVar = mythread.output
print(outputVar)
This sometimes works and sometimes doesn't. The console either prints out the actual returned value of myCustomFunction or None. So I figured this might be because the value wasn't passed to mythread.output before the python runs 'outputVar = mythread.output'. So I added a loop in the second block of code just for testing.
if not mythread.is_alive():
while mythread.output == None:
print("Reee")
outputVar = mythread.output
print(outputVar)
And to my surprise, the program printed out a bunch of "Reee"s and then printed out the correct value of outputVar.
This behavior is very weird and it's probably because I did something wrong (very likely in my FrameThread class). Please let me know how I could probably fix it, or how to implement the FrameThread subclass properly.

I've checked the code of the threading module and could not find why this is happening. Nevertheless, you can get rid of this problem if you join the thread before getting the result. You should anyway join the thread at some point. You should replace the second block of code by:
if not mythread.is_alive():
mythread.join()
outputVar = mythread.output
print(outputVar)
Using is_alive() like this is seldom useful: it is mostly meant to check if a join has timed out or not. For example:
mythread.join(0)
if mythread.is_alive():
# timed out
print('Still waiting for thread to finish...')
# do some work in the main thread...
else:
outputVar = mythread.output
print(outputVar)
I suggest that you consider using other types of synchronization or concurrent.futures

PySide QThread.terminate() causing fatal python error

I am using PySide version 1.2.2, which wraps the Qt v4.8 framework. I am in a situation where I have to choose between having my application wait for a QThread that I no longer need to exit normally (it is quite possible that the thread will block indefinitely), and giving the unresponsive thread a grace period (of several seconds), then calling QThread.terminate() on it. Though I wish I could, I cannot let the QThread object go out of scope while the underlying thread is still running, since this will throw the error "QThread: Destroyed while thread is still running" and almost surely cause a segfault.
Please note that I am aware that terminating QThreads is dangerous and highly discouraged. I am just trying to explore my options here.
When I try to terminate a thread however, my application crashes with the following error:
Fatal Python error: This thread state must be current when releasing
You can try this out yourself by copy/pasting and running the following code:
from PySide import QtCore, QtGui
class Looper(QtCore.QThread):
"""QThread that prints natural numbers, one by one to stdout."""
def __init__(self, *args, **kwargs):
super(Looper, self).__init__(*args, **kwargs)
self.setTerminationEnabled(True)
def run(self):
i = 0
while True:
self.msleep(100)
print(i)
i += 1
# Initialize and start a looper.
looper = Looper()
looper.start()
# Sleep main thread for 5 seconds.
QtCore.QThread.sleep(5)
# Terminate looper.
looper.terminate()
# After calling terminate(), we should call looper.wait() or listen
# for the QThread.terminated signal, but that is irrelevant for
# the purpose of this example.
app = QtGui.QApplication([])
app.exec_()
How do you properly terminate QThreads in Python?
I reckon that the error I am getting has got something to do with releasing of the Global Interpreter Lock, but I am not sure exactly what is going wrong, and how to fix it.

It seems that the error may be specific to PySide: running your example with PyQt4 does not produce any errors at all.
As for the general issue of how to terminate a QThread safely: it entirely depends on how much control you have over the work that is being done in the thread. If it is effectively a loop where you can periodically check a flag, then the solution is simple:
class Looper(QtCore.QThread):
...
def interrupt(self):
self._active = False
def run(self):
i = 0
self._active = True
while self._active:
self.msleep(100)
print(i)
i += 1
app = QtGui.QApplication([])
looper = Looper()
looper.finished.connect(app.quit)
looper.start()
QtCore.QTimer.singleShot(3000, looper.interrupt)
app.exec_()
The thread will finish cleanly once the run method returns, so you must find some mechanism to allow that happen. If you can't do that (perhaps because the work being done in the thread is largely outside of your control), you should probably consider switching to a multiprocessing approach instead.

Is there a way in Python to manually exit the "with" statement (context manager)

I am using a PySide (Qt) Gui that should exit a loop on a button click. A button click for PySide emits a signal and the signal calls the connected functions. However, when the signal calls the functions it uses it's own system for Error handling.
I don't really want to do the fix for that Signal Error handling.
Is there a way to break out of a "with" statement without raising errors.
import sys
import time
from PySide import QtGui, QtCore
class MyClassError(Exception): pass
class MyClass(QtGui.QProgressDialog):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Properties
self.setWindowFlags(QtCore.Qt.Dialog | QtCore.Qt.WindowTitleHint)
self.setWindowModality(QtCore.Qt.WindowModal)
self.setMinimumDuration(2000) # if progress is finished in 2 sec it wont show
self.setRange(0, 99)
# end Constructor
def breakOut(self):
print("break out")
self.__exit__(MyClassError, "User cancel!", "") # does not break out of "with"
raise MyClassError("User cancel!") # PySide just prints Exception
# end breakOut
def __enter__(self):
self.canceled.connect(self.breakOut)
return self
# end enter (with)
def __exit__(self, etype, value, traceback):
# self.canceled.disconnect(self.breakOut)
if etype is None or etype == MyClassError:
return True
raise
# end exit
# end class MyClass
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
window = QtGui.QMainWindow()
window.show()
myinst = MyClass()
window.setCentralWidget(myinst)
val = 0
with myinst:
for i in range(100):
myinst.setLabelText(str(i))
myinst.setValue(i)
val = i+1
time.sleep(0.1)
# end with
print(val)
sys.exit(app.exec_())
The fixes that I have seen for PySide/Qt error handling in signals doesn't look nice. It overrides QApplication which means that someone else using this class has to use the new QApplication.
Is there an easy way around PySide/Qt signal error handling or is there an alternative way of breaking out of a "with" statement.

Firstly, I've interpreted your comment on #dano's answer to mean you want to do some sort of long running operations within the with statement.
I think you have a few fundamental misunderstandings.
Your with statement (and the code within) is before the call to app.exec_(). The Qt Event loop does not start running until app.exec_() is called. So all your with statement is doing is queuing up events to be processed by the event loop once it is started. Thus, a qt signal to cancel the code within the with statement will never work because the signal will not be processed until the event loop is started, which is after your loop has finished (at least in the test case you provide above(.
The way to fix this is to place your with statement in a callback that is executed by the QT Event loop (eg a function that is queued up by a QTimer or similar)
Even assuming you fix the above, you will never be able to interrupt the
long running task with a button click (or a signal of any kind) because the long running task is in the same thread as the Qt Event loop, and thus the Qt Event loop cannot process any new commands (like pressing the cancel button) until your long running task finishes. While your long running task is
To fix that, you need to put your long running task in a thread (python thread or QThread) or another process. However, these approaches come with their own difficulties, largely centred around the fact that you should never attempt to update the GUI directly from a thread. There are some options to get around this, see here and here for questions that update the GUI safely from a thread.. I've wrapped some of this stuff up in a library if it helps, called qtutils.
I realise this doesn't directly solve your question of how to interrupt a with statement, however I hope I've made it clear that your current approach will not work. If you switch to threads or processes, you should be able to kill the thread/process immediately (if you wish), rather than waiting for the thread to read a condition and exit itself (though of course hard killing something may have unintended side effects, but since that is what you seem to want to do, I'll assume you've thought about this and have decided it will always be fine).

You can just check to see if it's been cancelled in the loop itself:
class MyClass(QtGui.QProgressDialog):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.cancelled = False
self.setWindowFlags(QtCore.Qt.Dialog | QtCore.Qt.WindowTitleHint)
self.setWindowModality(QtCore.Qt.WindowModal)
self.setMinimumDuration(2000) # if progress is finished in 2 sec it wont show
self.setRange(0, 99)
def breakOut(self):
print("break out")
self.cancelled = True
def __enter__(self):
self.canceled.connect(self.breakOut)
return self
def __exit__(self, etype, value, traceback):
self.canceled.disconnect(self.breakOut)
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
window = QtGui.QMainWindow()
window.show()
myinst = MyClass()
window.setCentralWidget(myinst)
val = 0
with myinst:
for i in range(100):
if myinst.cancelled:
break
myinst.setLabelText(str(i))
myinst.setValue(i)
val = i+1
time.sleep(0.1)
Raising an exception in breakOut will not have the affect you want, because it's being called in a completely different context from the for loop. Calling __exit__ won't make a with statement, regardless of PySide's error handling. I think you have the causation backwards: __exit__ gets called when you leave a with block, calling __exit__ will not force a with block to abort.

QObject::connect: Cannot queue arguments of type 'QTextCursor'

Im trying to send a signal from a non-main thread in PyQt but i dont know what am doing wrong! And when i execute the program it fails with this error:
QObject::connect: Cannot queue arguments of type 'QTextCursor'
(Make sure 'QTextCursor' is registered using qRegisterMetaType().)
here is my code:
class Sender(QtCore.QThread):
def __init__(self,q):
super(Sender,self).__init__()
self.q=q
def run(self):
while True:
pass
try: line = q.get_nowait()
# or q.get(timeout=.1)
except Empty:
pass
else:
self.emit(QtCore.SIGNAL('tri()'))
class Workspace(QMainWindow, Ui_MainWindow):
""" This class is for managing the whole GUI `Workspace'.
Currently a Workspace is similar to a MainWindow
"""
def __init__(self):
try:
from Queue import Queue, Empty
except ImportError:
while True:
#from queue import Queue, Empty # python 3.x
print "error"
ON_POSIX = 'posix' in sys.builtin_module_names
def enqueue_output(out, queue):
for line in iter(out.readline, b''):
queue.put(line)
out.close()
p= Popen(["java -Xmx256m -jar bin/HelloWorld.jar"],cwd=r'/home/karen/sphinx4-1.0beta5-src/sphinx4-1.0beta5/',stdout=PIPE, shell=True, bufsize= 4024)
q = Queue()
t = threading.Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()
self.sender= Sender(q)
self.connect(self.sender, QtCore.SIGNAL('tri()'), self.__action_About)
self.sender.start()
I think that my way of send parameter to the thread is wrong...
I need to know how to send parameters to a thread, in my case i need to send q to the worker thread.

Quite new to PyQt5, but this appears to happen when you try to do a GUI operation from a thread which is not the "application thread". I put this in quotes because it appears to be a mistake to think that, even in a fairly simple PyQt5 app, QApplication.instance().thread() will always return the same object.
The thing to do is to use the signal/slot mechanism to send any kind of data from a worker thread (a thread created in my case by extending QtCore.QRunnable, one other pattern apparently being QtCore.QThread and QtCore.QObject.moveToThread, see here).
Then also include a check in all your slot methods which are likely to receive data from a non-"application thread". Example which logs messages visually during execution:
def append_message(self, message):
# this "instance" method is very useful!
app_thread = QtWidgets.QApplication.instance().thread()
curr_thread = QtCore.QThread.currentThread()
if app_thread != curr_thread:
raise Exception('attempt to call MainWindow.append_message from non-app thread')
ms_now = datetime.datetime.now().isoformat(sep=' ', timespec='milliseconds')
self.messages_text_box.insertPlainText(f'{ms_now}: {message}\n')
# scroll to bottom
self.messages_text_box.moveCursor(QtGui.QTextCursor.End)
It's all too easy to just call this inadvertently and directly from a non-"application thread".
Making such a mistake then raise an exception is good, because it gives you a stack trace showing the culprit call. Then change the call so that it instead sends a signal to the GUI class, the slot for which could be the method in the GUI class (here append_message), or alternatively one which then in turn calls append_message.
In my example I've included the "scroll to bottom" line above because it was only when I added that line that these "cannot queue" errors started happening. In other words, it is perfectly possible to get away with a certain amount of non-compliant handling (in this case adding some more text with each call) without any error being raised... and only later do you then run into difficulties. To prevent this, I suggest that EVERY method in a GUI class with GUI functionality should include such a check!

Make sure 'QTextCursor' is registered using qRegisterMetaType().
Did you try to use qRegisterMetaType function?
The official manual says:
The class is used as a helper to marshall types in QVariant and in
queued signals and slots connections. It associates a type name to a
type so that it can be created and destructed dynamically at run-time.
Declare new types with Q_DECLARE_METATYPE() to make them available to
QVariant and other template-based functions. Call qRegisterMetaType()
to make type available to non-template based functions, such as the
queued signal and slot connections.

I would like to add the following notes to the #mike rodent's post which solved my problem (I'm using PyQt5):
Custom signals and slots can be used to avoid directly modifying GUI from thread other than "application thread" (I'm using Python threading module and the equivalent there to that is probably "main thread"). I find this website very useful for basic custom signal and slot setup. Pay attention to using a class (and not an instance) attribute.
To avoid the QObject::connect: Cannot queue arguments of type 'QTextCursor' message I needed to find the following locations and add some code:
Before the function __init__ of the class MainWindow: definition of class attribute; I needed to use something like class_attribute = pyqtSignal(str).
In the function __init__: self.class_attribute.connect(self.slot_name)
Inside of a thread (I mean the thread which is not the main thread): self.class_attribute.emit(str)
In the slot inside the main thread: "safety mechanism" proposed by #mike rodent.

How to thread wxPython progress bar

I'm trying to thread wx.ProgressDialog. I got a Progress threading class
class Progress(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
max = 1000000
dlg = wx.ProgressDialog("Progress dialog example",
"An informative message",
maximum = max,
parent=None,
style = wx.PD_CAN_ABORT
| wx.PD_APP_MODAL
| wx.PD_ELAPSED_TIME
| wx.PD_ESTIMATED_TIME
| wx.PD_REMAINING_TIME
)
keepGoing = True
count = 0
while keepGoing and count < max:
count += 1
wx.MilliSleep(250)
if count >= max / 2:
(keepGoing, skip) = dlg.Update(count, "Half-time!")
else:
(keepGoing, skip) = dlg.Update(count)
dlg.Destroy()
which gets called when I push a button by
class MiPPanel ( wx.Panel ):
[...]
def runmiP(self, event):
thread1 = Progress()
thread1.start()
When I run thread1.start() I get 100s of warnings of the type 2012-12-01 00:31:19.215 Python[3235:8807] *** __NSAutoreleaseNoPool(): Object 0x11a88f300 of class NSConcreteAttributedString autoreleased with no pool in place - just leaking
and the progress bar doesn't show up.
How can I use threading with wxPython to make a progress bar?

All wxPython widgets and manipulation should be in a single thread. If you want to have a dialog controlled by another thread then you will have to use timers and queues to message the dialog from the other thread.
Another way I understand is supposed to work (I have not tested this) it to create a completely separate wxApp in another thread just for your dialog. You will have to communicate somehow back to the main thread still.
Edit:
Here is a link to more information. It has some info at the bottom about using wx.CallAfter to update progress of a worker thread. It also shows how to run a single function in a separate thread without creating a separate class.
wxPython Threading

It looks like it's complaining about argument 4 not being a wxWindow (hence the argument 4 of type 'wxWindow *' part of the error).
Looking at your call, argument 4 is passing parent=self. So, what is self? In other words, what class is the method with this code a part of? Is it something that's a wx.Window or subclass of one, or is it something like an Application or a Progress that isn't?
(Actually, given that you're passing parent as a keyword argument, it's just a coincidence that it happened to be in the 4th position in your call to the wx.ProgressDialog constructor and also be in the 4th position to the call to the underlying C++ function, which is what's actually complaining. You'd get the same error if you switched maximum and parent, and I think it would still say argument 4 instead of argument 3.)
To verify this, try taking out the parent=self in the call that fails, and it should work, or adding parent=object() to the call with just None that works, and it should fail.
Of course that doesn't actually fix the problem. But to do that, you have to figure out what you wanted the parent to be and pass that instead of self.
PS, I don't think the problem has anything to do with your threading at all.