We Keep Coding

Compatibility Between PyPubSub and PyQt

I have been trying to find the most elegant way to decouple my programs from the GUI, such that I can change my front-end without needing to re-write a whole lot of code.
I work with threads a lot, so I often have the need to notify the main GUI thread of asynchronous happenings either through events (for wxPython) or signals (for PyQt). I have experimented a bit with PyPubSub, which may be what I am looking for, but while there are tons of wxPython examples (since it was originally included with it in early development).
I am not aware if there is a 'proper' way to use it with PyQt without running into race conditions. If anyone has some insight on this, I would really appreciate it!

PyPubSub's sendMessage() will call listeners in the same thread as the sender (default Python behavior). In a multithreaded GUI app, you must ensure that listeners that interact with GUI are called in the main thread. Also threads execute independently, so you need each thread to call its own listeners, based on a timed or idle callback mechanism.
The way to call listeners in the correct thread in PyQt is via signals. PyPubSub can still be used in a multithreaded PyQt GUI, but the mechanism used to transfer the "message" from sender to listener would have to be via a signal. There isn't one best way to do it I don't think, depends on details of your app design. You could for example have a QtPubsubHandler that derives from QObject and gets created in main thread, and a QtPubsubSender class that also derives from QObject and gets created in each worker thread. The QtPubSubSender defines a custom signal, say pubsub, which QtPubsubHandler connects to. Then to send a message, the sender does qtPubsubHandler.sendMessage(topic, **data), which causes a pubsub signal to get emitted, which Qt properly queues and eventually signals the QtPubsubHandler, which actually calls pub.sendMessage().
There are many other ways, but you get the general idea: two classes that derive from QObject, and one of them does the actual sending in the same thread as the intended listener, the other uses a signal so everything is thread safe. Actually you don't have to use PyQt signals: but then you would have to have a queue in main thread and have an idle callback that allows it to process any items on the queue.

Restart QThread with GUI

I am using QThread to do some calculations in a separate Thread.
The Thread gets started by a button click, witch launches the function StartMeasurement().
The Thread can finish the process by itself (after finished the calculations)
and emits the PyQT Signal finished. Or the thread can be stopped by the User by the stopBtn click.
The terminate() function is working, but I get a lot of troubles when I try to start the thread again.
Is it recommendable to use the movetoThread() approach here?
Or how could I ensure that the thread is stopped correctly to enable a proper restart. (means, starting new!)
# starts the measurment in a Thread: StartMeasurement()
def StartMeasurement(self):
self.thread = measure.CMeasurementThread(self.osziObj, self.genObj, self.measSetup)
self.thread.newSample.connect(self.plotNewSample)
self.thread.finished.connect(self.Done)
self.stopBtn.clicked.connect(self.thread.terminate)
self.stopBtn.clicked.connect(self.Stop)
self.thread.start()

It's not a problem. The general practice when working with QThread is to connect its finished() signal to the deleteLater() slot of the objects that have been moved to the separate thread via moveToThread(). It's done in order to properly manage the memory when you then destroy your thread because it's assumed that you will first quit the thread and then destroy its instance. ;) This should tell you that stopping a thread has nothing to do with the destruction of those objects UNLESS you have established the connection I've described above.
It is perfectly fine to restart a thread IF you have stopped it properly using quit() and wait() to actually wait untill the stopping is completed.
However my advice is to not do that unless that extra thread has a huge impact on your application for some reason (highly unlikely with modern machines).
Instead of restarting the thread consider the following options:
implement a pause flag that just makes the thread run without doing anything if it's set to true (I've used this example of mine many times to demonstrate such behaviour (check the worker.cpp and the doWork() function in particular) - it's in C++ but it can be ported to PyQt in no time)
use QRunnable - its designed to run something and then (unless autoDelete is set to true) return to the thread pool. It's really nice if you have tasks that occur every once in a while and you don't need a constatly running separate thread. If you want to use signals and slots (to get the result of the calculation done inside the QRunnable::run() you will have to first inherit from QObject and then from QRunnable
Use futures (check the Qt Concurrent module)
I suggest that you first read the Example use cases for the various threading technologies Qt provides.

Checking threadsafe-ness in python?

This is my first foray into threading, so apologies for any obvious mistakes.
I have a PyQt widget, from which a new process, prog, is run in a different thread. In my main thread, I'm also redirecting stdout to a read-only QTextEdit. However, I get errors referring to recursion, and I'm worried that my threads are interfering in each other in a way which causes a print statement to go into an infinite loop. I only get these errors if I run prog from the GUI, and not from the command line. My stdout redirect is using the code in this SO answer
In pseudo-code, this is basically what I've got:
gui.py
class widget(QWidget):
def __init__(self):
self.button = QPushButton("GO!", self)
self.button.clicked.connect(self.start)
def start(self):
self.thread = TaskThread()
sys.stdout = EmittingStream(textWritten = self.outputText)
self.thread.start()
def outputText(self):
#as in answer provided in link (EmittingStream in separate module)
prog.py
class TaskThread(QThread):
def run(self):
'''
Long complicated program; putting in simpler code here (e.g. loop printing to 10000) doesn't reproduce errors
'''
Is there any way of finding out if my recursion is caused by an infinite loop, or by anything else?
Is my code obviously thread-unsafe?
How do you make functions guaranteed to be threadsafe? (Links to tutorials / books will be good!)

This is tricky, but I think that your code is thread-unsafe. Specifically, looking at other stackoverflow answers (here and here) it appears that you should not be accessing a Qt GUI object from another thread than the one it was created in (even a QThread).
Since any call to print in your code now accesses a Qt GUI object, it seems this is very thread unsafe.
My suggestion to make it safe would be to:
Have a QThread, inside of which you have instantiated the output box (it is thread safe to access a Qt GUI object from the thread it was created in, and it does not have to be created in the main thread) Qt GUI objects are not reentrant and must be created and used in the main thread only, see here As such you will need a QThread to post events back to the main thread (using the Qt signals/slots mechanism which can be thread safe when done correctly)
Have this QThread blocking on reading from a Python Queue. When it gets something from the queue, it places it in the Qt text box posts it back to the main thread, and the main thread will update the output box.
Modify your EmmittingStream to place things in the Queue, rather than directly into the Qt output box.

I can see you have located where the error is. But without the code there is not much that I can tell.
Filling your needs of direction, I'll point to you to Python profiles. Since it looks like you need some python profiling tools.
http://docs.python.org/2/library/profile.html
and a answer about the subject
How can you profile a Python script?

In a Qt application you must use one single thread to handle the all the gui part. You can use other threads for computations, but not for user interface.
Just post messages about updates in a queue in the worker threads and use the main thread to pick up those messages and update the GUI.

Why Signals and Slots in PyQt?

I'm learning PyQt (I haven't used C++ Qt before).
I don't understand the reason for signals and slots. That is, it seems I can do everything by calling the methods of classes directly. Doing so seems more Pythonic, uses less code and is easier to read. What am I missing?
For clarification, I'm asking why I would do this:
def mouseReleaseEvent(self):
self.clicksignal.connect(ui.iconClicked)
self.clicksignal.emit()
when I can do this:
def mouseReleaseEvent(self):
ui.iconClicked()

The reason signals and slots exist is because you cannot change the GUI from any other thread of the application except the UI thread.
If you have some heavy CPU intensive calculation to do, or any task that waits for IO or something like that... if you do it the UI thread (for example if you fetch a url or something that lasts some time), your UI thread will be busy, and the GUI event loop wont be able to update itself, so the GUI will seem frozen.
To do these kind of operations. you execute them in a separate (background worker) thread, so the UI thread can continue updating the GUI.
Now the problem is that you cannot access the elements of the GUI and change their state from any other thread except the UI thread. So signals and slots are introduced. When you emit a signal it is guaranteed to be caught in the UI thread, and the slot will be executed in the UI thread.
I'm not exactly sure what you try to accomplish in your example, but this is the main reason why signals and slots exist. Basically UI thread should only deal with the UI, and everything else should be done in the background worker thread that sends a signal that gets caught in the UI thread and the slot that update the GUI is executed in the UI thread.

Well... yes, you can. But you need to think bigger. In your example code, caller of mouseReleaseEvent must have the reference to the object that receives the notification, and explicitly invoke appropriate method. Using slot & signals mechanism decouples event producer (e.g. widget) from event consumer - pretty much arbitrary other object. This makes setting up communication and control flow easier and external to the low-level UI components, which is the good thing. It also makes such components reusable - by moving wiring code outside, we make it independent of the application logic.

In addition to the answers by #ViktorKerkez and #Wilbur, signals and slots provide a notification system of fire and forget, as well as decoupling classes.
A great advantage of this is that a class can emit a signal, not knowing what or who is going to receive the message. It may be just one object that has a slot connected, or it could be dozens. Alternatively, you may want a single class with one slot that is connected to multiple signals. So it can be used as a notification system.
For example, imagine a program where many different types of objects send information to a log. The objects simply emit a Log(text) signal and do not care about what actually does the logging. These signals can be connected to a log class which can be either logging to a file, over a network, to the screen, or all at once. The objects logging don't care.

wxpython -- threads and window events

I have a wxPython application (http://www.OpenSTV.org) that counts ballots using methods that have multiple rounds. I'd like to do two things:
(1) For a large number of ballots, this can be a bit slow, so I'd like to show the user a progress dialog so he doesn't think the application is frozen.
(2) I'd like to allow the user to break ties manually, and this requires the counting code to show a dialog window.
To achieve (1), I create a thread to run the counting code, and this allows me to present a nice progress dialog to the user.
The problem with this, however, is that the counting code is not the main thread, and only the main thread in wxPython can process window events.
I suppose I could create a thread to run the progress dialog instead, but this seems awkward. Is there a better way of accomplishing both (1) and (2)?

Use Queue to communicate and synchronize among threads, with each thread "owning" and exclusively interacting with a resource that's not handy to share.
In GUI toolkits where only the main thread can really handle the GUI, the main thread should play along -- set up and start the threads doing the actual work, then do nothing but GUI work, using Queues to communicate to and from the other threads.
For (1), when your counting thread has an update, it should put it to the Queue where the main thread is waiting; when your main thread gets a suitable message on that Queue, it updates the progress dialog.
For (2), the counting thread sends the "have the user break a tie" request, main thread gets it and responds appropriately, and sends back the resolution on a separate Queue.
So in general, there are two kinds of communications: one that don't require a response, and others that do. For the former kind, just put the notification on the appropriate queue and simply proceed -- it will be acted on in due course. For the latter kind, my favorite idiom is to put on the appropriate queue a pair (request, response_queue). If otherwise identical requests differ in that some need a response and others don't, queueing (request, None) when no response is needed (and (request, q) where q's a Queue when a response IS needed) is a nice, easy, and general idiom, too.

There are several ways to call the main thread wxPython thread from a process thread. The simplest is wx.CallAfter() which will always execute the functional passed to it in the main thread. You can also use wx.PostEvent() and there's an example of this in the demo (labeled: Threads), and there are several more complicated but more customizable ways which are discussed in the last chapter of wxPython in Action.