So I wrote this software which used to implement 4 tabs (now 8). Now, every tab is its own QThread, plus one general GUI thread, obviously. The worker threads (tab threads) send data back to the GUI thread when this needs updating. For example, I have a statusBar text in the GUI thread which needs to be updated every time worker thread does something. I'm using the SIGNAL/SLOT for that, and the way I detect an incoming SIGNAL is like this:
def setStatus(self, status):
#Defining SIGNAL sender
self.sender = QObject().sender()
if "module1" in str(self.sender):
self.module1CurrentStatus.setText(status)
Now this works and all, but like I said, the software has grown from implementing 4 tabs to implementing 8 tabs. Is there a better way to implement this, instead of using sender()? I'm simply looking for something that can be written only once for all the modules, a way for the GUI to know automatically which thread the SIGNAL came from. I know that I could write a separate SIGNAL for every module, but I accomplish nothing with that.
If I understand your requirements correctly, you could use a QSignalMapper to re-emit the signals with an identifier.
The module name can be acquired via __name__, unless it is the file being run as a script .
Related
I have been trying to find the most elegant way to decouple my programs from the GUI, such that I can change my front-end without needing to re-write a whole lot of code.
I work with threads a lot, so I often have the need to notify the main GUI thread of asynchronous happenings either through events (for wxPython) or signals (for PyQt). I have experimented a bit with PyPubSub, which may be what I am looking for, but while there are tons of wxPython examples (since it was originally included with it in early development).
I am not aware if there is a 'proper' way to use it with PyQt without running into race conditions. If anyone has some insight on this, I would really appreciate it!
PyPubSub's sendMessage() will call listeners in the same thread as the sender (default Python behavior). In a multithreaded GUI app, you must ensure that listeners that interact with GUI are called in the main thread. Also threads execute independently, so you need each thread to call its own listeners, based on a timed or idle callback mechanism.
The way to call listeners in the correct thread in PyQt is via signals. PyPubSub can still be used in a multithreaded PyQt GUI, but the mechanism used to transfer the "message" from sender to listener would have to be via a signal. There isn't one best way to do it I don't think, depends on details of your app design. You could for example have a QtPubsubHandler that derives from QObject and gets created in main thread, and a QtPubsubSender class that also derives from QObject and gets created in each worker thread. The QtPubSubSender defines a custom signal, say pubsub, which QtPubsubHandler connects to. Then to send a message, the sender does qtPubsubHandler.sendMessage(topic, **data), which causes a pubsub signal to get emitted, which Qt properly queues and eventually signals the QtPubsubHandler, which actually calls pub.sendMessage().
There are many other ways, but you get the general idea: two classes that derive from QObject, and one of them does the actual sending in the same thread as the intended listener, the other uses a signal so everything is thread safe. Actually you don't have to use PyQt signals: but then you would have to have a queue in main thread and have an idle callback that allows it to process any items on the queue.
This is my first foray into threading, so apologies for any obvious mistakes.
I have a PyQt widget, from which a new process, prog, is run in a different thread. In my main thread, I'm also redirecting stdout to a read-only QTextEdit. However, I get errors referring to recursion, and I'm worried that my threads are interfering in each other in a way which causes a print statement to go into an infinite loop. I only get these errors if I run prog from the GUI, and not from the command line. My stdout redirect is using the code in this SO answer
In pseudo-code, this is basically what I've got:
gui.py
class widget(QWidget):
def __init__(self):
self.button = QPushButton("GO!", self)
self.button.clicked.connect(self.start)
def start(self):
self.thread = TaskThread()
sys.stdout = EmittingStream(textWritten = self.outputText)
self.thread.start()
def outputText(self):
#as in answer provided in link (EmittingStream in separate module)
prog.py
class TaskThread(QThread):
def run(self):
'''
Long complicated program; putting in simpler code here (e.g. loop printing to 10000) doesn't reproduce errors
'''
Is there any way of finding out if my recursion is caused by an infinite loop, or by anything else?
Is my code obviously thread-unsafe?
How do you make functions guaranteed to be threadsafe? (Links to tutorials / books will be good!)
This is tricky, but I think that your code is thread-unsafe. Specifically, looking at other stackoverflow answers (here and here) it appears that you should not be accessing a Qt GUI object from another thread than the one it was created in (even a QThread).
Since any call to print in your code now accesses a Qt GUI object, it seems this is very thread unsafe.
My suggestion to make it safe would be to:
Have a QThread, inside of which you have instantiated the output box (it is thread safe to access a Qt GUI object from the thread it was created in, and it does not have to be created in the main thread) Qt GUI objects are not reentrant and must be created and used in the main thread only, see here As such you will need a QThread to post events back to the main thread (using the Qt signals/slots mechanism which can be thread safe when done correctly)
Have this QThread blocking on reading from a Python Queue. When it gets something from the queue, it places it in the Qt text box posts it back to the main thread, and the main thread will update the output box.
Modify your EmmittingStream to place things in the Queue, rather than directly into the Qt output box.
I can see you have located where the error is. But without the code there is not much that I can tell.
Filling your needs of direction, I'll point to you to Python profiles. Since it looks like you need some python profiling tools.
http://docs.python.org/2/library/profile.html
and a answer about the subject
How can you profile a Python script?
In a Qt application you must use one single thread to handle the all the gui part. You can use other threads for computations, but not for user interface.
Just post messages about updates in a queue in the worker threads and use the main thread to pick up those messages and update the GUI.
I've run into situations as of late when writing scripts for both Maya and Houdini where I need to wait for aspects of the GUI to update before I can call the rest of my Python code. I was thinking calling time.sleep in both situations would have fixed my problem, but it seems that time.sleep just holds up the parent application as well. This means my script evaluates the exact same regardless of whether or not the sleep is in there, it just pauses part way through.
I have a thought to run my script in a separate thread in Python to see if that will free up the application to still run during the sleep, but I haven't had time to test this yet.
Thought I would ask in the meantime if anybody knows of some other solution to this scenario.
Maya - or more precisely Maya Python - is not really multithreaded (Python itself has a dodgy kind of multithreading because all threads fight for the dread global interpreter lock, but that's not your problem here). You can run threaded code just fine in Maya using the threading module; try:
import time
import threading
def test():
for n in range (0, 10):
print "hello"
time.sleep(1)
t = threading.Thread(target = test)
t.start()
That will print 'hello' to your listener 10 times at one second intervals without shutting down interactivity.
Unfortunately, many parts of maya - including most notably ALL user created UI and most kinds of scene manipulation - can only be run from the "main" thread - the one that owns the maya UI. So, you could not do a script to change the contents of a text box in a window using the technique above (to make it worse, you'll get misleading error messages - code that works when you run it from the listener but errors when you call it from the thread and politely returns completely wrong error codes). You can do things like network communication, writing to a file, or long calculations in a separate thread no problem - but UI work and many common scene tasks will fail if you try to do them from a thread.
Maya has a partial workaround for this in the maya.utils module. You can use the functions executeDeferred and executeInMainThreadWithResult. These will wait for an idle time to run (which means, for example, that they won't run if you're playing back an animation) and then fire as if you'd done them in the main thread. The example from the maya docs give the idea:
import maya.utils import maya.cmds
def doSphere( radius ):
maya.cmds.sphere( radius=radius )
maya.utils.executeInMainThreadWithResult( doSphere, 5.0 )
This gets you most of what you want but you need to think carefully about how to break up your task into threading-friendly chunks. And, of course, running threaded programs is always harder than the single-threaded alternative, you need to design the code so that things wont break if another thread messes with a variable while you're working. Good parallel programming is a whole big kettle of fish, although boils down to a couple of basic ideas:
1) establish exclusive control over objects (for short operations) using RLocks when needed
2) put shared data into safe containers, like Queue in #dylan's example
3) be really clear about what objects are shareable (they should be few!) and which aren't
Here's decent (long) overview.
As for Houdini, i don't know for sure but this article makes it sound like similar issues arise there.
A better solution, rather than sleep, is a while loop. Set up a while loop to check a shared value (or even a thread-safe structure like a Queue). The parent processes that your waiting on can do their work (or children, it's not important who spawns what) and when they finish their work, they send a true/false/0/1/whatever to the Queue/variable letting the other processes know that they may continue.
I'm learning PyQt (I haven't used C++ Qt before).
I don't understand the reason for signals and slots. That is, it seems I can do everything by calling the methods of classes directly. Doing so seems more Pythonic, uses less code and is easier to read. What am I missing?
For clarification, I'm asking why I would do this:
def mouseReleaseEvent(self):
self.clicksignal.connect(ui.iconClicked)
self.clicksignal.emit()
when I can do this:
def mouseReleaseEvent(self):
ui.iconClicked()
The reason signals and slots exist is because you cannot change the GUI from any other thread of the application except the UI thread.
If you have some heavy CPU intensive calculation to do, or any task that waits for IO or something like that... if you do it the UI thread (for example if you fetch a url or something that lasts some time), your UI thread will be busy, and the GUI event loop wont be able to update itself, so the GUI will seem frozen.
To do these kind of operations. you execute them in a separate (background worker) thread, so the UI thread can continue updating the GUI.
Now the problem is that you cannot access the elements of the GUI and change their state from any other thread except the UI thread. So signals and slots are introduced. When you emit a signal it is guaranteed to be caught in the UI thread, and the slot will be executed in the UI thread.
I'm not exactly sure what you try to accomplish in your example, but this is the main reason why signals and slots exist. Basically UI thread should only deal with the UI, and everything else should be done in the background worker thread that sends a signal that gets caught in the UI thread and the slot that update the GUI is executed in the UI thread.
Well... yes, you can. But you need to think bigger. In your example code, caller of mouseReleaseEvent must have the reference to the object that receives the notification, and explicitly invoke appropriate method. Using slot & signals mechanism decouples event producer (e.g. widget) from event consumer - pretty much arbitrary other object. This makes setting up communication and control flow easier and external to the low-level UI components, which is the good thing. It also makes such components reusable - by moving wiring code outside, we make it independent of the application logic.
In addition to the answers by #ViktorKerkez and #Wilbur, signals and slots provide a notification system of fire and forget, as well as decoupling classes.
A great advantage of this is that a class can emit a signal, not knowing what or who is going to receive the message. It may be just one object that has a slot connected, or it could be dozens. Alternatively, you may want a single class with one slot that is connected to multiple signals. So it can be used as a notification system.
For example, imagine a program where many different types of objects send information to a log. The objects simply emit a Log(text) signal and do not care about what actually does the logging. These signals can be connected to a log class which can be either logging to a file, over a network, to the screen, or all at once. The objects logging don't care.
I'm writing a program, with a PyQt frontend. To ensure that the UI doesn't freeze up, I use QThreads to emit signals back to the parent. Now, I have reached a point where I need my thread to stop running, emit a signal back to the parent, then wait on the parent to return an approval for the thread to continue (after the user interacts with the UI a little bit).
I've been looking into the QMutex class, along with QThread's wait function.
How should I go about doing this properly?
One approach is using a condition variable.
In my code, however, I prefer using Python's built-in Queue objects to synchronize data between threads. While I'm at it, I use Python's threads as opposed to PyQt threads, mainly because it allows me to reuse the non-GUI part of the code without an actual GUI.