I am fairly new in writing bigger programs in Python (I was only writing short scripts before). The program I'm about to write recives data from an external device, saves it to database and displays it in GUI (continuously). Since handling the data is time consuming I want to use threads (PySide QtCore QThreads to be specific). The best way to do this that I can think of is to set up two threads, one for the database processes and one for the handling of serial port, with GUI running in the MainThread. Now I've read a whole bunch of stuff about proper/improper QThreading, starting with Maya's post and digging deeper, up to this post where I found that:
When to subclass and when not to?
If you do not really need an event loop in the thread, you should subclass.
If you need an event loop and handle signals and slots within the thread, you may not need to subclass.
Question 1: Now the first thing I don't know is which way to go (subclassing or not). My guess is subclassing, since (Question2), but I'm not entirely sure and for now I'm sticking to moveToThread().
Question 2: The other thing I have no clue about is how to make the actions in my threads event-driven (the SerialThread receives the data from the device, sends it via signal to the DatabaseThread, the DatabaseThread collects the data and puts it in the database).
Question 3: When using moveToThread() I get AttributeError: 'PySide.QtCore.QThread' object has no attribute 'my_awesome_method' for every method I write for my class. It's clear to me that I don't understand the principle of operation here. How do I implement my methods that I want my_class to have while using my_class.moveToThread(my_thread)?
I've found all tutorials possible on QThreads and C++, so this discussion on QThreads in Python is interesting, but does not explain everything I want to know. I need a simple example with signals, slots and an exact explanation on how to use run(). Do I decorate it? Why? How? How does it work?
The short version of my code looks like this:
from PySide import QtCore
from app.thrads.gui.gui import Gui #my own class
from app.threads.db.db import Database #my own class
from app.threads.serial.serial import Serial #my own class
class AppManager():
def __init__(self):
self.gui = Gui()
self.db = Database()
self.db_thread = QtCore.QThread()
self.db.moveToThread(self.db_thread)
self.db_thread.start()
self.serial = Serial()
self.serial_thread = QtCore.QThread()
self.serial.moveToThread(self.serial_thread)
self.serial_thread.start()
and my Database and Serial class look somewhat like this:
from PySide import QtCore
from .db_signal import DbSignal
class Database(QtCore.QObject):
def __init__(self):
super(Database, self).__init__()
self.signal = DbSignal()
def my_awesome_method(self):
''' does something awesome '''
pass
Question 1
I think the best way is the easiest: don't subclass, just create two different threads. In the first, move the Database object, in the second the Serial one. As it, you won't make mistakes in the implementation of your sub-classed threads,and bug fixing will be quicker.
Question 2
I don't know the architecture of your application, but you could do as follows, after creating the threads and moving the worker objects in them:
self.serial_thread.started.connect(self.serial.start)
self.serial_thread.finished.connect(self.serial.stop)
self.db_thread.started.connect(self.db.start)
self.serial_thread.finished.connect(self.db.stop)
self.serial.data_ready.connect(self.db.handle_new_data)
# Starting the database first, as it we won't lose a single packet of data
self.db_thread.start()
self.serial_thread.start()
In fact, the advantage of QThreads is that they doesn't really modify the code.
Question 3
I think the problem is you're trying to call my_awesome_method with the QThread where your data base or your serial listener has been moved to. On the contrary, you should call the method with the object itself, as if it wasn't in a different thread !
# Bad!
obj.moveToThread(thread)
thread.method_of_obj(param)
# Good!
obj.moveToThread(thread)
obj.method_of_obj(param)
Related
The point would be to have the functionality of reading only when there is something to be read, instead of using pyserial which doesn't have a special method for that. I guess this may go into a bigger question of whether signals and slots could be used without a GUI classes (that inherit from other objects). I could get the serial port to write, but not read, with
from PyQt5 import QtCore, QtSerialPort
serial_port = QtSerialPort.QSerialPort('COM3')
serial_port.open(QtCore.QIODevice.ReadWrite)
serial_port.write(bytes([255]))
def handle_ready_read():
while serial_port.canReadLine():
print(serial_port.readAll())
print('here')
serial_port.close()
serial_port.readyRead.connect(handle_ready_read)
Nothing prints out, even though something is read when using pyserial.
You do not need GUI to use Qt. There is a dedicated GUI module in Qt and what does not depend on it, doesn't need a GUI.
However, to use slots and signals you need to run Qt's event loop. The regular way would be to use a QCoreApplication.
app = QCoreApplication([])
# setup your serial port here
sys.exit(app.exec_())
I am rookie to Python threading and am looking for guidance on how to implement mulithreading on an application with two classes, GUI and weblogic that need need to run in parallel.
The GUI class is all about user interaction and will have code like "if you click here run this weblogic method." The GUI must be in its own thread because the weblogic method can be slow and so waiting for it to complete will cause the GUI to seem unresponsive. The good news is that GUI code has no ability to modify any data inside the weblogic thread and so locking is not an issue. (The GUI just calls a weblogic method with the appropriate variables and then processes the resulting output.)
The web logic class includes multiple methods and constructors and some of the methods can be time consuming to run. The class also includes constructors that are vital to all the methods in the class. (Think userid, password, API access url and the like)
So what I am looking to implement is a way to instantiate the two threads in parallel. Hence, the GUI would run in its own thread and issue commands to the weblogic thread which would respond accordingly.
I started looking at the Threading module and while I have gotten it to work for simplistic functions, I have not been successful in running an entire class inside a thread. Since I am new to all of this, I am looking for guidance on the right strategy. Should I even be using Threading? Is there better approach?
Here is some pseudo code to illustrate what I am trying to accomplish:
GUI Code:
class GUI():
def createGUI(self):
# create GUI layout here
def button1(self):
# if clicked
outputtodisplay = weblogic.filelist()
# display outputtodisplay
def button2(self)
# if clicked assume that we have a file selector box which provides a filename
weblogic.uploadfile(filename)
# show an upload success indicator
Weblogic:
class weblogic():
def __init__(self, count, instance):
# run some stuff to setup
self.uid = xxx
self.pwd = yyy
self.url = zzz
def listfiles():
# get file list using uid, pwd, url
return output
def uploadfile(self,filename):
# upload file using uid, pwd, url
return success/failure
I have a piece of code that displays a Gui which has a QTextEdit Field. I would like to print to this field in real time similar to how the print function outputs to the console.
I have tried using multiple instances of the append function. Ex:
self.textEdit.append(_translate("MainWindow", ">>> Text", None))
The problem is that no matter where they are in the code, they seem to only show after the program is executed. My goal is to have them show in line like the print function does on the console.
I feel like this is an easy answer, but I have had no luck searching.. I am fairly new to Python and any help or guidance will be appreciated.
Thanks in advance!
Indeed, as mata mentioned the freezing comes from doing all your work in the same (main) thread, which also handles UI updates. One way to solve your responsiveness issue is indeed to frequently use QApplication.processEvents() in your blocking code. That will give the impression of a responsive GUI to the user if frequent enough.
However, using threads in Python (whether native or QThread) will not always work. That is because of the existence of the Global Interpreter Lock (GIL, the wiki has a good short intro). In short, Python does not allow more than one thread to execute code at the same time.
If your background task is light, or is based on IO, you can get around this as most IO-heavy modules for Python release the GIL while doing their job. However, if you are performing heavy computations in Python, the GIL will be locked by your processes, and as such your UI will still be unresponsive.
Consider the following example, built using PySide:
import sys, random
from threading import Thread
from time import sleep
from urllib import urlopen
from PySide import QtCore, QtGui
class Window(QtGui.QMainWindow):
update_signal = QtCore.Signal(int)
def __init__(self):
QtGui.QMainWindow.__init__(self)
self.progress_bar = QtGui.QProgressBar(self)
self.progress_bar.setRange(0, 10)
self.setCentralWidget(self.progress_bar)
self.update_signal[int].connect(self.progress_bar.setValue)
self.show()
self.t = Thread(target=self.worker)
self.t.start()
def worker(self):
while self.progress_bar.value() < 10:
self.update_signal.emit(self.progress_bar.value()+1)
print "Starting Sleep"
sleep(5)
print "End of Sleep"
if __name__ == '__main__':
qapp = QtGui.QApplication(sys.argv)
win = Window()
sys.exit(qapp.exec_())
Then, try replacing the worker function to:
def worker(self):
while self.progress_bar.value() < 10:
self.update_signal.emit(self.progress_bar.value()+1)
v = 0
print "Starting Add"
for i in xrange(5000000):
v = v+random.uniform(0, 100)
print "End of Add"
The first case maintains a responsive UI, as the call to sleep() releases the GIL. But the second example does not, as the computationally-intense algorithm keeps the lock.
One solution could be using the multiprocessing package.
From the docs:
multiprocessing is a package that supports spawning processes using an
API similar to the threading module. The multiprocessing package
offers both local and remote concurrency, effectively side-stepping
the Global Interpreter Lock by using subprocesses instead of threads.
Due to this, the multiprocessing module allows the programmer to fully
leverage multiple processors on a given machine.
A simple, illustrative example of using python multipocessing.
Also, if you have further interest, this blog post about multi-processing techniques might be of interest.
This means that you're probably doing all your work in the GUI thread. This is a common mistake, and it means that the GUI will freeze and not respond while there is something else going on.
You can add a call to QApplication.processEvents() to allow for the GUI to update after you change the text in your QTextEdit, but that will only partially solve the problem, the GUI will nevertheless freeze up between those calls.
The solution is simple: do the work in a separate thread. You should read Threading Basics from the Qt documentation, that should get you started.
There are alot of very similar questions to this but I can't find one that applies specifically to what I'm trying to do.
I have a simulation (written in SimPy) that I'm writing a GUI for, the main output of the simulation is text - to the console from 'print' statements. Now, I thought the simplest way would be to create a seperate module GUI.py, and import my simulation program into it:
import osi_model
I want all the print statements to be captured by the GUI and appear inside a Textctrl, which there's countless examples of on here, along these lines:
class MyFrame(wx.Frame):
def __init__(self, *args, **kwds):
<general frame initialisation stuff..>
redir=RedirectText(self.txtCtrl_1)
sys.stdout=redir
class RedirectText:
def __init__(self,aWxTextCtrl):
self.out=aWxTextCtrl
def write(self,string):
self.out.WriteText(string)
I am also starting my simulation from a 'Go' button:
def go_btn_click(self, event):
print 'GO'
self.RT = threading.Thread(target=osi_model.RunThis())
self.RT.start()
This all works fine, and the output from the simulation module is captured by the TextCtrl, except the GUI locks up and becomes unresponsive - I still need it to be accessible (at the very minimum to have a 'Stop' button). I'm not sure if this is a botched attempt at creating a new thread that I've done here, but I assume a new thread will be needed at some stage in this process.
People suggest using wx.CallAfter, but I'm not sure how to go about this considering the imported module doesn't know about wx, and also I can't realistically go through the entire simulation architecture and change all the print statements to wx.CallAfter, and any attempt to capture the shell from inside the imported simulation program leads to the program crashing.
Does anybody have any ideas about how I can best achieve this? So all I really need is for all console text to be captured by a TextCtrl while the GUI remains responsive, and all text is solely coming from an imported module.
(Also, secondary question regarding a Stop button - is it bad form to just kill the simulation thread?).
Thanks,
Duncan
I would suggest looking into this WX wiki article about long running tasks.
It specifically addresses the situation you're dealing with using a "start" button to being a long running process. Several different examples are given using different techniques like threads and idle handlers.
I think you would have to redirect stdout to a log file (or simple SQLite database?) and then use your thread to check the log file for updates which it would then pass along to the GUI using wx.CallAfter or similar. You might be able to use something like a socket server built in Python: http://wiki.wxpython.org/AsynchronousSockets. I think the wxPython Cookbook mentioned something about using an RPC server too (probably this one: http://docs.python.org/library/simplexmlrpcserver.html), but I don't remember the details.
You should also try asking on the official wxPython mailing list. They're very friendly over there.
I am trying to make an application in Python using PyQt that can fetch the generated content of a list of URLs and process the fetched source with the help of multiple threads. I need to run about ten QWebViews at once. As ridiculous as that might sound, when it comes to hundreds of URLs, using threaded QWebViews gets the results over 3 times faster than normal.
Here is the test code that I have been having problems with...
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQt4.QtWebKit import *
class Worker(QThread):
def __init__(self, url, frame):
QThread.__init__(self)
self.url = url
self.frame = frame
def run(self):
self.frame.load(QUrl(self.url))
print len(self.frame.page().mainFrame().toHtml())
app = QApplication(sys.argv)
webFrame = QWebView()
workerList = []
for x in range(1):
worker = Worker('http://www.google.com', webFrame)
workerList.append(worker)
for worker in workerList:
worker.start()
sys.exit(app.exec_())
Above, I tried initializing the QWebView in the main QApplication only to get:
QObject: Cannot create children for a parent that is in a different thread.
So then I tried initializing the QWebView in the QThread; but then, the QWebView remained unchanged and blank without outputting any errors or anything. This was probably due to a cache error.
I have the feeling that I am missing out on something or skipping a very important step. Since threaded QWebViews in PyQt isn't a really documented topic, I would really appreciate any help on how to successfully implement this.
There are multiple issues with your question and code:
You are talking about QWebFrame, but are actually passing a QWebView to your worker(s). Since this is a QWidget, it belongs to the main (GUI) thread and should not be modified by other threads.
One QWebView / QWebFrame can only load one URL at a time, so you cannot share it across multiple workers.
QWebFrame.load() loads data asynchronously, i.e. a call to load() returns immediately and there will be no data to read yet. You will have to wait for the loadFinished() signal to be emitted before you can access the data.
Since the actual loading is done by the networking layer of the operating system, and the load() method does not block, there is no need to run it in a separate thread in the first place. Why do you claim that this should be faster -- it makes no sense.
Since you want to load hundreds of URLs in parallel (or about 10, you are mentioning both in the same sentence), are you sure that you want to use QWebFrame, which is a presentation class? Do you actually want to render the HTML or are you just interested in the data retrieved?