We Keep Coding

How to use a timer or equivalent within a python thread without blocking the GUI

I am a writing a GUI based application using the PyQT framework which connects to a device, sends commands, reads the corresponding data and displays this in real time to a table, graph widget and to a file.
Once the run button is clicked it starts a thread which sends the external device commands according to a procedure table and emits signals with the data to various methods to change the GUI.
When the run button is clicked it executes the following lines:
worker = Worker(self.runProcedure)
worker.signals.updateResults.connect(self.updateResultsTable)
worker.signals.writeResults.connect(self.writeResultsFile)
worker.signals.finished.connect(self.procedure_complete)
self.threadpool.start(worker)
within the runProcedure commands are sent to the device from the procedure table and the data read from the device is put into a list 'hfData' using code similar to that listed below:
while float(currentForceReading) <= float(target) and stopAlarm == 0:
ts = dt.now().timestamp()
hfData = (connection.readline()).split()
updateResults_callback.emit(ts, hfData) #method to update results table and graphs
writeResults_callback.emit(ts, hfData) #method to write results to a file
One of the options in the application is to hold for a period of time where it continues collecting data from the device without sending new commands.
I am looking for a way to continue taking measurements whilst in this hold time and continue updating the GUI.
I have tried to implement the following code, however this while loop blocks the GUI from updating:
stepHoldTime = float(procedureModel.data(procedureModel.index(row,4), Qt.DisplayRole))
if(stepHoldTime != 0):
endTime = time.monotonic() + stepHoldTime
while(time.monotonic() < endTime):
ts = dt.now().timestamp()
hfData = (connection.readline()).split()
updateResults_callback.emit(ts,hfData)
Is there a correct way to implement this functionality?

Instead of while-loop you could run QTimer which will execute code every few milliseconds.
It is minimal example which shows how to run function every 1000ms and update time in label.
from PyQt5.QtWidgets import QApplication, QLabel
from PyQt5.QtCore import QTimer, QDateTime
class Window(QLabel):
def __init__(self, parent=None):
super().__init__(parent)
self.showTime()
help(QTimer)
self.timer = QTimer()
self.timer.timeout.connect(self.showTime)
self.timer.start(1000)
def showTime(self):
text = QDateTime.currentDateTime().toString('yyyy-MM-dd hh:mm:ss')
self.setText(text)
if __name__ == '__main__':
app = QApplication([])
win = Window()
win.show()
app.exec()
But your problem can be more complex and it may need more complex solution. You may have to show minimal working code which we could run and see problem - and test some ideas to resolve problem.

You can start your "reader" from a separate thread
class YourUi(QtWidgets.QMainWindow):
update_data = QtCore.pyqtSignal(str)
def __init__(self):
super(YourUi, self).__init__()
self.update_data.connect(self.do_update_data)
t = threading.Thread(target=self.update_worker, args=(self.update_data,), daemon=True)
t.start()
#staticmethod
def update_worker(signal):
connection = create_conncetion()
while True:
hfData = (connection.readline()).split()
signal.emit(hfData)
time.sleep(0.1)
def do_update_data(self, hf_data):
# access GUI elemetns from main thread to prevent freezing
self.some_qt_label.setText(str(hf_data))

Continuously Check SQLAlchemy Database Connection on Pyside2

First of all, I want to figure out how to check database status every second. so that the user will able to tell if the database is up or not without even clicking or triggering anything. I've read that this will create a problem as mentioned in the comments here
so here's my minimal reproducible example:
import sys
import os
import shiboken2
from PySide2 import QtCore, QtGui, QtWidgets
from PySide2.QtWidgets import QMainWindow, QFileDialog, QMessageBox, QWidget, QDialog, QProxyStyle
from sqlalchemy import create_engine, inspect
class MyWidget(QtWidgets.QWidget):
def __init__(self):
QtWidgets.QWidget.__init__(self)
self.resize(200, 200)
self.path = os.path.abspath(os.path.dirname(sys.argv[0]))
self.button = QtWidgets.QPushButton("Open File")
self.labelFile = QtWidgets.QLabel("empty")
self.labelData = QtWidgets.QLabel("None")
self.layout = QtWidgets.QVBoxLayout()
self.layout.addWidget(self.button)
self.layout.addWidget(self.labelFile)
self.layout.addWidget(self.labelData)
self.setLayout(self.layout)
self.button.clicked.connect(self.open_file)
self.process = None
self.CreateEngine = CreateEngine(self)
self.CreateEngine.result.connect(self.start_timer)
self.CreateEngine.start()
def open_file(self):
x = QFileDialog.getOpenFileName(self,"Just To Spice This Code",self.path,"CSV Files (*.csv)")
self.labelFile.setText(x[0]) #just to check that GUI doesn't freeze
def start_timer(self,engine): #callback from CreateEngine
self.timer = QtCore.QTimer(self)
self.timer.timeout.connect(lambda: self.continuously_check(engine))
self.timer.start(1000) #check connetion every second, as real-time as possible
def continuously_check(self,engine): #this gonna get called every second, yes it isn't effective i know
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.start()
def update_connection_label(self,x): #update connection status on GUI
self.labelData.setText("DB Status: "+str(x))
def closeEvent(self,event): #to handle QThread: Destroyed while thread is still running
print("begin close event")
if(self.process is not None):
if(shiboken2.isValid(self.process)): #to check whether the object is deleted. ->
self.process.wait() #-> this will get messy when the DB connection is down
self.process.quit() #-> (IMO):since i stack so many CheckConnection objects maybe?
print("end close event")
class CreateEngine(QtCore.QThread): #creating engine on seperate thread so that it wont block GUI
result = QtCore.Signal(object)
def __init__(self, parent):
QtCore.QThread.__init__(self, parent)
self.engine = None
def run(self):
self.engine = create_engine('mysql+pymysql://{}:{}#{}:{}/{}'.format("root","","localhost","3306","adex_admin"))
self.result.emit(self.engine)
class CheckConnection(QtCore.QThread): #constantly called every second, yes its not a good approach ->
result = QtCore.Signal(str) #-> i wonder how to replace all this with something appropriate
def __init__(self, parent,engine):
QtCore.QThread.__init__(self, parent)
self.engine = engine
def run(self):
try:
self.engine.execute('SELECT 1').fetchall()
self.result.emit("Connected")
except:
self.result.emit("Not Connected")
self.deleteLater() #somehow this doesn't do it job very well. maybe blocked?
#-> especially when the connection is busted. this thread gets stuck quite long to finish
if __name__ == "__main__":
#idk why when you start this without connection it's running really slow on showing the status of DB
#you must wait like 4 seconds until the connection status is showed up, which is really bad
#but once it's live. it could read database status really fast
app = QtWidgets.QApplication(sys.argv)
widget = MyWidget()
widget.show()
sys.exit(app.exec_())
I've created this example just to reproduce the same problem I'm facing in my real app. so the problem is that closeEvent takes too long to terminate the checking process and also blocking the GUI. The reason why I create 'closeEvent' is that I had this problem which produce [QThread: Destroyed while thread is still running] when the app is closed.
also, whenever the database isn't reachable it makes the QThread finishes way longer than it should unlike when the database is reachable. but we can retrieve the status pretty much like we want (every second of live DB Status). I also tried a silly approach like this
...
def continuously_check(self,engine):
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.finished.connect(lambda: QtCore.QTimer.singleShot(1000,self.continuously_check))
self.process.start()
...
hoping that it won't keep creating objects before the thread even finished (ps: obviously this won't work). so what's the best approach when it comes to this? sorry for multiple problems at a time.

Update widget with data from another Python process, without blocking it

I'm running a Python script that extracts and localizes some files. I'd like to use a QDialog to show the progress status through a QProgressBar, and the list of files that are being copied.
Let me start by saying that the localization script cannot be integrated in the PyQt script - otherwise I'm aware that the solution would be quite straightforward. I need to keep the localization script separated from the UI and have them running at the same time.
I thought about running the UI from the localization script through a thread, to avoid it blocking the localization process. But at that point I have no idea how to update the UI elements as I don't have an instance I can call and update since I've launched it with the thread.
This is a simplified example of the dialog code:
from PyQt5 import QtCore, QtWidgets
import sys
class Ui_dialog_main(object):
def setupUi(self, dialog_main):
dialog_main.setWindowTitle("Test")
dialog_main.resize(390, 120)
self.progress_bar = QtWidgets.QProgressBar(dialog_main)
self.progress_bar.setGeometry(QtCore.QRect(10, 60, 371, 30))
self.label_localizing = QtWidgets.QLabel(dialog_main)
self.label_localizing.setGeometry(QtCore.QRect(10, 10, 81, 25))
self.label_localizing.setText("Package:")
QtCore.QMetaObject.connectSlotsByName(dialog_main)
def start():
app = QtWidgets.QApplication(sys.argv)
dialog_main = QtWidgets.QDialog()
ui = Ui_dialog_main()
ui.setupUi(dialog_main)
dialog_main.show()
sys.exit(app.exec_())
And this is how I'm starting the thread in the localizer file:
thread = Thread(target=LocManager.start)
thread.start()
where LocManager is the name of the ui .py file.
Of course this way the main script doesn't get stuck by the ui, which is one of my goals - but I have no idea how to update the progress bar and label in this situation. I've found several threads discussing similar problems, but nothing that would exactly help with mine.
I hope my description was clear enough.
UPDATE:
I found a solution for this here, using pipes. Even if I'm not sure this is the appropriate way to tackle this issue, it definitely did the trick. Instead of working with a "Bi-directional Communication" between two PyQt GUIs, as described in the link, I modified the code to have a bi-directional communication between my GUI and my localization script.

One way to solve this is to run the dialog in a separate process, and then use some form of IPC to send the updates. The solution below uses Qt's QLocalServer and QLocalSocket classes to pass a dict encoded with json to the dialog process. The server emits a signal whenever new data is received, which the dialog connects to in order to process updates. When the sending process exits, the server process is automatically shut down.
test.py:
import time
from dlg_server import send_data
for message in 'One Two Three Four Five'.split():
send_data(message=message)
time.sleep(2)
dlg_server.py:
import sys, os, json, atexit
from PyQt5 import QtCore, QtWidgets, QtNetwork
SERVER = 'dlg_server'
_tries = 0
def send_data(**data):
socket = QtNetwork.QLocalSocket()
socket.connectToServer(SERVER, QtCore.QIODevice.WriteOnly)
if socket.waitForConnected(500):
socket.write(json.dumps(data).encode('utf-8'))
if not socket.waitForBytesWritten(2000):
raise RuntimeError('could not write to socket: %s' %
socket.errorString())
socket.disconnectFromServer()
elif socket.error() == QtNetwork.QAbstractSocket.HostNotFoundError:
global _tries
if _tries < 10:
if not _tries:
if QtCore.QProcess.startDetached(
'python', [os.path.abspath(__file__)]):
atexit.register(lambda: send_data(shutdown=True))
else:
raise RuntimeError('could not start dialog server')
_tries += 1
QtCore.QThread.msleep(100)
send_data(**data)
else:
raise RuntimeError('could not connect to server: %s' %
socket.errorString())
else:
raise RuntimeError('could not send data: %s' % socket.errorString())
class Server(QtNetwork.QLocalServer):
dataReceived = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.newConnection.connect(self.handleConnection)
if not self.listen(SERVER):
raise RuntimeError(self.errorString())
def handleConnection(self):
data = {}
socket = self.nextPendingConnection()
if socket is not None:
if socket.waitForReadyRead(2000):
data = json.loads(str(socket.readAll().data(), 'utf-8'))
socket.disconnectFromServer()
socket.deleteLater()
if 'shutdown' in data:
self.close()
self.removeServer(self.fullServerName())
QtWidgets.qApp.quit()
else:
self.dataReceived.emit(data)
class Dialog(QtWidgets.QDialog):
def __init__(self):
super().__init__()
self.setGeometry(50, 50, 200, 30)
layout = QtWidgets.QVBoxLayout(self)
self.label = QtWidgets.QLabel()
layout.addWidget(self.label)
def handleDataReceived(self, data):
self.show()
self.label.setText(data.get('message', ''))
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
dialog = Dialog()
server = Server()
server.dataReceived.connect(dialog.handleDataReceived)
app.exec_()

PyQt SimpleHTTPServer: GUI freezes on starting server

I am trying to create a simple desktop app using PyQt that runs a SimpleHTTPServer on clicking a start server button. I have tried using threads(both python threads and Qthread) and understand that this is not possible as it runs into issues with the GIL. Here's the code
def btn_startserver_clicked(self):
server_thread=threading.Thread(target=start_server())
server_thread.start()
def start_server():
#to get server's IP
host=([(s.connect(('8.8.8.8', 80)), s.getsockname()[0], s.close()) for s in [socket.socket(socket.AF_INET, socket.SOCK_DGRAM)]][0][1])
start=8000
end=56999
PORT = random.randint(start,end)
print host,":",PORT
httpd=ThreadedServer(("",PORT), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.handle_request()`
I tried creating another process but the same thing happened. Also, if I create another process a new window pops up each time a request is served.
def btn_startserver_clicked(self):
if __name__=='__main__':
server_process=Process(target=start_server())
server_process.start()
Is there any way around this? I feel using multiprocessing is the right approach but I am new to this and can't figure out why it still freezes.
Thanks

The issue with your examples that lock the GUI is that rather than passing a reference to the function when creating the thread, you are actually running the function immediately and the thread is never created. For example, you should be doing:
server_thread=threading.Thread(target=start_server)
Note that I drop the brackets on start_server otherwise the code waits for start_server() to finish executing before creating the threading.Thread object, and uses the return value from start_server() as the value for the target attribute.
A final suggestion, you should really store the created thread as self.server_thread to prevent it from being garbage collected.

Not exactly sure what you're trying to do, but this might help you get started:
import sys
from urllib.request import urlopen
from http.server import HTTPServer, SimpleHTTPRequestHandler
from PyQt4 import QtCore, QtGui
HOST, PORT = '127.0.0.1', 12345
class HttpDaemon(QtCore.QThread):
def run(self):
self._server = HTTPServer((HOST, PORT), SimpleHTTPRequestHandler)
self._server.serve_forever()
def stop(self):
self._server.shutdown()
self._server.socket.close()
self.wait()
class Window(QtGui.QWidget):
def __init__(self):
super(Window, self).__init__()
self.button = QtGui.QPushButton('Start', self)
self.button.clicked.connect(self.handleButton)
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.button)
self.httpd = HttpDaemon(self)
def handleButton(self):
if self.button.text() == 'Start':
self.httpd.start()
self.button.setText('Test')
else:
urlopen('http://%s:%s/index.html' % (HOST, PORT))
def closeEvent(self, event):
self.httpd.stop()
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())

How to prevent the GUI from freezing using QThread?

I have a GUI which needs to perform work that takes some time and I want to show the progress of this work, similar to the following:
import sys
import time
from PyQt4 import QtGui, QtCore
class MyProgress(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
# start loop with signal
self.button = QtGui.QPushButton('loop', self)
self.connect(self.button, QtCore.SIGNAL('clicked()'), self.loop)
# test button
self.test_button = QtGui.QPushButton('test')
self.connect(self.test_button, QtCore.SIGNAL('clicked()'), self.test)
self.pbar = QtGui.QProgressBar(self)
self.pbar.setMinimum(0)
self.pbar.setMaximum(100)
# layout
vbox = QtGui.QVBoxLayout()
vbox.addWidget(self.test_button)
vbox.addWidget(self.button)
vbox.addWidget(self.pbar)
self.setLayout(vbox)
self.show()
def update(self):
self.pbar.setValue(self.pbar.value() + 1)
def loop(self):
for step in range(100):
self.update()
print step
time.sleep(1)
def test(self):
if self.test_button.text() == 'test':
self.test_button.setText('ok')
else:
self.test_button.setText('test')
app = QtGui.QApplication(sys.argv)
view = MyProgress()
view.loop() # call loop directly to check whether view is displayed
sys.exit(app.exec_())
When I execute the code the loop method is called and it prints out the values as well as updates the progress bar. However the view widget will be blocked during the execution of loop and although this is fine for my application it doesn't look nice with Ubuntu. So I decided to move the work to a separate thread like this:
import sys
import time
from PyQt4 import QtGui, QtCore
class Worker(QtCore.QObject):
def __init__(self, parent=None):
QtCore.QObject.__init__(self, parent)
def loop(self):
for step in range(10):
print step
time.sleep(1)
class MyProgress(QtGui.QWidget):
def __init__(self, parent=None):
QtGui.QWidget.__init__(self, parent)
# test button
self.test_button = QtGui.QPushButton('test')
self.connect(self.test_button, QtCore.SIGNAL('clicked()'), self.test)
self.pbar = QtGui.QProgressBar(self)
self.pbar.setMinimum(0)
self.pbar.setMaximum(100)
# layout
vbox = QtGui.QVBoxLayout()
vbox.addWidget(self.test_button)
vbox.addWidget(self.pbar)
self.setLayout(vbox)
self.show()
def test(self):
if self.test_button.text() == 'test':
self.test_button.setText('ok')
else:
self.test_button.setText('test')
app = QtGui.QApplication(sys.argv)
view = MyProgress()
work = Worker()
thread = QtCore.QThread()
work.moveToThread(thread)
# app.connect(thread, QtCore.SIGNAL('started()'), work.loop) # alternative
thread.start()
work.loop() # not called if thread started() connected to loop
sys.exit(app.exec_())
When I run this version of the script the loop starts running (the steps are displayed in the terminal) but the view widget is not shown. This is the first thing I can't quite follow. Because the only difference from the previous version here is that the loop runs in a different object however the view widget is created before and therefore should be shown (as it was the case for the previous script).
However when I connected the signal started() from thread to the loop function of worker then loop is never executed although I start the thread (in this case I didn't call loop on worker). On the other hand view is shown which makes me think that it depends whether app.exec_() is called or not. However in the 1st version of the script where loop was called on view it showed the widget although it couldn't reach app.exec_().
Does anyone know what happens here and can explain how to execute loop (in a separate thread) without freezing view?
EDIT: If I add a thread.finished.connect(app.exit) the application exits immediately without executing loop. I checked out the 2nd version of this answer which is basically the same what I do. But in both cases it finishes the job immediately without executing the desired method and I can't really spot why.

The example doesn't work because communication between the worker thread and the GUI thread is all one way.
Cross-thread commnunication is usually done with signals, because it is an easy way to ensure that everything is done asynchronously and in a thread-safe manner. Qt does this by wrapping the signals as events, which means that an event-loop must be running for everything to work properly.
To fix your example, use the thread's started signal to tell the worker to start working, and then periodically emit a custom signal from the worker to tell the GUI to update the progress bar:
class Worker(QtCore.QObject):
valueChanged = QtCore.pyqtSignal(int)
def loop(self):
for step in range(0, 10):
print step
time.sleep(1)
self.valueChanged.emit((step + 1) * 10)
...
thread = QtCore.QThread()
work.moveToThread(thread)
thread.started.connect(work.loop)
work.valueChanged.connect(view.pbar.setValue)
thread.start()
sys.exit(app.exec_())