Goal
I have a process that logs on a file (realtime.log) while running and I want to print every new line of that file in my application in realtime. In other words I want to redirect the output from the process to the GUI. This means that I have two different processes running: the "engine" and the GUI.
I have already achieved this by using Tkinter but since I have to make a more complex, professional and good looking GUI I decided to switch to Qt for Python (PySide2).
Problem
Python often crashes when I launch the GUI with the error message: Python has stopped working. The window starts printing the lines and at some point it stops working.
After many attempts and searches I got to a point where the program only crashes if I click on the GUI window. Moreover, the program doesn't crash suddenly but it crashes at the end of the engine's execution.
Environment
Windows 10
Python 3.6.5
PySide2 5.12.6
Code
Note that this is a simplified version.
datalog_path = "realtime.log"
def get_array_from_file(file_path):
try:
with open(file_path, 'r') as file:
lines = file.readlines()
return lines
except:
print('error in file access')
class Streamer(QRunnable):
def __init__(self, stream, old_array, edit):
super().__init__()
self.stream = stream
self.old_array = old_array
self.edit = edit
def run(self):
try:
while self.stream:
array_file = get_array_from_file(datalog_path)
if len(array_file) != len(self.old_array):
for line in array_file[len(self.old_array) - 1:len(array_file)]:
self.edit.append(line)
# print(line)
self.old_array.append(line)
except:
print('problem in streaming funct')
class Window(QMainWindow):
def __init__(self):
super().__init__()
layout = QVBoxLayout()
self.setWindowTitle("DATALOG")
self.thread_pool = QThreadPool()
self.edit = QTextEdit()
self.stream = True
self.old_array = get_array_from_file(datalog_path)
self.streamer = Streamer(self.stream, self.old_array, self.edit)
self.thread_pool.start(self.streamer)
window = QWidget()
layout.addWidget(self.edit)
window.setLayout(layout)
self.setCentralWidget(window)
def closeEvent(self, event):
self.stream = False
event.accept()
if __name__ == '__main__':
app = QApplication(sys.argv)
win = Window()
win.show()
app.exec_()
The #hyde answer points out explains the reason for the problem but its solution is not applicable in PySide2 (in PyQt5 a small modification would have to be made, see this), an alternative is to create a QObject that has the signals:
class Signaller(QtCore.QObject):
textChanged = Signal(str)
class Streamer(QRunnable):
def __init__(self, stream, old_array):
super().__init__()
self.stream = stream
self.old_array = old_array
self.signaller = Signaller()
def run(self):
try:
while self.stream:
array_file = get_array_from_file(datalog_path)
if len(array_file) != len(self.old_array):
for line in array_file[len(self.old_array) - 1:len(array_file)]:
self.signaller.textChanged.emit(line)
# print(line)
self.old_array.append(line)
except:
print('problem in streaming funct')
self.stream = True
self.old_array = get_array_from_file(datalog_path)
self.streamer = Streamer(self.stream, self.old_array)
self.streamer.signaller.textChanged.connect(self.edit.append)
self.thread_pool.start(self.streamer)
While I'm not too familiar with Python Qt, issue is probably, that you use a GUI object edit from a different thread. This is not allowed, the GUI part must all run in the same (main) thread!
To fix this, you need to have some other way for the thread to communicate UI changes. Since your QRunnable is not a QObject, you can't just emit a signal, but you can use QMetaObject::invokeMethod on it's invokable methods. Please let me know if this works directly:
# self.edit.append(line) # can't do this from a thread!
# instead, invoke append through GUI thread event loop
QtCore.QMetaObject.invokeMethod(self.edit,
'append',
QtCore.Qt.QueuedConnection,
QtCore.QGenericArgument('QString', line)
Related
I have a Qt Python program that logs data over a serial port. I'd like this program to always log data while running even when the application is not visible. Currently, when the application is not visible, the logging will pause after about ~45 seconds. Once the application window becomes visible again, logging resumes. The logging portion of the code is in a second thread using QRunnable and QThreadPool.
I've tried searching for the cause (or solution), but have not had much luck. Part of my problem is that I'm not sure if this issue is related to the OS, IDE, language, etc.
High-level details:
OS: macOS 12.4
IDE: vscode
Language/frameworks: Python3 / Qt (pyside6)
Does anyone have any ideas on why this application/thread might be pausing? Is it possible to have the application to continue to log data even when it is not visible? My hope is that once I'm pointed in the right direction I'll be able to address the issue.
UPDATE
Example code
class LogSignals(QObject):
result = Signal(dict)
class LogWorker(QRunnable):
def __init__():
super().__init__()
self.signals = LogSignals()
def run(self):
try:
for i in range(N_LOG_SAMPLES):
result = self.getSerialData()
self.signals.result.emit(result)
except:
traceback.print_exc()
exctype, value = sys.exc_info()[:2]
class MainWindow(QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
self.threadpool = QThreadPool()
def startLog(self):
log_worker = LogWorker()
log_worker.signals.result.connect(self.updateLogData)
self.threadpool.start(log_worker)
#Slot()
def updateLogData(self, result: dict):
#save data
app = QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec()
You would need to check if your application is active, and start or drop the logging accordingly. You can get this by using:
QWidget *QApplication::activeWindow()
This would return a nullptr if your application does not have an active window. So, you would write something like this:
if (application->activeWindow())
; // Start / Keep logging
else
; // Stop logging
First of all, I want to figure out how to check database status every second. so that the user will able to tell if the database is up or not without even clicking or triggering anything. I've read that this will create a problem as mentioned in the comments here
so here's my minimal reproducible example:
import sys
import os
import shiboken2
from PySide2 import QtCore, QtGui, QtWidgets
from PySide2.QtWidgets import QMainWindow, QFileDialog, QMessageBox, QWidget, QDialog, QProxyStyle
from sqlalchemy import create_engine, inspect
class MyWidget(QtWidgets.QWidget):
def __init__(self):
QtWidgets.QWidget.__init__(self)
self.resize(200, 200)
self.path = os.path.abspath(os.path.dirname(sys.argv[0]))
self.button = QtWidgets.QPushButton("Open File")
self.labelFile = QtWidgets.QLabel("empty")
self.labelData = QtWidgets.QLabel("None")
self.layout = QtWidgets.QVBoxLayout()
self.layout.addWidget(self.button)
self.layout.addWidget(self.labelFile)
self.layout.addWidget(self.labelData)
self.setLayout(self.layout)
self.button.clicked.connect(self.open_file)
self.process = None
self.CreateEngine = CreateEngine(self)
self.CreateEngine.result.connect(self.start_timer)
self.CreateEngine.start()
def open_file(self):
x = QFileDialog.getOpenFileName(self,"Just To Spice This Code",self.path,"CSV Files (*.csv)")
self.labelFile.setText(x[0]) #just to check that GUI doesn't freeze
def start_timer(self,engine): #callback from CreateEngine
self.timer = QtCore.QTimer(self)
self.timer.timeout.connect(lambda: self.continuously_check(engine))
self.timer.start(1000) #check connetion every second, as real-time as possible
def continuously_check(self,engine): #this gonna get called every second, yes it isn't effective i know
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.start()
def update_connection_label(self,x): #update connection status on GUI
self.labelData.setText("DB Status: "+str(x))
def closeEvent(self,event): #to handle QThread: Destroyed while thread is still running
print("begin close event")
if(self.process is not None):
if(shiboken2.isValid(self.process)): #to check whether the object is deleted. ->
self.process.wait() #-> this will get messy when the DB connection is down
self.process.quit() #-> (IMO):since i stack so many CheckConnection objects maybe?
print("end close event")
class CreateEngine(QtCore.QThread): #creating engine on seperate thread so that it wont block GUI
result = QtCore.Signal(object)
def __init__(self, parent):
QtCore.QThread.__init__(self, parent)
self.engine = None
def run(self):
self.engine = create_engine('mysql+pymysql://{}:{}#{}:{}/{}'.format("root","","localhost","3306","adex_admin"))
self.result.emit(self.engine)
class CheckConnection(QtCore.QThread): #constantly called every second, yes its not a good approach ->
result = QtCore.Signal(str) #-> i wonder how to replace all this with something appropriate
def __init__(self, parent,engine):
QtCore.QThread.__init__(self, parent)
self.engine = engine
def run(self):
try:
self.engine.execute('SELECT 1').fetchall()
self.result.emit("Connected")
except:
self.result.emit("Not Connected")
self.deleteLater() #somehow this doesn't do it job very well. maybe blocked?
#-> especially when the connection is busted. this thread gets stuck quite long to finish
if __name__ == "__main__":
#idk why when you start this without connection it's running really slow on showing the status of DB
#you must wait like 4 seconds until the connection status is showed up, which is really bad
#but once it's live. it could read database status really fast
app = QtWidgets.QApplication(sys.argv)
widget = MyWidget()
widget.show()
sys.exit(app.exec_())
I've created this example just to reproduce the same problem I'm facing in my real app. so the problem is that closeEvent takes too long to terminate the checking process and also blocking the GUI. The reason why I create 'closeEvent' is that I had this problem which produce [QThread: Destroyed while thread is still running] when the app is closed.
also, whenever the database isn't reachable it makes the QThread finishes way longer than it should unlike when the database is reachable. but we can retrieve the status pretty much like we want (every second of live DB Status). I also tried a silly approach like this
...
def continuously_check(self,engine):
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.finished.connect(lambda: QtCore.QTimer.singleShot(1000,self.continuously_check))
self.process.start()
...
hoping that it won't keep creating objects before the thread even finished (ps: obviously this won't work). so what's the best approach when it comes to this? sorry for multiple problems at a time.
I'm running a Python script that extracts and localizes some files. I'd like to use a QDialog to show the progress status through a QProgressBar, and the list of files that are being copied.
Let me start by saying that the localization script cannot be integrated in the PyQt script - otherwise I'm aware that the solution would be quite straightforward. I need to keep the localization script separated from the UI and have them running at the same time.
I thought about running the UI from the localization script through a thread, to avoid it blocking the localization process. But at that point I have no idea how to update the UI elements as I don't have an instance I can call and update since I've launched it with the thread.
This is a simplified example of the dialog code:
from PyQt5 import QtCore, QtWidgets
import sys
class Ui_dialog_main(object):
def setupUi(self, dialog_main):
dialog_main.setWindowTitle("Test")
dialog_main.resize(390, 120)
self.progress_bar = QtWidgets.QProgressBar(dialog_main)
self.progress_bar.setGeometry(QtCore.QRect(10, 60, 371, 30))
self.label_localizing = QtWidgets.QLabel(dialog_main)
self.label_localizing.setGeometry(QtCore.QRect(10, 10, 81, 25))
self.label_localizing.setText("Package:")
QtCore.QMetaObject.connectSlotsByName(dialog_main)
def start():
app = QtWidgets.QApplication(sys.argv)
dialog_main = QtWidgets.QDialog()
ui = Ui_dialog_main()
ui.setupUi(dialog_main)
dialog_main.show()
sys.exit(app.exec_())
And this is how I'm starting the thread in the localizer file:
thread = Thread(target=LocManager.start)
thread.start()
where LocManager is the name of the ui .py file.
Of course this way the main script doesn't get stuck by the ui, which is one of my goals - but I have no idea how to update the progress bar and label in this situation. I've found several threads discussing similar problems, but nothing that would exactly help with mine.
I hope my description was clear enough.
UPDATE:
I found a solution for this here, using pipes. Even if I'm not sure this is the appropriate way to tackle this issue, it definitely did the trick. Instead of working with a "Bi-directional Communication" between two PyQt GUIs, as described in the link, I modified the code to have a bi-directional communication between my GUI and my localization script.
One way to solve this is to run the dialog in a separate process, and then use some form of IPC to send the updates. The solution below uses Qt's QLocalServer and QLocalSocket classes to pass a dict encoded with json to the dialog process. The server emits a signal whenever new data is received, which the dialog connects to in order to process updates. When the sending process exits, the server process is automatically shut down.
test.py:
import time
from dlg_server import send_data
for message in 'One Two Three Four Five'.split():
send_data(message=message)
time.sleep(2)
dlg_server.py:
import sys, os, json, atexit
from PyQt5 import QtCore, QtWidgets, QtNetwork
SERVER = 'dlg_server'
_tries = 0
def send_data(**data):
socket = QtNetwork.QLocalSocket()
socket.connectToServer(SERVER, QtCore.QIODevice.WriteOnly)
if socket.waitForConnected(500):
socket.write(json.dumps(data).encode('utf-8'))
if not socket.waitForBytesWritten(2000):
raise RuntimeError('could not write to socket: %s' %
socket.errorString())
socket.disconnectFromServer()
elif socket.error() == QtNetwork.QAbstractSocket.HostNotFoundError:
global _tries
if _tries < 10:
if not _tries:
if QtCore.QProcess.startDetached(
'python', [os.path.abspath(__file__)]):
atexit.register(lambda: send_data(shutdown=True))
else:
raise RuntimeError('could not start dialog server')
_tries += 1
QtCore.QThread.msleep(100)
send_data(**data)
else:
raise RuntimeError('could not connect to server: %s' %
socket.errorString())
else:
raise RuntimeError('could not send data: %s' % socket.errorString())
class Server(QtNetwork.QLocalServer):
dataReceived = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.newConnection.connect(self.handleConnection)
if not self.listen(SERVER):
raise RuntimeError(self.errorString())
def handleConnection(self):
data = {}
socket = self.nextPendingConnection()
if socket is not None:
if socket.waitForReadyRead(2000):
data = json.loads(str(socket.readAll().data(), 'utf-8'))
socket.disconnectFromServer()
socket.deleteLater()
if 'shutdown' in data:
self.close()
self.removeServer(self.fullServerName())
QtWidgets.qApp.quit()
else:
self.dataReceived.emit(data)
class Dialog(QtWidgets.QDialog):
def __init__(self):
super().__init__()
self.setGeometry(50, 50, 200, 30)
layout = QtWidgets.QVBoxLayout(self)
self.label = QtWidgets.QLabel()
layout.addWidget(self.label)
def handleDataReceived(self, data):
self.show()
self.label.setText(data.get('message', ''))
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
dialog = Dialog()
server = Server()
server.dataReceived.connect(dialog.handleDataReceived)
app.exec_()
I am making an instrument interface in Qt5 and it works fine. The only issue is that it's slow to start because the interface __init__ contains a time-consuming method (5-10 seconds) used to connect to the instrument. Currently, nothing shows up for several seconds then the whole interface shows up with the "successfully connected to the instrument" message already written in its console (a textEdit widget).
What I would like is to have the interface show up instantly, and only after it's shown it should start the communication protocol. I am sure this is just a matter of moving one line around, but I can't figure it out. Any help is appreciated.
Here is a minimal example of the program structure:
# ================================================
# Interface management.
# ================================================
class RENAMEMELATER(Ui_renamemetoo, QObject):
def __init__(self, parent):
super(Ui_renamemetoo, self).__init__()
self.ui = Ui_renamemetoo()
self.ui.setupUi(parent)
# Redirect IDE console towards GUI console.
sys.stdout = EmittingStream()
sys.stdout.textWritten.connect(self.redirect_console_messages)
sys.stderr = EmittingStream()
sys.stderr.textWritten.connect(self.redirect_console_messages)
# Initialize PC/instrument communication (MOVE SOMEWHERE ELSE?)
self.T = TaborSE5082("USB0::0x168C::0x5082::0000218391::INSTR") # TIME CONSUMING.
def redirect_console_messages(self, text):
"""All print() from the program are appended on a textEdit
instead of the IDE console."""
self.ui.Console_textEdit.append(text.rstrip("\n"))
def close_program(self):
"""Call those functions after the user clicked on exit."""
self.T.CLOSE()
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
print("Program terminated.")
# ================================================
# Program execution.
# ================================================
if __name__ == "__main__":
# Define the app.
if not QtWidgets.QApplication.instance():
app = QtWidgets.QApplication(sys.argv)
else:
app = QtWidgets.QApplication.instance()
# Start the interface.
Form = QtWidgets.QWidget()
prog = RENAMEMELATER(Form)
Form.show()
# Handle what happens at program exit.
app.setQuitOnLastWindowClosed(True)
app.aboutToQuit.connect(prog.close_program)
# Launch.
app.exec()
In the main I can use app.aboutToQuit to close the instrument. Maybe there is some sort of app.isDoneLoading that I could .connect to my instrument initialization in the same way?
Thank you.
A task that takes 5 to 10 seconds is heavy so apart from not showing the GUI you can freeze it so the solution is to run it in another thread:
def __init__(self, parent):
# ...
threading.Thread(target=self.callback, daemon=True).start()
def callback(self):
self.T = TaborSE5082("USB0::0x168C::0x5082::0000218391::INSTR")
# another code
So i am trying to run a PyQT GUI while another functions is gathering information in the background. If Information is found the GUI should update itself.
I am new in Threading so i googled a lot and found some good HowTo's although it does not work as it should.
when i run the program it just ends itself after 3 s.
Maybe you see some major mistake ive done.
Here is the basic code i am trying to get to run
class scan_Thread(QThread):
def __init__(self, samp_rate, band, speed, ppm, gain, args, prn):
QThread.__init__(self)
self.samp_rate=samp_rate
self.band=band
self.speed=speed
self.ppm=ppm
self.gain=gain
self.args=args
self.prn=prn
def __del__(self):
self.wait()
def run(self):
do_scan(self.samp_rate, self.band, self.speed,
self.ppm, self.gain, self.args, self.prn)
def start_gui():
app = QtGui.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())
#app.exec_()
#sys.exit()
def main(options = None):
def printfunc(found_list):
for info in sorted(found_list):
print info
get_thread = scan_Thread(options.samp_rate, options.band, options.speed,
options.ppm, options.gain, options.args, printfunc)
get_thread.start()
start_gui()
Thanks!
Many of the objects of the Qt classes and therefore of PyQt need to start some object of type Application (QCoreApplication, QtGuiApplication or QApplication), but only one of these objects must exist.
In your particular case QThread needs it. The previous classes are responsible for generating the necessary loops.
So you should modify your code to the following:
def main(options = None):
app = QtGui.QApplication(sys.argv) // before creating any PyQt object.
def printfunc(found_list):
for info in sorted(found_list):
print info
get_thread = scan_Thread(options.samp_rate, options.band, options.speed,
options.ppm, options.gain, options.args, printfunc)
get_thread.start()
window = Window()
window.show()
sys.exit(app.exec_())