I'm running a Python script that extracts and localizes some files. I'd like to use a QDialog to show the progress status through a QProgressBar, and the list of files that are being copied.
Let me start by saying that the localization script cannot be integrated in the PyQt script - otherwise I'm aware that the solution would be quite straightforward. I need to keep the localization script separated from the UI and have them running at the same time.
I thought about running the UI from the localization script through a thread, to avoid it blocking the localization process. But at that point I have no idea how to update the UI elements as I don't have an instance I can call and update since I've launched it with the thread.
This is a simplified example of the dialog code:
from PyQt5 import QtCore, QtWidgets
import sys
class Ui_dialog_main(object):
def setupUi(self, dialog_main):
dialog_main.setWindowTitle("Test")
dialog_main.resize(390, 120)
self.progress_bar = QtWidgets.QProgressBar(dialog_main)
self.progress_bar.setGeometry(QtCore.QRect(10, 60, 371, 30))
self.label_localizing = QtWidgets.QLabel(dialog_main)
self.label_localizing.setGeometry(QtCore.QRect(10, 10, 81, 25))
self.label_localizing.setText("Package:")
QtCore.QMetaObject.connectSlotsByName(dialog_main)
def start():
app = QtWidgets.QApplication(sys.argv)
dialog_main = QtWidgets.QDialog()
ui = Ui_dialog_main()
ui.setupUi(dialog_main)
dialog_main.show()
sys.exit(app.exec_())
And this is how I'm starting the thread in the localizer file:
thread = Thread(target=LocManager.start)
thread.start()
where LocManager is the name of the ui .py file.
Of course this way the main script doesn't get stuck by the ui, which is one of my goals - but I have no idea how to update the progress bar and label in this situation. I've found several threads discussing similar problems, but nothing that would exactly help with mine.
I hope my description was clear enough.
UPDATE:
I found a solution for this here, using pipes. Even if I'm not sure this is the appropriate way to tackle this issue, it definitely did the trick. Instead of working with a "Bi-directional Communication" between two PyQt GUIs, as described in the link, I modified the code to have a bi-directional communication between my GUI and my localization script.
One way to solve this is to run the dialog in a separate process, and then use some form of IPC to send the updates. The solution below uses Qt's QLocalServer and QLocalSocket classes to pass a dict encoded with json to the dialog process. The server emits a signal whenever new data is received, which the dialog connects to in order to process updates. When the sending process exits, the server process is automatically shut down.
test.py:
import time
from dlg_server import send_data
for message in 'One Two Three Four Five'.split():
send_data(message=message)
time.sleep(2)
dlg_server.py:
import sys, os, json, atexit
from PyQt5 import QtCore, QtWidgets, QtNetwork
SERVER = 'dlg_server'
_tries = 0
def send_data(**data):
socket = QtNetwork.QLocalSocket()
socket.connectToServer(SERVER, QtCore.QIODevice.WriteOnly)
if socket.waitForConnected(500):
socket.write(json.dumps(data).encode('utf-8'))
if not socket.waitForBytesWritten(2000):
raise RuntimeError('could not write to socket: %s' %
socket.errorString())
socket.disconnectFromServer()
elif socket.error() == QtNetwork.QAbstractSocket.HostNotFoundError:
global _tries
if _tries < 10:
if not _tries:
if QtCore.QProcess.startDetached(
'python', [os.path.abspath(__file__)]):
atexit.register(lambda: send_data(shutdown=True))
else:
raise RuntimeError('could not start dialog server')
_tries += 1
QtCore.QThread.msleep(100)
send_data(**data)
else:
raise RuntimeError('could not connect to server: %s' %
socket.errorString())
else:
raise RuntimeError('could not send data: %s' % socket.errorString())
class Server(QtNetwork.QLocalServer):
dataReceived = QtCore.pyqtSignal(object)
def __init__(self):
super().__init__()
self.newConnection.connect(self.handleConnection)
if not self.listen(SERVER):
raise RuntimeError(self.errorString())
def handleConnection(self):
data = {}
socket = self.nextPendingConnection()
if socket is not None:
if socket.waitForReadyRead(2000):
data = json.loads(str(socket.readAll().data(), 'utf-8'))
socket.disconnectFromServer()
socket.deleteLater()
if 'shutdown' in data:
self.close()
self.removeServer(self.fullServerName())
QtWidgets.qApp.quit()
else:
self.dataReceived.emit(data)
class Dialog(QtWidgets.QDialog):
def __init__(self):
super().__init__()
self.setGeometry(50, 50, 200, 30)
layout = QtWidgets.QVBoxLayout(self)
self.label = QtWidgets.QLabel()
layout.addWidget(self.label)
def handleDataReceived(self, data):
self.show()
self.label.setText(data.get('message', ''))
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
dialog = Dialog()
server = Server()
server.dataReceived.connect(dialog.handleDataReceived)
app.exec_()
Related
So I have a TCP server, which just echo's back whatever I send to it. And I have a GUI client which sends the stuff. But since I have to maintain the connection I can't seem to get the label I want to change once in a certain amount of time, I was trying to use signals but I seem to be doing something terribly wrong since the application freezes as soon as I connect to the server on button click. Here's what I got so far. Also, eventually I need to get 2 servers to echo to the client the information, and I guess that will pose even more of a problem and I will have to use multithreading.
from PyQt5 import QtWidgets, QtCore
from PyQt5.QtWidgets import *
import sys
import socket
import time
class MyClient(QMainWindow):
updateText = QtCore.pyqtSignal(str)
def __init__(self):
super(MyClient, self).__init__()
self.setGeometry(200, 200, 300, 300)
self.setWindowTitle("Krs")
self.initSocket()
self.initUI()
def initSocket(self):
self.ClientSocket = socket.socket()
self.s1 = ['127.0.0.1', 1233]
def initUI(self):
self.label = QLabel(self)
self.label.setText("nuthin")
self.label.move(50,50)
self.updateText.connect(self.label.setText)
self.b1 = QtWidgets.QPushButton(self)
self.b1.setText("S1")
self.b1.clicked.connect(lambda: self.conntoS(self.s1))
def conntoS(self, addrs):
self.ClientSocket.connect((addrs[0], addrs[1]))
while True:
time.sleep(1)
self.ClientSocket.send(str.encode("Anything"))
Response = self.ClientSocket.recv(1024)
self.upd(Response.decode('utf-8'))
#QtCore.pyqtSlot()
def upd(self, txt):
self.updateText.emit(txt)
def window():
app = QApplication(sys.argv)
win = MyClient()
win.show()
sys.exit(app.exec_())
window()
You should not run while True loop with time.sleep() inside gui thread, because it freezes the eventloop: while it run, widgets have no chance to recieve any events, and no chance to handle them - no repaints, no signals can be emited, no handlers invoked. All long operations should be performed outside of the gui thread.
You need to create worker, move your blocking function to it and run it separate QThread, or in your case you can use QTcpSocket asyncronous signal-slot api.
Example of the right way to use QThread in PyQt?
Python QTcpSocket and QTcpServer receiving message
First of all, I want to figure out how to check database status every second. so that the user will able to tell if the database is up or not without even clicking or triggering anything. I've read that this will create a problem as mentioned in the comments here
so here's my minimal reproducible example:
import sys
import os
import shiboken2
from PySide2 import QtCore, QtGui, QtWidgets
from PySide2.QtWidgets import QMainWindow, QFileDialog, QMessageBox, QWidget, QDialog, QProxyStyle
from sqlalchemy import create_engine, inspect
class MyWidget(QtWidgets.QWidget):
def __init__(self):
QtWidgets.QWidget.__init__(self)
self.resize(200, 200)
self.path = os.path.abspath(os.path.dirname(sys.argv[0]))
self.button = QtWidgets.QPushButton("Open File")
self.labelFile = QtWidgets.QLabel("empty")
self.labelData = QtWidgets.QLabel("None")
self.layout = QtWidgets.QVBoxLayout()
self.layout.addWidget(self.button)
self.layout.addWidget(self.labelFile)
self.layout.addWidget(self.labelData)
self.setLayout(self.layout)
self.button.clicked.connect(self.open_file)
self.process = None
self.CreateEngine = CreateEngine(self)
self.CreateEngine.result.connect(self.start_timer)
self.CreateEngine.start()
def open_file(self):
x = QFileDialog.getOpenFileName(self,"Just To Spice This Code",self.path,"CSV Files (*.csv)")
self.labelFile.setText(x[0]) #just to check that GUI doesn't freeze
def start_timer(self,engine): #callback from CreateEngine
self.timer = QtCore.QTimer(self)
self.timer.timeout.connect(lambda: self.continuously_check(engine))
self.timer.start(1000) #check connetion every second, as real-time as possible
def continuously_check(self,engine): #this gonna get called every second, yes it isn't effective i know
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.start()
def update_connection_label(self,x): #update connection status on GUI
self.labelData.setText("DB Status: "+str(x))
def closeEvent(self,event): #to handle QThread: Destroyed while thread is still running
print("begin close event")
if(self.process is not None):
if(shiboken2.isValid(self.process)): #to check whether the object is deleted. ->
self.process.wait() #-> this will get messy when the DB connection is down
self.process.quit() #-> (IMO):since i stack so many CheckConnection objects maybe?
print("end close event")
class CreateEngine(QtCore.QThread): #creating engine on seperate thread so that it wont block GUI
result = QtCore.Signal(object)
def __init__(self, parent):
QtCore.QThread.__init__(self, parent)
self.engine = None
def run(self):
self.engine = create_engine('mysql+pymysql://{}:{}#{}:{}/{}'.format("root","","localhost","3306","adex_admin"))
self.result.emit(self.engine)
class CheckConnection(QtCore.QThread): #constantly called every second, yes its not a good approach ->
result = QtCore.Signal(str) #-> i wonder how to replace all this with something appropriate
def __init__(self, parent,engine):
QtCore.QThread.__init__(self, parent)
self.engine = engine
def run(self):
try:
self.engine.execute('SELECT 1').fetchall()
self.result.emit("Connected")
except:
self.result.emit("Not Connected")
self.deleteLater() #somehow this doesn't do it job very well. maybe blocked?
#-> especially when the connection is busted. this thread gets stuck quite long to finish
if __name__ == "__main__":
#idk why when you start this without connection it's running really slow on showing the status of DB
#you must wait like 4 seconds until the connection status is showed up, which is really bad
#but once it's live. it could read database status really fast
app = QtWidgets.QApplication(sys.argv)
widget = MyWidget()
widget.show()
sys.exit(app.exec_())
I've created this example just to reproduce the same problem I'm facing in my real app. so the problem is that closeEvent takes too long to terminate the checking process and also blocking the GUI. The reason why I create 'closeEvent' is that I had this problem which produce [QThread: Destroyed while thread is still running] when the app is closed.
also, whenever the database isn't reachable it makes the QThread finishes way longer than it should unlike when the database is reachable. but we can retrieve the status pretty much like we want (every second of live DB Status). I also tried a silly approach like this
...
def continuously_check(self,engine):
self.process = CheckConnection(self,engine)
self.process.result.connect(self.update_connection_label)
self.process.finished.connect(lambda: QtCore.QTimer.singleShot(1000,self.continuously_check))
self.process.start()
...
hoping that it won't keep creating objects before the thread even finished (ps: obviously this won't work). so what's the best approach when it comes to this? sorry for multiple problems at a time.
Goal
I have a process that logs on a file (realtime.log) while running and I want to print every new line of that file in my application in realtime. In other words I want to redirect the output from the process to the GUI. This means that I have two different processes running: the "engine" and the GUI.
I have already achieved this by using Tkinter but since I have to make a more complex, professional and good looking GUI I decided to switch to Qt for Python (PySide2).
Problem
Python often crashes when I launch the GUI with the error message: Python has stopped working. The window starts printing the lines and at some point it stops working.
After many attempts and searches I got to a point where the program only crashes if I click on the GUI window. Moreover, the program doesn't crash suddenly but it crashes at the end of the engine's execution.
Environment
Windows 10
Python 3.6.5
PySide2 5.12.6
Code
Note that this is a simplified version.
datalog_path = "realtime.log"
def get_array_from_file(file_path):
try:
with open(file_path, 'r') as file:
lines = file.readlines()
return lines
except:
print('error in file access')
class Streamer(QRunnable):
def __init__(self, stream, old_array, edit):
super().__init__()
self.stream = stream
self.old_array = old_array
self.edit = edit
def run(self):
try:
while self.stream:
array_file = get_array_from_file(datalog_path)
if len(array_file) != len(self.old_array):
for line in array_file[len(self.old_array) - 1:len(array_file)]:
self.edit.append(line)
# print(line)
self.old_array.append(line)
except:
print('problem in streaming funct')
class Window(QMainWindow):
def __init__(self):
super().__init__()
layout = QVBoxLayout()
self.setWindowTitle("DATALOG")
self.thread_pool = QThreadPool()
self.edit = QTextEdit()
self.stream = True
self.old_array = get_array_from_file(datalog_path)
self.streamer = Streamer(self.stream, self.old_array, self.edit)
self.thread_pool.start(self.streamer)
window = QWidget()
layout.addWidget(self.edit)
window.setLayout(layout)
self.setCentralWidget(window)
def closeEvent(self, event):
self.stream = False
event.accept()
if __name__ == '__main__':
app = QApplication(sys.argv)
win = Window()
win.show()
app.exec_()
The #hyde answer points out explains the reason for the problem but its solution is not applicable in PySide2 (in PyQt5 a small modification would have to be made, see this), an alternative is to create a QObject that has the signals:
class Signaller(QtCore.QObject):
textChanged = Signal(str)
class Streamer(QRunnable):
def __init__(self, stream, old_array):
super().__init__()
self.stream = stream
self.old_array = old_array
self.signaller = Signaller()
def run(self):
try:
while self.stream:
array_file = get_array_from_file(datalog_path)
if len(array_file) != len(self.old_array):
for line in array_file[len(self.old_array) - 1:len(array_file)]:
self.signaller.textChanged.emit(line)
# print(line)
self.old_array.append(line)
except:
print('problem in streaming funct')
self.stream = True
self.old_array = get_array_from_file(datalog_path)
self.streamer = Streamer(self.stream, self.old_array)
self.streamer.signaller.textChanged.connect(self.edit.append)
self.thread_pool.start(self.streamer)
While I'm not too familiar with Python Qt, issue is probably, that you use a GUI object edit from a different thread. This is not allowed, the GUI part must all run in the same (main) thread!
To fix this, you need to have some other way for the thread to communicate UI changes. Since your QRunnable is not a QObject, you can't just emit a signal, but you can use QMetaObject::invokeMethod on it's invokable methods. Please let me know if this works directly:
# self.edit.append(line) # can't do this from a thread!
# instead, invoke append through GUI thread event loop
QtCore.QMetaObject.invokeMethod(self.edit,
'append',
QtCore.Qt.QueuedConnection,
QtCore.QGenericArgument('QString', line)
I've got a Python 2.7 application running with PyQt4 that has a QWebView in it, that has two way communication to and from Javascript.
The application is multithreaded via QThreadPool, QRunnables, so I'm communicating with a ViewController class with signals.
When I run the application, the QWebView loads my HTML with external JS and CSS just fine. I'm able to interact with the Javascript functions via the main program thread and ViewController class.
Once the user selects a directory and certain criteria are met, it starts looping through QRunnable tasks one at a time. During that time it calls back to the ViewController -> Javascript via Signal slots, just as expected. The problem is when I'm calling those ViewController methods that execute evaluateJavaScript, I get a Javascript error returned,
undefined line 1: SyntaxError: Parse error
I've done lots of trial error back and forth, but can't seem to figure out why evaluateJavaScript won't run in these instances. I've tried sending simple Javascript calls ranging from test functions that don't accept any arguments (thinking maybe it was some weird encoding issue), to just sending things like Application.main.evaluateJavaScript("alert('foo')"), which normally work outside of the threads. The only other thing I can think of is that maybe self.main.addToJavaScriptWindowObject('view', self.view) needs to be called in the threads again, but I've run a dir() on Application.main and it appears to have the evaluateJavaScript method attached to it already.
Any thoughts on why this could be occurring, when the scope seems to be correct, and the ViewController appears to be communicating just fine to the QWebView otherwise? Answers in Qt C++ will probably work as well, if you've seen this happen before!
I tried to simplify the code for example purposes:
# coding: utf8
import subprocess as sp
import os.path, os, sys, time, datetime
from os.path import basename
import glob
import random
import string
from PyQt4 import QtCore, QtGui, QtWebKit
from PyQt4.QtCore import QObject, pyqtSlot, QThreadPool, QRunnable, pyqtSignal
from PyQt4.QtGui import QApplication, QFileDialog
from PyQt4.QtWebKit import QWebView
from ImportController import *
class Browser(QtGui.QMainWindow):
def __init__(self):
QtGui.QMainWindow.__init__(self)
self.resize(800,500)
self.centralwidget = QtGui.QWidget(self)
self.mainLayout = QtGui.QHBoxLayout(self.centralwidget)
self.mainLayout.setSpacing(0)
self.mainLayout.setMargin(0)
self.frame = QtGui.QFrame(self.centralwidget)
self.gridLayout = QtGui.QVBoxLayout(self.frame)
self.gridLayout.setMargin(0)
self.gridLayout.setSpacing(0)
self.html = QtWebKit.QWebView()
# for javascript errors
errors = WebPage()
self.html.setPage(errors)
self.main = self.html.page().mainFrame()
self.gridLayout.addWidget(self.html)
self.mainLayout.addWidget(self.frame)
self.setCentralWidget(self.centralwidget)
path = os.getcwd()
if self.checkNetworkAvailability() and self.checkApiAvailbility():
self.default_url = "file://"+path+"/View/mainView.html"
else:
self.default_url = "file://"+path+"/View/errorView.html"
# load the html view
self.openView()
# controller class that sends and receives to/from javascript
self.view = ViewController()
self.main.addToJavaScriptWindowObject('view', self.view)
# on gui load finish
self.html.loadFinished.connect(self.on_loadFinished)
# to javascript
def selectDirectory(self):
# This evaluates the directory we've selected to make sure it fits the criteria, then parses the XML files
pass
def evaluateDirectory(self, directory):
if not directory:
return False
if os.path.isdir(directory):
return True
else:
return False
#QtCore.pyqtSlot()
def on_loadFinished(self):
# open directory select dialog
self.selectDirectory()
def openView(self):
self.html.load(QtCore.QUrl(self.default_url))
self.html.show()
def checkNetworkAvailability(self):
#TODO: make sure we can reach the outside world before trying anything else
return True
def checkApiAvailbility(self):
#TODO: make sure the API server is alive and responding
return True
class WebPage(QtWebKit.QWebPage):
def javaScriptConsoleMessage(self, msg, line, source):
print '%s line %d: %s' % (source, line, msg)
class ViewController(QObject):
def __init__(self, parent=None):
super(ViewController, self).__init__(parent)
#pyqtSlot()
def did_load(self):
print "View Loaded."
#pyqtSlot()
def selectDirectoryDialog(self):
# FROM JAVASCRIPT: in case they need to re-open the file dialog
Application.selectDirectory()
def prepareImportView(self, displayPath):
# TO JAVASCRIPT: XML directory parsed okay, so let's show the main
Application.main.evaluateJavaScript("prepareImportView('{0}');".format(displayPath))
def generalMessageToView(self, target, message):
# TO JAVASCRIPT: Send a general message to a specific widget target
Application.main.evaluateJavaScript("receiveMessageFromController('{0}', '{1}')".format(target, message))
#pyqtSlot()
def startProductImport(self):
# FROM JAVASCRIPT: Trigger the product import loop, QThreads
print "### view.startProductImport"
position = 1
count = len(Application.data.products)
importTasks = ProductImportQueue(Application.data.products)
importTasks.start()
#pyqtSlot(str)
def updateProductView(self, data):
# TO JAVASCRIPT: Send product information to view
print "### updateProductView "
Application.main.evaluateJavaScript('updateProductView("{0}");'.format(QtCore.QString(data)) )
class WorkerSignals(QObject):
''' Declares the signals that will be broadcast to their connected view methods '''
productResult = pyqtSignal(str)
class ProductImporterTask(QRunnable):
''' This is where the import process will be fired for each loop iteration '''
def __init__(self, product):
super(ProductImporterTask, self).__init__()
self.product = product
self.count = ""
self.position = ""
self.signals = WorkerSignals()
def run(self):
print "### ProductImporterTask worker {0}/{1}".format(self.position, self.count)
# Normally we'd create a dict here, but I'm trying to just send a string for testing purposes
self.signals.productResult.emit(data)
return
class ProductImportQueue(QObject):
''' The synchronous threadpool that is going to one by one run the import threads '''
def __init__(self, products):
super(ProductImportQueue, self).__init__()
self.products = products
self.pool = QThreadPool()
self.pool.setMaxThreadCount(1)
def process_result(self, product):
return
def start(self):
''' Call the product import worker from here, and format it in a predictable way '''
count = len(self.products)
position = 1
for product in self.products:
worker = ProductImporterTask("test")
worker.signals.productResult.connect(Application.view.updateProductView, QtCore.Qt.DirectConnection)
self.pool.start(worker)
position = position + 1
self.pool.waitForDone()
if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
Application = Browser()
Application.raise_()
Application.show()
Application.activateWindow()
sys.exit(app.exec_())
You know, I love PyQt4 but after searching and searching, I believe this is actually a bug and not as designed.
I've since moved on and am trying to implement this in CEFPython with WxPython, which seems to have a much more elegant implementation for this specific purpose.
I am trying to create a simple desktop app using PyQt that runs a SimpleHTTPServer on clicking a start server button. I have tried using threads(both python threads and Qthread) and understand that this is not possible as it runs into issues with the GIL. Here's the code
def btn_startserver_clicked(self):
server_thread=threading.Thread(target=start_server())
server_thread.start()
def start_server():
#to get server's IP
host=([(s.connect(('8.8.8.8', 80)), s.getsockname()[0], s.close()) for s in [socket.socket(socket.AF_INET, socket.SOCK_DGRAM)]][0][1])
start=8000
end=56999
PORT = random.randint(start,end)
print host,":",PORT
httpd=ThreadedServer(("",PORT), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.handle_request()`
I tried creating another process but the same thing happened. Also, if I create another process a new window pops up each time a request is served.
def btn_startserver_clicked(self):
if __name__=='__main__':
server_process=Process(target=start_server())
server_process.start()
Is there any way around this? I feel using multiprocessing is the right approach but I am new to this and can't figure out why it still freezes.
Thanks
The issue with your examples that lock the GUI is that rather than passing a reference to the function when creating the thread, you are actually running the function immediately and the thread is never created. For example, you should be doing:
server_thread=threading.Thread(target=start_server)
Note that I drop the brackets on start_server otherwise the code waits for start_server() to finish executing before creating the threading.Thread object, and uses the return value from start_server() as the value for the target attribute.
A final suggestion, you should really store the created thread as self.server_thread to prevent it from being garbage collected.
Not exactly sure what you're trying to do, but this might help you get started:
import sys
from urllib.request import urlopen
from http.server import HTTPServer, SimpleHTTPRequestHandler
from PyQt4 import QtCore, QtGui
HOST, PORT = '127.0.0.1', 12345
class HttpDaemon(QtCore.QThread):
def run(self):
self._server = HTTPServer((HOST, PORT), SimpleHTTPRequestHandler)
self._server.serve_forever()
def stop(self):
self._server.shutdown()
self._server.socket.close()
self.wait()
class Window(QtGui.QWidget):
def __init__(self):
super(Window, self).__init__()
self.button = QtGui.QPushButton('Start', self)
self.button.clicked.connect(self.handleButton)
layout = QtGui.QVBoxLayout(self)
layout.addWidget(self.button)
self.httpd = HttpDaemon(self)
def handleButton(self):
if self.button.text() == 'Start':
self.httpd.start()
self.button.setText('Test')
else:
urlopen('http://%s:%s/index.html' % (HOST, PORT))
def closeEvent(self, event):
self.httpd.stop()
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
window = Window()
window.show()
sys.exit(app.exec_())