I have a PyQt5 QWebEnginePage with a QWebEngineUrlRequestInterceptor. After running app.exec_(), the Interceptor works as intended, but after the page finishes loading, that is, the callback
in self.loadFinished.connect(self._on_load_finished) is executed, and self.app.quit() is ran, the QWebEngineUrlRequestInterceptor.interceptRequest() function is called again, resulting in the Error Received signal 11 <unknown> 000000000000
and the script crashing.
class WebEngineUrlRequestInterceptor(QWebEngineUrlRequestInterceptor):
def __init__(self, on_network_call):
super().__init__()
self.on_network_call = on_network_call
def interceptRequest(self, info):
self.on_network_call(info)
class PyQtWebClient(QWebEnginePage):
def __init__(self, url):
self.app = QApplication(sys.argv)
interceptor = WebEngineUrlRequestInterceptor(self.on_network_call)
profile = QWebEngineProfile()
profile.setRequestInterceptor(interceptor)
super().__init__(profile, None)
self.loadFinished.connect(self._on_load_finished)
self.html = ""
self.network_requests = {}
self.load(QUrl(url))
self.app.exec_()
def on_network_call(self, info):
# Something ...
def _on_load_finished(self):
self.toHtml(self.callable)
def callable(self, html_str):
self.html = html_str
self.app.quit()
Tried PyQt5.11.2 and PyQt5.10.1
I expected one of two things:
- self.loadFinished should not be called if there are still pending requests on the page.
- if the self.loadFinished is called and my app exists, the interceptor's thread should stop.
loadFinished indicates that the content of the page has finished loading as indicated by the docs:
void QWebEnginePage::loadFinished(bool ok)
This signal is emitted when the page finishes loading content. This
signal is independent of script execution or page rendering. ok will
indicate whether the load was successful or any error occurred.
But that does not mean that the page keeps making requests, for example you could make requests through AJAX, so do not confuse those concepts.
In the case QWebEngineUrlRequestInterceptor could be solving pending requests since that part is not handled by Qt but by chromium.
One problem that I see in your code is that QWebEngineProfile is destroyed before QWebEnginePage is destroyed causing problems. The solution in this case is to make the profile and interceptor members of the class.
class PyQtWebClient(QWebEnginePage):
def __init__(self, url):
self.app = QApplication(sys.argv)
self.interceptor = WebEngineUrlRequestInterceptor(self.on_network_call)
self.profile = QWebEngineProfile()
self.profile.setRequestInterceptor(self.interceptor)
super().__init__(self.profile, None)
# ...
Finally I recommend using the latest version of PyQt5 5.13.0 and PyQtWebEngine 5.13.0 as it brings improvements such as thread-safe and page specific url request interceptors.
Related
I'm attempting to use QThreads to update my custom tool's Qt-based UI inside of Maya. I have a thread that executes arbitrary methods and returns the result via an emitted signal, which I then use to update my UI. Here's my custom QThread class:
from PySide import QtCore
class Thread(QtCore.QThread):
result = QtCore.Signal(object)
def __init__(self, parent, method, **kwargs):
super(Thread, self).__init__(parent)
self.parent = parent
self.method = method
self.kwargs = kwargs
def run(self):
result = self.method(**self.kwargs)
self.result.emit(result)
The methods I'm passing to the thread are basic requests for getting serialized data from a web address, for example:
import requests
def request_method(address):
request = requests.get(address)
return request.json()
And here is how I use the thread in my custom tool to dynamically update my UI:
...
thread = Thread(parent=self, method=request_method, address='http://www.example.com/')
thread.result.connect(self._slot_result)
thread.start()
def _slot_result(self, result):
# Use the resulting data to update some UI element:
self.label.setText(result)
...
This workflow works in other DCCs like Nuke, but for some reason it causes Maya to sometimes crash inconsistently. No error message, no log, just a hard crash.
This makes me think that my QThread workflow design is obviously not Maya-friendly. Any ideas how best to avoid crashing Maya when using QThreads and what may be causing this particular issue?
This doesn't answer directly what's going on with your QThread, but to show you another way to go about threading with guis in Maya.
Here's a simple example of a gui that has a progress bar and a button. When the user clicks the button it will create a bunch of worker objects on a different thread to do a time.sleep(), and will update the progress bar as they finish. Since they're on a different thread it won't lock the user from the gui so they can still interact with it as it updates:
from functools import partial
import traceback
import time
from PySide2 import QtCore
from PySide2 import QtWidgets
class Window(QtWidgets.QWidget):
"""
Your main gui class that contains a progress bar and a button.
"""
def __init__(self, parent=None):
super(Window, self).__init__(parent)
# Create our main thread pool object that will handle all the workers and communication back to this gui.
self.thread_pool = ThreadPool(max_thread_count=5) # Change this number to have more workers running at the same time. May need error checking to make sure enough threads are available though!
self.thread_pool.pool_started.connect(self.thread_pool_on_start)
self.thread_pool.pool_finished.connect(self.thread_pool_on_finish)
self.thread_pool.worker_finished.connect(self.worker_on_finish)
self.progress_bar = QtWidgets.QProgressBar()
self.button = QtWidgets.QPushButton("Run it")
self.button.clicked.connect(partial(self.thread_pool.start, 30)) # This is the number of iterations we want to process.
self.main_layout = QtWidgets.QVBoxLayout()
self.main_layout.addWidget(self.progress_bar)
self.main_layout.addWidget(self.button)
self.setLayout(self.main_layout)
self.setWindowTitle("Thread example")
self.resize(500, 0)
def thread_pool_on_start(self, count):
# Triggers right before workers are about to be created. Start preparing the gui to be in a "processing" state.
self.progress_bar.setValue(0)
self.progress_bar.setMaximum(count)
def thread_pool_on_finish(self):
# Triggers when all workers are done. At this point you can do a clean-up on your gui to restore it to it's normal idle state.
if self.thread_pool._has_errors:
print "Pool finished with no errors!"
else:
print "Pool finished successfully!"
def worker_on_finish(self, status):
# Triggers when a worker is finished, where we can update the progress bar.
self.progress_bar.setValue(self.progress_bar.value() + 1)
class ThreadSignals(QtCore.QObject):
"""
Signals must inherit from QObject, so this is a workaround to signal from a QRunnable object.
We will use signals to communicate from the Worker class back to the ThreadPool.
"""
finished = QtCore.Signal(int)
class Worker(QtCore.QRunnable):
"""
Executes code in a seperate thread.
Communicates with the ThreadPool it spawned from via signals.
"""
StatusOk = 0
StatusError = 1
def __init__(self):
super(Worker, self).__init__()
self.signals = ThreadSignals()
def run(self):
status = Worker.StatusOk
try:
time.sleep(1) # Process something big here.
except Exception as e:
print traceback.format_exc()
status = Worker.StatusError
self.signals.finished.emit(status)
class ThreadPool(QtCore.QObject):
"""
Manages all Worker objects.
This will receive signals from workers then communicate back to the main gui.
"""
pool_started = QtCore.Signal(int)
pool_finished = QtCore.Signal()
worker_finished = QtCore.Signal(int)
def __init__(self, max_thread_count=1):
QtCore.QObject.__init__(self)
self._count = 0
self._processed = 0
self._has_errors = False
self.pool = QtCore.QThreadPool()
self.pool.setMaxThreadCount(max_thread_count)
def worker_on_finished(self, status):
self._processed += 1
# If a worker fails, indicate that an error happened.
if status == Worker.StatusError:
self._has_errors = True
if self._processed == self._count:
# Signal to gui that all workers are done.
self.pool_finished.emit()
def start(self, count):
# Reset values.
self._count = count
self._processed = 0
self._has_errors = False
# Signal to gui that workers are about to begin. You can prepare your gui at this point.
self.pool_started.emit(count)
# Create workers and connect signals to gui so we can update it as they finish.
for i in range(count):
worker = Worker()
worker.signals.finished.connect(self.worker_finished)
worker.signals.finished.connect(self.worker_on_finished)
self.pool.start(worker)
def launch():
global inst
inst = Window()
inst.show()
Aside from the main gui, there's 3 different classes.
ThreadPool: This is responsible to create and manage all worker objects. This class is also responsible to communicate back to the gui with signals so it can react accordingly while workers are completing.
Worker: This is what does the actual heavy lifting and whatever you want to process in the thread.
ThreadSignals: This is used inside the worker to be able to communicate back to the pool when it's done. The worker class isn't inherited by QObject, which means it can't emit signals in itself, so this is used as a work around.
I know this all looks long winded, but it seems to be working fine in a bunch of different tools without any hard crashes.
One of the engineers at our studio discovered a few bugs related to the use of Python threads and PyQt/PySide. Please refer to:
[PySide 1.x] https://bugreports.qt.io/browse/PYSIDE-810
[PySide 2.x] https://bugreports.qt.io/browse/PYSIDE-813
Notes from the reporter:
Although QObject is reentrant, the GUI classes, notably QWidget and all its subclasses, are not reentrant. They can only be used from the main thread.
I'm new to python and in my project I have completed implementing a function which has implementation to get data from a micro-controller (sampling data over UART to my PC).
Which takes several seconds, it depends on how many samples I wish to collect. This works fine with straightforward python script.
However, I wish to implement a GUI and I chose PyQt to do it. All I want to do is call the function when a button is pressed.
I will try to explain what I want to achieve in sequential steps below:
Click the button.
Button is disabled.
Call the function collectDataFromUART().
Wait for/detect the data collection to complete (Several Seconds)
Re-enable the button.
I have a button clicked handler, as shown below:
self.ui.pushButton1.clicked.connect(self.handlepushButton1)
def handlepushButton1(self):
self.ui.textEdit1.append("Started")
collectDataFromUART()
What I'm not able to understand is how to detect the completion of the function collectDataFromUART() and only then re-enable the button.
Can anyone throw light on this? Examples will be really very helpful.
Help! Thank you.
I suggest put collectDataFromUART() in another thread and emit message when it done. Something like this:
mysignal = QtCore.pyqtSignal()
class NamedThread(QtCore.QThread):
def __init__(self, parent=None):
super(self.__class__, self).__init__(parent)
def run(self):
<some code from collectDataUART>
self.mysignal.emit()
class NamedWidget(QtGui.QWidget):
def __init__(self, parent=None):
<some code>
self.thread = NamedThread()
self.pushButton1.clicked.connect(self.handlepushButton1)
self.thread.mysignal.connect(lambda: self.pushButton1.setEnabled(True), QtCore.Qt.QueuedConnection)
def handlepushButton1(self):
self.pushButton1.setDisabled(True)
self.thread.start()
You also can add some information about status of execution in signal. To do so you need pyqtSiglnal([type of data you want to send]) after that just call emit with some data self.mysignal[type of data].emit(<data>)
For my opinion, sound like your should handle it by create QThread to receive your UART data. First, your have initiate thread for receive your UART data and close button. Next, wait for this thread. If success or fail, Create signal send back to main widget. Last, Handle signal data it and do want your want;
Little example;
import sys
import time
from PyQt4 import QtGui
from PyQt4 import QtCore
def collectDataFromUART ():
# Your collect data from UART
time.sleep(1)
data = 'UART'
return data
class QCustomThread (QtCore.QThread):
status = QtCore.pyqtSignal(object, object)
def __init__ (self, parentQWidget = None):
super(QCustomThread, self).__init__(parentQWidget)
def run (self):
try:
data = collectDataFromUART()
errorCode = None
except Exception, error:
data = None
errorCode = str(error)
self.status.emit(data, errorCode)
self.exit(0)
class QCustomMainWindow (QtGui.QMainWindow):
def __init__ (self):
super(QCustomMainWindow, self).__init__()
self.startQPushButton = QtGui.QPushButton('START')
self.startQPushButton.released.connect(self.requestWork)
self.setCentralWidget(self.startQPushButton)
def requestWork (self):
self.startQPushButton.setEnabled(False)
myQCustomThread = QCustomThread(self)
myQCustomThread.status.connect(self.relpyWork)
myQCustomThread.start()
def relpyWork (self, data, errorCode):
self.startQPushButton.setEnabled(True)
if errorCode == None:
QtGui.QMessageBox.information(self, 'Information', data)
else:
QtGui.QMessageBox.critical(self, 'Critical', errorCode)
myQApplication = QtGui.QApplication(sys.argv)
myQCustomMainWindow = QCustomMainWindow()
myQCustomMainWindow.show()
sys.exit(myQApplication.exec_())
I trying to write a simple parser, which checks some web pages and if content on these pages is changed, then script sends url to the headless webkit browser which runs using PySide binding to Qt and makes a screenshot. I want this browser always running in the background separate process waiting for url to appear in queue. As soon as url comes, it makes screenshot, saves it and then returns to waiting.
I try to implement this behavior with this code(i cut the parser part):
import multiprocessing
import sys
from datetime import datetime
from PySide import QtGui, QtWebKit, QtCore
class Browser(QtWebKit.QWebPage):
def __init__(self, queue_in, queue_out):
self.app = QtGui.QApplication(sys.argv)
QtWebKit.QWebPage.__init__(self)
self.queue_out = queue_out
self.queue_in = queue_in
self.setViewportSize(QtCore.QSize(900, 900))
self.mainFrame().setScrollBarPolicy(QtCore.Qt.Horizontal, QtCore.Qt.ScrollBarAlwaysOff)
self.mainFrame().setScrollBarPolicy(QtCore.Qt.Vertical, QtCore.Qt.ScrollBarAlwaysOff)
self.mainFrame().loadFinished.connect(self._makeScreenshot)
self.makeScreenshotOf()
def makeScreenshotOf(self):
self.mainFrame().setUrl(QtCore.QUrl.fromEncoded(self.queue_in.get()))
def _makeScreenshot(self):
image = QtGui.QImage(self.viewportSize(), QtGui.QImage.Format_ARGB32)
painter = QtGui.QPainter(image)
self.mainFrame().render(painter)
painter.end()
file_name = datetime.now().strftime("%Y-%m-%d %H-%M-%S-%f") + ".png"
image.save(file_name)
self.queue_out.put(file_name)
self.makeScreenshotOf()
if __name__ == "__main__":
multiprocessing.set_start_method('spawn')
queue_in = multiprocessing.Queue()
queue_out = multiprocessing.Queue()
t = threading.Thread(target = Browser, args = (queue_in, queue_out))
t.start()
queue_in.put(url)
The problem is that on the first run process successfully stays on hold, awaiting url to appear in queue, but as soon as it gets url, process just stops, ignoring Qt connection
self.mainFrame().loadFinished.connect(self._makeScreenshot)
The thing is that if i directly inherit from Process
class Browser(multiprocessing.Process):
def __init__(self, queue_in, queue_out):
multiprocessing.Process.__init__(self)
self.queue_out = queue_out
self.queue_in = queue_in
self.app = QtGui.QApplication(sys.argv)
self.browser = QtWebKit.QWebPage()
...
if __name__ == "__main__":
queue_in = multiprocessing.Queue()
queue_out = multiprocessing.Queue()
b = Browser(queue_in, queue_out)
Then connection is not ignored and all works perfectly, but as a side effect self.queue_in.get() invoked in a Browser process also blocks main process (if queue is empty).
Questions:
Why the Qt connection is not working in the first case and working in the other?
Why in the second case queue.get() blocks the main process? How to prevent this?
Queue.get() blocks if the queue is empty. Use get_nowait(), which will raise an exception if there's nothing there.
Well it seems that call to app.exec_() was essential. Everything works now. Except that i get
warning qapplication was not created in the main()
Despite everything works anyway, i decided to move it into the main
app = QtGui.QApplication(sys.argv)
browser = Browser(queue_in, queue_out)
app.exec_()
and run parser part in a separate process.
UPD
Figured how to run QApplication in a process
class QtApp(QtGui.QApplication):
"""
docstring
"""
def __init__(self, args, url_queue, filename_queue):
QtGui.QApplication.__init__(self, args)
browser = Browser(url_queue, filename_queue)
self.exec_()
browser_process = multiprocessing.Process(target=QtApp,
args=(sys.argv, url_queue, filename_queue))
Normally, when I perform a network request via Qt 4.8 I don't need to explicitly run QApplication.processEvents() (see this StackOverflow code example).
However when I issue the network request from JavaScript in a QWebView, this will not work unless I call that method until the request is finished, as seen below (syntax-highlighted gist here).
CLARIFICATION:
The request is not even sent when I omit the processEvents() call although the finished slots seem to be attached as I understand it.
from PyQt4 import QtCore, QtGui, QtNetwork, QtWebKit
class MainWindow(QtGui.QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
QtNetwork.QNetworkProxyFactory.setUseSystemConfiguration(True)
self.view = QtWebKit.QWebView(self)
self.setCentralWidget(self.view)
self.view.setPage(QtWebKit.QWebPage(self.view))
self.view.page().mainFrame().javaScriptWindowObjectCleared.connect(self.refreshJS)
self.view.setHtml(
'''<html>
<body>
LOADING...
<script>
<!--
APP.request();
//-->
</script>
</body>
</html>'''
)
#QtCore.pyqtSlot()
def request(self):
request = QtNetwork.QNetworkRequest(QtCore.QUrl('http://localhost/test.php'))
manager = QtNetwork.QNetworkAccessManager()
manager.finished.connect(self.managerFinished)
reply = manager.post(request, b'a=A')
reply.finished.connect(self.finished)
############################################################
### FIXME: Request never even *sent* if this is missing ###
############################################################
while not reply.isFinished():
QtGui.QApplication.processEvents()
##########################################################
print('request FINISHED? '+str(reply.isFinished())+', ERROR '+str(reply.error()))
def finished(self):
print('finished')
def managerFinished(self):
print('managerFinished')
def refreshJS(self):
print('refreshJS')
self.view.page().mainFrame().addToJavaScriptWindowObject('APP', self)
if __name__ == '__main__':
import os, sys
app = QtGui.QApplication(sys.argv)
MainWindow().show()
sys.exit(app.exec_())
There is one big difference between your usage of QNetworkAccessManager and one pointed in the link. Notice that your version is synchronous: you block the event loop in 'request' slot by waiting for reply, thus QNetworkAccessManager cannot work unless you push the events by hand. From the other side the pointed example is asynchronous: it creates request, connects 'finished' signal and lets event loop to work. This way event loop is not blocked and network manager works well. I understand that in your situation 'request' slot needs to be synchronous, thus processEvents is a must here.
EDIT
If you want to make the request asynchronous, then you need to be sure that QNetworkAccessManager object is persistent outside the request method. Initialize it in __init__(self) and use it to POST in the request. ProbablyQNetworkReply` needs to be persistent too, you need to check it.
I'm new to Python, I want open a web page(here is google.com) using pywebkitgtk
then countdown with another thread,
when time's up, send a signal to webview, download the html as file
Is there a way to open a web-page in gtk.main and countdown in background thread, then send a signal to GUI, make GUI do something..
reference material:
Downloading a page’s content with python and WebKit
using a separate thread to run code, two approaches for threads in PyGTK.
here is my code, it cannot run, I guess I do not understand Python's Class...
#!/usr/bin/env python
import sys, threading
import gtk, webkit
import time
import gobject
gobject.threads_init()
google = "http://www.google.com"
class WebView(webkit.WebView):
#return page's content
def get_html(self):
self.execute_script('oldtitle=document.title;document.title=document.documentElement.innerHTML;')
html = self.get_main_frame().get_title()
self.execute_script('document.title=oldtitle;')
return html
#wait 5 senconds and send a signal
class TimeSender(gobject.GObject, threading.Thread):
def __init__(self):
self.__gobject_init__()
threading.Thread.__init__(self)
def run(self):
print "sleep 5 seconds"
time.sleep(5)
self.emit("Sender_signal")
gobject.type_register(TimeSender)
gobject.signal_new("Sender_signal", TimeSender, gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, ())
#PywebkitGTK, open google.com, receive signal
class Window(gtk.Window, gobject.GObject):
def __init__(self, time_sender, url):
self.__gobject_init__()
gtk.Window.__init__(self)
time_sender.connect('Sender_signal', self._finished_loading)
self._url = url
def open_page(self):
view = WebView()
view.get_html()
view.open(self._url)
self.add(view)
gtk.main()
#write html to file
def _finished_loading(self, view):
with open("pagehtml.html", 'w') as f:
f.write(view.get_html())
gtk.main_quit()
'''
def user_callback(object):
with open("pagehtml2.html", 'w') as f:
f.write(view.get_html())
gtk.main_quit()
'''
if __name__ == '__main__':
time_sender = TimeSender()
window = Window(time_sender, google)
#time_sender.connect("Sender_signal", user_callback)
time_sender.start()
window.open_page()
I got an error:
AttributeError: 'TimeSender' object has no attribute 'get_html'
I've been confused for a few days... thanks
Looks like you are confused about singals/objects and threads. _finished_loading method does not get view as a parameter as yo are not passing it. If you make it global it should work. Following piece of code works as expected.
#!/usr/bin/env python
import sys, threading
import gtk, webkit
import time
import gobject
gobject.threads_init()
google = "http://www.google.com"
class WebView(webkit.WebView):
#return page's content
def get_html(self):
self.execute_script('oldtitle=document.title;document.title=document.documentElement.innerHTML;')
html = self.get_main_frame().get_title()
self.execute_script('document.title=oldtitle;')
return html
#wait 5 senconds and send a signal
class TimeSender(gobject.GObject, threading.Thread):
def __init__(self):
self.__gobject_init__()
threading.Thread.__init__(self)
def myEmit(self):
window.emit("Sender_signal")
def run(self):
print "sleep 5 seconds"
time.sleep(5)
gobject.idle_add(self.myEmit)
gobject.type_register(TimeSender)
#PywebkitGTK, open google.com, receive signal
class Window(gtk.Window, gobject.GObject):
def __init__(self, time_sender, url):
self.__gobject_init__()
gtk.Window.__init__(self)
self.connect('Sender_signal', self._finished_loading)
self._url = url
def open_page(self):
self.view = WebView()
self.view.get_html()
self.view.open(self._url)
self.add(self.view)
gtk.main()
#write html to file
def _finished_loading(self, view1):
with open("pagehtml.html", 'w') as f:
f.write(self.view.get_html())
gtk.main_quit()
'''
def user_callback(object):
with open("pagehtml2.html", 'w') as f:
f.write(view.get_html())
gtk.main_quit()
'''
if __name__ == '__main__':
gobject.signal_new("Sender_signal", Window, gobject.SIGNAL_RUN_FIRST, gobject.TYPE_NONE, ())
time_sender = TimeSender()
window = Window(time_sender, google)
#time_sender.connect("Sender_signal", user_callback)
time_sender.start()
window.open_page()