I have some simple code that loads google.com with the PyQt4 library.
This is the code:
import sys
from PyQt4.QtGui import QApplication
from PyQt4.QtCore import QUrl
from PyQt4.QtWebKit import QWebView
class Browser(QWebView):
def __init__(self):
QWebView.__init__(self)
self.loadFinished.connect(self._result_available)
def _result_available(self, ok):
frame = self.page().mainFrame()
#print(frame.toHtml())
app.exit()
if __name__ == '__main__':
app = QApplication(sys.argv)
view = Browser()
view.load(QUrl('http://www.google.com'))
print('start')
app.exec_()#hangs the main thread
print('end')
My problem with this code is that app.exec_() hangs the main thread for a while, between the print start and print end.
Is there a way to scrape a website in PyQt4 without making the main thread hang for a while.
I would like to resume the main thread's normal execution of code after app.exec_().
It's probably just taking time to download the content from the web. app.exec_() is what runs your application basically. So if anything is indeed hanging it's actually some other factor that influences the proper execution of your application. Any statements that are after the app.exec_() will be executed once you close the application. Usually app.exec_() is called at the end of the main (in C++ you do return app.exec_() which you can also do in PyQt of course).
When working with content that requires time to be downloaded and displayed in the UI, you have to add multithreading in order to allow the main thread to continue work properly (thus not creating the so called UI freeze) but at the same time do some work in the background. The sole purpose of the main thread is to keep track of the UI and run it. Anything else that you add to the main thread will prevent the UI from working in a fluent manner. You have several options here - inheriting QThread (I wouldn't suggest doing that for this scenario in particular), using QThread + QObject in order to incorporate slots and signal in your application (for example: page downloaded? -> if yes, signal the UI to display the content), QRunnable etc.
So my advice is to download the content in a separate thread and once it's done you can add it to your UI as well as provide some other form of visual feedback (this also includes print statements :P).
Related
I am working on a scientific algorithm (image processing), which is written in C++, and uses lots of parallelization, handled by OpenMP. I need it to be callable from Python, so I created a CPython package, which handles the wrapping of the algorithm.
Now I need some UI, as user interaction is essential for initializing some stuff. My problem is that the UI freezes when I run the algorithm. I start the algorithm in a separate thread, so this shouldn't be a problem (I even proved it by replacing the function call with time.sleep, and it works fine, not causing any freeze). For testing I reduced the UI to two buttons: one for starting the algorithm, and another just to print some random string to console (to check UI interactions).
I also experienced something really weird. If I started moving the mouse, then pressed the button to start the computation, and after that kept moving the mouse continuously, the UI did not freeze, so hovering over the buttons gave them the usual blueish Windows-style tint. But if I stopped moving my mouse for a several seconds over the application window, clicked a button, or swapped to another window, the UI froze again. It's even more strange that the UI stayed active if I rested my mouse outside of the application window.Here's my code (unfortunately I cannot share the algorithm for several reasons, but I hope I manage to get some help even like this):
if __name__ == "__main__":
from PyQt5.QtWidgets import QApplication, QPushButton, QVBoxLayout, QWidget
from PyQt5.QtCore import QThread, QObject, pyqtSignal
import time
from CustomAlgorithm import Estimator # my custom Python package implemented in C++
class Worker(QObject):
finished = pyqtSignal()
def run(self):
estimator = Estimator()
estimator.calculate()
# if the above two lines are commented, and the next line is uncommented,
# everything's fine
# time.sleep(5)
print("done")
app = QApplication([])
thread = QThread()
window = QWidget()
layout = QVBoxLayout()
# button to start the calculation
btn = QPushButton("start")
layout.addWidget(btn)
btn.clicked.connect(thread.start)
# button to print some text to console
btn2 = QPushButton("other button")
layout.addWidget(btn2)
btn2.clicked.connect(lambda: print("other button clicked"))
window.setLayout(layout)
# handling events
worker = Worker(app)
worker.moveToThread(thread)
thread.started.connect(worker.run)
worker.finished.connect(thread.quit)
worker.finished.connect(worker.deleteLater)
thread.finished.connect(thread.deleteLater)
window.show()
app.exec_()
I tried multiple variants of using threads, like threading.Thread, multiprocessing.Process, PyQt5.QtCore.QThread (as seen above), even napari's worker implementation, but the result was the same. I even tried removing omp from the code, just in case it interferes somehow with python threads, but it didn't help.
As for the reason I use python, is that the final goal is to make my implementation available in napari.
Any help would be highly appreciated!
Because of Python's "Global Interpreter Lock", only one thread can run Python code at a time. However, other threads can do I/O at the same time.
If you want to allow other threads to run (just like I/O does) you can surround your code with these macros:
Py_BEGIN_ALLOW_THREADS
// computation goes here
Py_END_ALLOW_THREADS
Other Python threads will be allowed to run while the computation is happening. You can't access anything from Python between these two lines - so get that data in order before Py_BEGIN_ALLOW_THREADS.
Reference
I have a code which looks like:
app = QApplication(sys.argv)
self.interface = Interface()
# The figure
self.fig = self.interface.fig
self.ax = self.fig.add_subplot(111)
self.interface.show()
app.exec_()
print 'this is not printed'
The problem is that once app.exec_() is executed, nothing is until I close the window that pops up.
How can I continue running the code?
That is intended. What you have to do is use signals/slots, code inside your Qt classes, or spawn off threads before you call app.exec().
Signals and slots are your defacto way of interacting with Qt. Basically a signal is any "event" or custom "event" and slots can be thought of as "event handlers". For instance when someone hits a button on a GUI it creates a signal that seeks out any handler that is connected to it. You can connect none, one, or many slots to each signal (you can even connect the same one multiple times)! Here is a good reference for this in python.
Coding inside your Qt classes usually means creating slots that do useful work for you. Remember you don't want to hold up the event loop too long so spawn a new thread if you do this.
The third option available to you is to spin off other threads. Be careful interacting with Qt from threads, if you do you MUST us signals and slots. Implement threading as suggested in this SO.
app.exec_() does not lock anything, it runs a GUI event loop that waits for user actions (events) and dispatches them to the right widget for handling. It does this until there are no top level windows left open; if you leave at least one top level window of your app open, then exec() never returns, it can't (your app will be terminated during system shutdown). When no more top level windows the app cleans up and returns from exec(). At that point the GUI is no longer in the event loop.
Whatever it is you want to do after exec() it is likely you would either put it in a QThread or in a signal handler (which you would connect, for example, to a "Go!" button; you would connect a "Cancel" button to a handler that closes the app window).
You can put code after exec() but it would be rather unconventional: if anything goes wrong it is unlikely that user can see the problem since the GUI is no longer visible, a GUI app doesn't usually have a console terminal open where the error might be seen, there will not typically be a console for a GUI app (ie you will run the app via pythonw.exe instead of python.exe), or you have to open a new window and exec() again in order to show an error message and wait till user clicks ok, destroy message window in ok handler so app.exec() once again returns.
In addition to the previous answer, not every time when all windows are closed does the GUI event loop, run by app.exec_(), stop. If you want to terminate it manually you can use app.quit() inside any of the event handlers. It stops GUI event loop and launches your code after app.exec_().
First answer is a lot of words about nothing.
A dirty trick to remove blocking would be to abuse mathplot lib:
import matplotlib
import matplotlib as mpl
import matplotlib.cbook as cbook
import matplotlib.collections as mcol
import matplotlib.patches as mpatches
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import matplotlib.cm as cm # colors
from mpl_toolkits.mplot3d import Axes3D # imports options for 3-D plotting
from matplotlib.backends.backend_pdf import PdfPages
from PyQt5 import QtWidgets, uic
# ==============================================================
if not QtWidgets.QApplication.instance():
app = QtWidgets.QApplication(sys.argv)
else:
app = QtWidgets.QApplication.instance()
#endelse
MainWindow = QtWidgets.QMainWindow()
MainWindow=uic.loadUi('E:\Python\Python_lib\MyGui.ui',MainWindow)
MainWindow.show()
# There are two ways to start the gui, and i.e. it looping
# The correct way if to use app.exec_(), which
# blocks the command line :
#d=app.exec_()
# However, this will block the command line making
# We can misuse mathplot lib creating a Figure instead
# which seems to start the Gui loop without blocking the command line:
ImageFig = Figure((3,4))
ImagePanel = FigureCanvas(ImageFig)
ImagePanel.draw()
I was attempting to implement the code from this question, but I have discovered that QTimer does not fire under certain circumstances.
The simplified code:
app = QApplication([])
view = QWebEngineView()
view.show()
def callback():
sys.exit(0)
QTimer.singleShot(1000, callback)
app.exec()
The application runs forever with this code (ie. callback is never executed). However, if you comment out view.show() then it exits after one second as it should.
As far as I can tell, this problem is particular to QWebEngineView; if we replace the QWebEngineView with a generic QWidget or QLabel, then it works as expected and exits after one second.
What is the cause of this, and how can I fix it?
As prompted by #eyllanesc, I have discovered that this is a bug introduced by the 5.14.0 release of the PyQtWebEngine module; the previous version (5.13.2) does not show this behavior.
I am experiencing a weird behavior from this simple line of code, which I wanted to use outside a pyqt GUI class
from PyQt4 import QtGui
FilePath=QtGui.QFileDialog.getOpenFileName(None,'choose the file')
When I first implemented it, it worked (probably because I run some other code before that I cannot trace back).
Then after restarting python, it stopped working and it crashes saying:
QWidget: Must construct a QApplication before a QPaintDevice
The same exact code works fine when implemented in a GUI, where the first argument is of course self.
But I would like to use the same filedialog in an external function I use for a different purpose, outside a GUI context.
Is it possible and what could be a way/workaround to implement that ? I would like to avoid using wx, easygui or tk
In order to use the Qt UI elements (such as QFileDialog), you generally have to have a QApplication running. It seems a little unorthodox, but you can get your snippet to run just by instantiating a QApplication prior to your QFileDialog, like so
from PyQt4 import QtGui
app = QtGui.QApplication([])
FilePath=QtGui.QFileDialog.getOpenFileName(None,'choose the file')
Technically, this will work, though I'm not quite sure I can endorse doing this.
Qt applications are built around an event loop and require QApplication instance. Dialogs have their own event loop, so if all you want is a wizard (a series of dialogs that open/close in sequence), then in principle all you need is the QApplication instance. The following works in PyQt 5:
from PyQt5 import Qt
app = Qt.QApplication([])
FilePath=Qt.QFileDialog.getOpenFileName(None,'choose the file')
print(FilePath)
However, if you have other code that assumes an application event loop, all bets are off. In that case, you could do the following:
from PyQt4 import Qt
app = Qt.QApplication([])
FilePath=Qt.QFileDialog.getOpenFileName(None,'choose the file')
print(FilePath)
...create widgets....
QTimer.singleShot(someWidgetMethod)
app.exec_()
The single-shot timer will fire only during the app.exec_(), thus calling some of your code as part of the event loop.
All that being said,it is much better to create a main window and connect slots to signals and call app.exec_(). There are many examples included with PyQt, check those.
I am trying to make an application in Python using PyQt that can fetch the generated content of a list of URLs and process the fetched source with the help of multiple threads. I need to run about ten QWebViews at once. As ridiculous as that might sound, when it comes to hundreds of URLs, using threaded QWebViews gets the results over 3 times faster than normal.
Here is the test code that I have been having problems with...
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQt4.QtWebKit import *
class Worker(QThread):
def __init__(self, url, frame):
QThread.__init__(self)
self.url = url
self.frame = frame
def run(self):
self.frame.load(QUrl(self.url))
print len(self.frame.page().mainFrame().toHtml())
app = QApplication(sys.argv)
webFrame = QWebView()
workerList = []
for x in range(1):
worker = Worker('http://www.google.com', webFrame)
workerList.append(worker)
for worker in workerList:
worker.start()
sys.exit(app.exec_())
Above, I tried initializing the QWebView in the main QApplication only to get:
QObject: Cannot create children for a parent that is in a different thread.
So then I tried initializing the QWebView in the QThread; but then, the QWebView remained unchanged and blank without outputting any errors or anything. This was probably due to a cache error.
I have the feeling that I am missing out on something or skipping a very important step. Since threaded QWebViews in PyQt isn't a really documented topic, I would really appreciate any help on how to successfully implement this.
There are multiple issues with your question and code:
You are talking about QWebFrame, but are actually passing a QWebView to your worker(s). Since this is a QWidget, it belongs to the main (GUI) thread and should not be modified by other threads.
One QWebView / QWebFrame can only load one URL at a time, so you cannot share it across multiple workers.
QWebFrame.load() loads data asynchronously, i.e. a call to load() returns immediately and there will be no data to read yet. You will have to wait for the loadFinished() signal to be emitted before you can access the data.
Since the actual loading is done by the networking layer of the operating system, and the load() method does not block, there is no need to run it in a separate thread in the first place. Why do you claim that this should be faster -- it makes no sense.
Since you want to load hundreds of URLs in parallel (or about 10, you are mentioning both in the same sentence), are you sure that you want to use QWebFrame, which is a presentation class? Do you actually want to render the HTML or are you just interested in the data retrieved?