I am creating an application which automates loading webpages and creating screenshots (no, I cannot use one of the existing solutions) using PySide. One part of the app gets a list of URLs and loads each URL in turn using a new QWebPage object. After the page is loaded a screenshot is taken and the QWebPage object is deleted.
Every now and then, given enough runs, I get the following error from PySide, as a RuntimeError exception:
Invalid Signal signature: loadStarted()
Failed to connect signal loadStarted().
The first line is printed to STDERR (probably by Qt?) and the second line is the Python exception.
loadStarted() is a built-in QWebPage signal, not something I created. This works 90% of the time, and I could not figure out what makes it fail occasionally.
To be honest, this app is quite unusual in design, as it hooks PySide/Qt into a uWSGI served web app - this means that for example I am not using the QApplication event loop but rather a local event loop for each page load. I am also not experienced in either Qt or Python so it's possible I'm making a lot of mistakes, but I can't figure out what those are.
I am thinking this post might have something to do with it, but I'm not sure.
Any suggestions on where to look next?
UPDATE: the signal is connected through the following code:
class MyWebPage(QWebPage):
def __init__(self, parent=None):
super(MyWebPage, self).__init__(parent)
self.loadStarted.connect(self.started)
self.loadFinished[bool].connect(self.finished)
MyWebPage objects are created as children of a different single QObject instance which is not deleted until the process shuts down. They are deleted by calling page.deleteLater() once I am done with them. Since I am running a local event loop, I trigger deferred deletions to happen after exiting local event loop by calling:
# self.eventLoop is the local event loop, which at this stage is not running
self.eventLoop.processEvents()
# self.app is the QApplication instance
self.app.sendPostedEvents(None, QEvent.DeferredDelete)
I was having the same problem (I'd get these errors every once in a while, but I couldn't consistently reproduce it). I think you may be right that it has something to do with methods not existing when you try to connect the signals to them - just to test that, I put the .connect calls in a separate method, and the errors went away. For example:
EDIT:
(a few hours later) I guess I spoke too soon - I just got the error again.
UPDATE:
(a few weeks later)
I played around with the syntax a lot and occasionally even got a RuntimeError (possibly this bug in PySide?). I'm still not completely sure why, but as the error happens inconsistently, you're probably safe in forcing it like this:
class MyWebPage(QWebPage):
def __init__(self, parent=None):
super(MyWebPage, self).__init__(parent)
success = False
while not success:
try:
success = self.loadStarted.connect(self.started)
except RuntimeError:
success = False
success = False
while not success:
try:
success = self.loadFinished[bool].connect(self.finished)
except RuntimeError:
success = False
If you really want to be safe, you could maybe keep a loop counter and just crash the program if the signal doesn't connect correctly before some threshold.
For what its worth, this and other issues were finally solved in a decent way after I upgraded to PySide 1.2.1.
Related
After reading A LOT of data on the subject I still couldn't find any actual solution to my problem (there might not be any).
My problem is as following:
In my project I have multiple drivers working with various hardware's (IO managers, programmable loads, power supplies and more).
Initializing connection to these hardware's is costly (in time), and I cant open and then close the connection for every communication iteration between us.
Meaning I cant do this (Assuming programmable load implements enter / exit):
start of code...
with programmable_load(args) as program_instance:
programmable_load_instance.do_something()
rest of code...
So I went for a different solution :
class programmable_load():
def __init__(self):
self.handler = handler_creator()
def close_connection(self):
self.handler.close_connection()
self.handler = None
def __del__(self):
if (self.handler != None):
self.close_connection()
For obvious reasons I dont 'trust' the destructor to actually get called so I explicitly call close_connection() when I want to end my program (for all drivers).
The problem happens when I abruptly terminate the process, for example when I run via debug mode and quit debugging.
In these cases the process terminates without running through any destructors.
I understand that the OS will clear all memory unused at this point, but is there any way to clear the memory in an organized manner?
and if not, is there a way to make the quit debugging function pass through a certain set of functions? Does the python process know it got a quite debugging event or does it treat it as a normal termination?
Operating system: Windows
According to this documentation:
If a process is terminated by TerminateProcess, all threads of the
process are terminated immediately with no chance to run additional
code.
(Emphasis mine.) This implies that there is nothing you can do in this case.
As detailed here, signals don't work very well on ms-windows.
As was mentioned in a comment, you could use atexit to do the cleanup. But that only works if the process is asked to close (e.g. QUIT signal on Linux) and not just killed (as is likely the case when stopping the debugging session). Similarily if you force your computer to turn off (e.g. long press power button or remove power) then it won't be called either. There is no 'solution' to that for obvious reasons. Your program can't expect to be called when the power suddenly goes off or when it is forcefully killed. The point of forcefully killing is to definitely kill the process now. If it first called your clean-up code then you could delay that which defeats the purpose. That is why there are signals such as to ask your process to stop. This is not Python specific. The same concept also applies across operating systems.
Bonus (design suggestion, not a solution): I would argue that you can still make use of the context manager (using with). Your problem is not unique. Database connections are usually kept alive for longer as well. It is a question of the scope. Move the context further up to the application level. Then it is clear what the boundary is and you don't need any magic (you are probably also aware of #contextmanager to make that a breeze).
I haven't tested properly as I don't have wingide installed over here so I can't grant you this will work but what about using setconsolectrlhandler? For instance, try something like this:
import os
import sys
import win32api
if __name__ == "__main__":
def callback(sig, func=None):
print("Exit handler called!")
try:
win32api.SetConsoleCtrlHandler(callback, True)
except Exception as e:
print("Captured exception", e)
sys.exit(1)
print("Press to quit")
input()
print("Bye!")
It'll be able to handle CTRL+C and CTRL+BREAK signals:
I've been fighting for three hours now to get this process multithreaded, so that I can display a progress box. I finally got it working, insomuch as the process completes as expected, and all the functions call, including the ones to update the progress indicator on the window.
However, the window never actually displays. This is a PyGObject interface designed in Glade. I am not having fun.
def runCompile(obj):
compileWindow = builder.get_object("compilingWindow")
compileWindow.show_all()
pool = ThreadPool(processes=1)
async_result = pool.apply_async(compileStrings, ())
output = async_result.get()
#output = compileStrings() #THIS IS OLD
compileWindow.hide()
return output
As I mentioned, everything works well, except for the fact that the window doesn't appear. Even if I eliminate the compileWindow.hide() command, the window never shows until the process is done. In fact, the whole stupid program freezes until the process is done.
I'm at the end of my rope. Help?
(By the way, the "recommended" processes of using generators doesn't work, as I HAVE to have a return from the "long process".)
I'm not a pyGobject expert and i don't really understand your code. I think that you should post more. Why are you calling the builder in a function? you can call it at the init of the GUI?
Anyways.. It seems that you are having the common multithread problems..
are you using at the startup GObject.threads_init() and Gdk.threads_init() ?
Then, if you want to show a window from a thread you need to use Gdk.threads_enter() and Gdk.threads_leave().
here is an useful doc
I changed the overall flow of my project, so that may affect it. However, it is imperative that Gtk be given a chance to go through its own main loop, by way of...
if Gtk.events_pending():
Gtk.main_iteration()
In this instance, I only want to call it once, to ensure the program doesn't hang.
(The entire program source code can be found on SourceForge. The function in question is on line 372 as of this posting, in function compileModel().
I am attempting to convert code from Summerfield's article on (old-style) PyQt Signals/Slots to new-style PySide code. One example is a pure console application, which I have never worked with before. Unfortunately, when I try to run it multiple times, I am told that the previous application is still running.
It is a simple app: it basically lets you set a number, and reports back if the number is new:
from PySide import QtCore
import sys
class TaxRate(QtCore.QObject):
rateChangedSig=QtCore.Signal(float)
def __init__(self):
QtCore.QObject.__init__(self)
self.rate = 17.5
def getRate(self):
return self.rate
def setRate(self, newRate):
if newRate != self.rate:
self.rate = newRate
self.rateChangedSig.emit(self.rate) #was self.emit(SIGNAL("rateChanged"), self.rate)
#QtCore.Slot() #technically not really needed
def rateChangedSlot(value):
print("Tax rate changed to {0:.2f} %".format(value))
if __name__=="__main__":
qtApp = QtCore.QCoreApplication(sys.argv) #origional had QtGui.QApplication, but there is no GUI
vat = TaxRate()
vat.rateChangedSig.connect(rateChangedSlot) #was vat.connect(vat, SIGNAL("rateChanged"), rateChanged)
vat.setRate(8.5) # A change will occur (new rate is different)
qtApp.quit()
sys.exit(qtApp.exec_())
Overall, it works as expected, except the final two lines do not kill the process. When I try to run the program twice, the second time my IDE (Spyder) always tells me that it is already running in a separate process. If I try running it from the command line, the window just hangs.
Strangely, when I comment out the last two lines I do not get this warning. This is the opposite of what I expect (based on previous experience with PySide GUI applications and the documentation for quit()).
Following the Closing a window example at Zetcode, I tried replacing qtApp.quit() with qtApp.instance().quit(), which yielded the same non-killing result.
So, how do I kill this thing?
One idea is that I shouldn't have even started it in the first place (as suggested here). Even though it is a pure console app, Summerfield's original program initializes with app=QtGui.QApplication(sys.argv), and it does not contain the last two lines. Things run fine, multiple times. However, isn't there a concern that each run would create a new process, so his program seems to be effectively multiplying processes without warning? (Note in practice I don't think this is happening on my system, so the answer seems to be 'No' for reasons I don't understand).
What is the correct way to control/initialize/kill a console app using PySide?
(This is ignoring the question, for the time being, why one would ever use PySide for a pure console application in Python as has been pointed out previously. But if anyone were to be interested in answering that separate question, I could start a separate question for it).
Potentially relevant post:
Pyside applications not closing properly
The problem is because you call QCoreApplication.quit() before you call QCoreApplication.exec_(). The call to quit is not queued up in the event loop, it happens immediately. The call to QCoreApplication.exec_() starts the event loop which only ends when a call to QCoreApplication.exit() (or QCoreApplication.quit()) is made while the event loop is running.
This is somewhat explained in the Qt documentation of QCoreApplication but it is very easy to miss.
I imagine you don't really need to call exec_() as you aren't using any events in your current code (most events are to do with window/mouse/keyboard though you might conceivably use some in the future like those generated by QTimer). It really depends what you want to do with the program in the future.
If you don't call exec_(), then your script will exit as you would normally expect any Python script to do (the only blocking function in your code is the call to exec_(), remove that and nothing keeps it running.)
I'm doing something similar to this question but have a more subtle problem.
I have an API client class that makes HTTP requests; I store the QNetworkReply in the object, so as to access its data from the slot connected to the "finished" signal. On the next request, this is replaced by the next QNetworkReply, so Python should be able to free the previous request object and thus the underlying network resources. Instead, the old reply objects seem to get stuck somewhere, causing a memory leak, and if the app runs long enough, a delay on quit, presumably because all the requests ever issued are finally being deleted.
Simplified but complete example:
import sys, signal
from PySide import QtCore, QtNetwork
class Poller(QtCore.QObject):
url = QtCore.QUrl("http://localhost:5000/")
def __init__(self, parent=None):
super(Poller,self).__init__(parent)
self.manager = QtNetwork.QNetworkAccessManager()
def start(self):
request = QtNetwork.QNetworkRequest(self.url)
self.reply = self.manager.get(request)
self.reply.finished.connect(self.readReply)
self.reply.error.connect(self.error)
def readReply(self):
text = self.reply.readAll()
self.reply.close() # doesn't help
self.reply.deleteLater() # doesn't help
QtCore.QTimer.singleShot(10, self.start)
#QtCore.Slot(QtNetwork.QNetworkReply.NetworkError)
def error(self, err):
self.reply.finished.disconnect()
sys.stderr.write('Error: ' + str(err) + '\n')
QtCore.QTimer.singleShot(10, self.start)
signal.signal(signal.SIGINT, signal.SIG_DFL)
app = QtCore.QCoreApplication(sys.argv)
p = Poller()
p.start()
app.exec_()
The http server being polled doesn't much matter; for this test I'm using the Flask hello world. In fact, since this doesn't do connection keepalive, if I check "netstat" I see a steadily increasing number of zombie TCP connections in a TIME_WAIT state, and eventually start getting PySide.QtNetwork.QNetworkReply.NetworkError.UnknownNetworkError once 30,000+ ports have been used up; further evidence that the QNetworkReply is not being properly freed.
Same problem happens with PySide or PyQt4. What am I doing wrong, or could this be a bug?
First, turns out it's normal for TCP connections to stick around for a while in TIME_WAIT after closing. I was only hitting the limit because I had set the singleShot timer to 0ms for testing.
I rewrote the example in C++. deleteLater worked as expected, and I could reproduce both the memory growth and slow quit by omitting it. (Since the memory is managed by Qt, all the reply objects had to be deleted by the QNetworkAccessManager destructor).
Interestingly, on closer examination, the slow quit does not happen in Python when deleteLater is used, but the memory growth does. So I guess the C++ object is getting deleted but there are still resources being used somewhere. This is still mysterious to me.
The fix, though, is to call setParent(None) on the QNetworkReply. This can be done at any time, even when it is first returned from the QNetworkAccessManager. It is initially parented to the manager; changing the parent to null means Qt isn't responsible for memory management, and it will be properly handled by Python, even without using deleteLater.
(I found this hint somewhere online; unfortunately I can't find it now, or I would link it.)
Edit: I thought this worked in my testing, but my app is still leaking memory.
Edit 2: I have no leak with PyQt4 on Python 2, but I do with PySide, and with both on Python 3.
Is there a crash reporting framework that can be used for pure Python Tkinter applications? Ideally, it should work cross-platform.
Practically speaking, this is more of 'exception reporting' since the Python interpreter itself hardly crashes.
Here's a sample crash reporter:
Rather than polluting your code with try..except everywhere, you should just implement your own except hook by setting sys.excepthook. Here is an example:
import sys
import traceback
def install_excepthook():
def my_excepthook(exctype, value, tb):
s = ''.join(traceback.format_exception(exctype, value, tb))
dialog = ErrorReportDialog(None, s)
dialog.exec_()
sys.excepthook = my_excepthook
Call install_exception() when your application starts.
ErrorReportDialog is a Qt dialog I've made. traceback.format_exception() will format argument passed to the except hook in the same way it does in Python's interpreter.
EDIT: I forgot to mention a little gotcha with that. It doesn't work with threads (well, at least it didn't last time I checked). For code running in another thread, you will need to wrap it in a try..except block.
Stick try excepts everywhere your application can crash (I/O, networking etc.). Whenever an except is called, call a function that will kill the old window, spawn a new tkinter notification window, or a custom one with your error message.
Do a root.after to the new window and send your error report (urllib).
Put a restart button if you wish.
There is no crash reporting framework - as tkinter is not that type of GUI. It's pretty much a wrapper for simple command line apps.
Go pyqt/gtk or wxpython if you want the features seen in the screen-shot above. But I'm pretty sure that where ever you go, you'll have to write your own reporter.