I have a pywebview, that launches a Qt window utilizing the QtWebEngine with Chromium on Windows.
When starting the webview in debug mode, the following output is of particular interest for me:
DevTools listening on ws://127.0.0.1:8228/devtools/browser/<GUID>
This line is output by the Chromium engine itself and I want to make the debug port and GUID available in my application. For that, my idea is to redirect all terminal outputs to a StringIO stream.
Here is a minimum example of what I have done so far, to redirect all outputs:
from contextlib import redirect_stdout, redirect_stderr
from io import StringIO
import logging
from PyQt5 import QtCore
import webview
def qt_message_handler(mode, context, message):
# Redirect message to stdout
print(message)
if __name__ == '__main__':
# Let the handler redirect all qt messages to stdout
QtCore.qInstallMessageHandler(qt_message_handler)
stream = StringIO()
# Redirect stdout and stderr to stream
with redirect_stdout(stream), redirect_stderr(stream):
# Redirect all loging outputs to stream
stream_handler = logging.StreamHandler(stream)
loggers = [logging.getLogger(name) for name in logging.root.manager.loggerDict]
for logger in loggers:
for handler in logger.handlers:
logger.removeHandler(handler)
logger.addHandler(stream_handler)
# Start the webview window
window = webview.create_window('Webview window')
webview.start(gui='qt', debug=True)
Ths approach redirects all outputs to the stream object as intended except the one line output by Chromium mentioned at the beginning.
So i played around with the QTWEBENGINE_CHROMIUM_FLAGS environment variable. Since pywebview overrides this environment variable, I changed it directly in the pywebview module. However no argument passed to Chromium by this environment variable seems to change the output behaviour of this specific line. Looking at the Chromium source code, this makes sense:
std::string message = base::StringPrintf(
"\nDevTools listening on ws://%s%s\n", ip_address->ToString().c_str(),
browser_guid.c_str());
fprintf(stderr, "%s", message.c_str());
fflush(stderr);
Since the message is directly printed to stderr no Chromium logging flags have an influence. But it's clearly evident, that the message gets printed to stderr, so why does a redirect of stderr in Python have no effect on this message? Is there any way to redirect this message in python?
Bonus question:
A further look into the Chromium source code shows, that the debug port and GUID are also written into a file:
if (!output_directory.empty()) {
base::FilePath path =
output_directory.Append(kDevToolsActivePortFileName);
std::string port_target_string = base::StringPrintf(
"%d\n%s", ip_address->port(), browser_guid.c_str());
if (base::WriteFile(path, port_target_string.c_str(),
static_cast<int>(port_target_string.length())) < 0) {
LOG(ERROR) << "Error writing DevTools active port to file";
}
}
I just can't figure out where Chromium stores this data. Where can this file be located on Windows?
I have made some findings that, while not directly answering the question, do lead to a conclusion:
Why is the stderr of Chromium not redirected? This is because Chromium is launched in a child process by the QtWebEngine. While it would be possible, to get the output of a child process in Python, it would require to mess around with the QtWebEngine. At this point I decided, that this is not worth the effort.
Regarding my bonus question, I found out that the file containting the debug port and GUID should be named DevToolsActivePort. However, this file is not created when using the QtWebEngine and I haven't found out yet, why this is the case. See this question for a possible answer in the future: QtWebEngine: DevToolsActivePort file is missing
How did I solve my issue? I found out, that a query exists which returns the websocket url among other things: http://<debug_server_url>/json/version (see this answer). While you can determine the debug port within the QtWebEngine, I solved it for my pywebview application similar to this:
import requests
from PySide6 import QtCore
import webview
debug_server_url = None
def qt_message_handler(mode, context, message):
# Check if the message is the one containing the debug server url
if message.startswith('Remote debugging server started successfully.'):
# Debug server url is at the end of the message
global debug_server_url
debug_server_url = message.split()[-1]
def on_loaded():
if debug_server_url is not None:
websocket_debugger_url = requests.get(debug_server_url + '/json/version').json()['webSocketDebuggerUrl']
print(websocket_debugger_url)
if __name__ == '__main__':
# Define a custom qt message handler
QtCore.qInstallMessageHandler(qt_message_handler)
window = webview.create_window('Webview window')
window.events.loaded += on_loaded
webview.start(gui='qt', debug=True)
Related
So, I create a small program that uses flask to receive some requests and do a few things over selenium. All bits that deal with selenium are in another file that I tried to run first using a thread, and when it did not worked, a process. I believe that the problem is because I use a while true to keep my selenium working. The selenium part knows what to do because it keep checking the variable that I update from them flask part...
This is pretty much my main class that runs the selenium and them start flask, but it never start flask. It get locked on the .start().
if __name__ == "__main__":
# Logging
log_format = '%(asctime)s [%(filename)s:%(lineno)d] %(message)s'
logging.basicConfig(format=log_format,
level=logging.INFO,
stream=sys.stdout)
# Start Selenium
browser = Process(target=selenium_file.run_stuff())
browser.start()
print('TEST')
# Flask
app.run(debug=True)
Not really sure how I could solve this problem (if it's a problem)...
Exchange browser = Process(target=selenium_file.run_stuff()) with browser = Process(target=selenium_file.run_stuff)
You don't pass the function run_stuff but you already execute it and hence it blocks your program until run_stuff returns.
I have tried, for a lot of time now, to create an executable for a Python project. In this project, I need to use:
PyQt(4) : for my GUI,
PySerial : to communicate with an arduino,
Subprocess : to launch some avr things with a .bat file
In fact, the executable is created, but when I try to start it nothing happens, except my mouse tell me that she is occupied.
So, I tried to figured out from where could be the problem, by writing some basic programs, which condense every functions I need for my project. Everything is functional when I launch this from python (3.5), but doesn't when I execute the file generated by pyinstaller. (The interface.py file is here, in a pastebin.com file, if you want, I thought it's not very relevant : it's only a form with a pushbutton)
from PyQt4 import QtGui
from interface import Ui_Form
import serial
import subprocess
import sys, os
class win(QtGui.QWidget, Ui_Form):
"""docstring for win"""
def __init__(self):
super(win, self).__init__()
self.setupUi(self)
self.ser = serial.Serial("COM3", 9600)
self.pathBat = "cmd.bat"
def on_pushButton_clicked(self):
#if (self.ser.isOpen() and self.serAvr.isOpen()):
if True:
self.ser.write("start".encode())
p = subprocess.call(self.pathBat, creationflags=subprocess.CREATE_NEW_CONSOLE, **self.subprocess_args())
if p == 1:
self.writeLog("Works")
self.ser.write("stop".encode())
#self.writeLog(p.returncode)
def subprocess_args(include_stdout=True):
# The following is true only on Windows.
if hasattr(subprocess, 'STARTUPINFO'):
# On Windows, subprocess calls will pop up a command window by default
# when run from Pyinstaller with the ``--noconsole`` option. Avoid this
# distraction.
si = subprocess.STARTUPINFO()
si.dwFlags |= subprocess.STARTF_USESHOWWINDOW
# Windows doesn't search the path by default. Pass it an environment so
# it will.
env = os.environ
else:
si = None
env = None
ret = {}
# On Windows, running this from the binary produced by Pyinstaller
# with the ``--noconsole`` option requires redirecting everything
# (stdin, stdout, stderr) to avoid an OSError exception
# "[Error 6] the handle is invalid."
ret.update({'stdin': subprocess.PIPE,
'stderr': subprocess.PIPE,
'startupinfo': si,
'env': env })
return ret
app = QtGui.QApplication(sys.argv)
v = win()
v.show()
sys.exit(app.exec_())
I added "cmd.bat" to the data in .spec file for pyinstaller, and the function subprocess_arg is here to avoid problems with subprocess (as mentionned on the documentation here)
Firstly I thought the problem was linked to subprocess, I tried to delete all the references to it, still not working. Same for Serial. Moreover, I tried to debug the executable by setting debug = True in the .spec file, but if I try to execute the file from the console, nothing happend at all, it stays blocked on the first line.
So if anybody can help ! thank you in advance !
Maybe the "frozen" application does not find the "cmd.bat"!? You could test it by replacing it with the absolute path.
Your executable unpacks the "cmd.bat" in a temporary folder accessible in python with sys._MEIPASS. You should find your files with something like os.path.join(sys._MEIPASS, "cmd.bat") !?
In case you need it: getattr(sys, 'frozen', False) indicates whether your code is frozen or not (but only for PyInstaller).
I have a Django app in which the users can create reports at various places throughout the webapp. As a little nifty feature I would like to make a "send as PDF" feature for these reports, and I'm almost there.
What I do is that before the report is returned as a HttpResponse through Django I send the raw HTML content through a small PySide/QT snippet (as seen below).
The problem is that I can't get the QApplication to quit.
I have tried with the standard QCoreApplication.exit() but with no luck. If I try to convert a new report right after the first one, the console says a "QApplication instance already exists".
I am using Django 1.2.5, Python 2.7, QT 4.8 and PySide 1.1 on OS X 10.7.3 (for testing).
Code:
def makepdf(response,filename):
try:
app = QApplication(sys.argv)
except:
app = QCoreApplication.instance()
web = QWebView()
stylelink = "%s%s" % (media_root,'images/css/reportGenerator.css')
web.settings().setUserStyleSheetUrl(QUrl.fromLocalFile(stylelink))
web.setHtml(response)
printer = QPrinter()
printer.setPageSize(QPrinter.A4)
printer.setOutputFormat(QPrinter.PdfFormat)
printer.setOutputFileName(filename)
def convertToPdf():
web.print_(printer)
#webresponse = web.close()
#QObject.disconnect()
#print QCoreApplication.instance().children()[0].interrupt()
#qthread = QCoreApplication.instance().thread()#.cleanup()
#qthread.exit()
QCoreApplication.exit()
QObject.connect(web, SIGNAL("loadFinished(bool)"), convertToPdf())
Code comments:
Currently I have made a try/except clause to be able to keep the code running, by using the current QApplication instance (to avoid the 'instance exists error'), but this just doesn't seem right?. I mean, having a QAppliation running for the duration of my Apache server (which will be the case in production) seems a bit off. Shouldn't it be possible to quit it after the PDF conversion is done?
Other than the QCoreApplication.exit() I have tried to use the sys.exit(app.exec_()) method, as is often seen in examples and snippets. This however just causes an error and makes Python crash and the code works perfectly without this. (well it creates the PDF, but won't quit).
All the lines commented out are the previous attempts I have made to quit the QApplication.
In short: I don't know what it is, but it just won't quit. Can anyone suggest why?
Update: After the latest answer I have edited the code to respond to the input. This is how the final part looks now:
def convertToPdf():
web.print_(printer)
app.exit()
web.loadFinished.connect(convertToPdf)
app.exec_()
I do, however, still get an error:
2012-04-30 00:16:10.791 Python[21241:1803] * Assertion failure in +[NSUndoManager _endTopLevelGroupings], /SourceCache/Foundation/Foundation-833.24/Misc.subproj/NSUndoManager.m:324
Qt has caught an exception thrown from an event handler. Throwing exceptions from an event handler is not supported in Qt. You must reimplement QApplication::notify() and catch all exceptions there.
This error only occurs when I implement app.exec_(). However without app.exec_() it is just the normal issue again, with no quitting.
Any ideas?
Update 2: this is the newest code, fixed in accordance with matas latest suggestion:
app = QApplication(sys.argv)
web = QWebView()
printer = QPrinter()
printer.setPageSize(QPrinter.A4)
printer.setOutputFormat(QPrinter.PdfFormat)
printer.setOutputFileName(filename)
def convertToPdf():
web.print_(printer)
app.exit()
web.loadFinished.connect(convertToPdf)
web.setHtml(response)
I still have the same problem however.
what i can say is this:
QObject.connect(web, SIGNAL("loadFinished(bool)"), convertToPdf())
here you call convertToPdf(), if you want to connect the signal, omit the parenthesis!
you could also use this much clearer syntax:
web.loadFinished.connect(convertToPdf)
you may also want to add a parameter to convertToPdf, as it is called with a boolean indicating wheather loading was successful or not.
And using app.exit() should be enough.
oh, and when you use Gui-Components you need to use a QApplication. A QCoreApplication won't do!
edit: it's important to call web.setHtml after you've connected the loadFinished signal! otherwise if the loading already is finished, your function will never be executed!
edit: this works without any problem for me:
#!/usr/bin/env python
from PySide.QtGui import *
from PySide.QtWebKit import *
import sys
from subprocess import Popen, PIPE
def createPdf(html, filename):
app = QApplication(sys.argv)
web = QWebView()
printer = QPrinter()
printer.setPageSize(QPrinter.A4)
printer.setOutputFormat(QPrinter.PdfFormat)
printer.setOutputFileName(filename)
def convertToPdf():
web.print_(printer)
app.exit()
app.deleteLater()
web.loadFinished.connect(convertToPdf)
web.setHtml(html)
app.processEvents()
def createPdfInSubprocess(html, filename):
p = Popen(["python", __file__, filename],
stdin=PIPE, stdout=PIPE, stderr=PIPE)
out, err = p.communicate(html)
return (p.returncode, out, err)
if __name__ == '__main__':
if len(sys.argv) > 1:
# read html from stdin, filename from cmdline
html = "\n".join(sys.stdin.readlines())
createPdf(html, sys.argv[1])
# print("done")
else:
# test if it's working
ret = createPdfInSubprocess(
"<html><h1>test</h1>it's working...</html>", "test.pdf")
print(ret)
Even without all the calls to app.exit(), app.deleteLater(), app.processEvents(), it still works... but can't hurt to have that.
One more important thing: QApplications have to be created from the main thread! So if it's running inside a django app it's probably not going to work, that's why i added the subprocess stuff...
PySide is designed such that there is only ever one instance of QCoreApplication within a process. The best reference to this (possibly undocumented fact) that I could find was http://bugs.pyside.org/show_bug.cgi?id=855.
Basically, there may be dangling references to the qapplication that prevent it from being reaped by the garbage collector, so even if you tell the application to exit and you delete your reference, there may still be other references lying around.
I've developed a Python application that runs both in the GUI mode and the console mode. If any arguments are specified, it runs in a console mode else it runs in the GUI mode.
I've managed to freeze this using cx_Freeze. I had some problems hiding the black console window that would pop up with wxPython and so I modified my setup.py script like this:
import sys
from cx_Freeze import setup, Executable
base = None
if sys.platform == "win32":
base = "Win32GUI"
setup(
name = "simple_PyQt4",
version = "0.1",
description = "Sample cx_Freeze PyQt4 script",
executables = [Executable("PyQt4app.py", base = base)])
This works fine but now when I try to open up my console and run the executable from there, it doesn't output anything. I don't get any errors or messages so it seems that cx_Feeze is redirecting the stdout somewhere else.
Is is possible to get it to work with both mode? Nothing similar to this seems to be documented anywhere. :(
Thanks in advance.
Mridang
I found this bit on this page:
Tip for the console-less version: If
you try to print anything, you will
get a nasty error window, because
stdout and stderr do not exist (and
the cx_freeze Win32gui.exe stub will
display an error Window). This is a
pain when you want your program to be
able to run in GUI mode and
command-line mode. To safely disable
console output, do as follows at the
beginning of your program:
try:
sys.stdout.write("\n")
sys.stdout.flush()
except IOError:
class dummyStream:
''' dummyStream behaves like a stream but does nothing. '''
def __init__(self): pass
def write(self,data): pass
def read(self,data): pass
def flush(self): pass
def close(self): pass
# and now redirect all default streams to this dummyStream:
sys.stdout = dummyStream()
sys.stderr = dummyStream()
sys.stdin = dummyStream()
sys.__stdout__ = dummyStream()
sys.__stderr__ = dummyStream()
sys.__stdin__ = dummyStream()
This way, if the program starts in
console-less mode, it will work even
if the code contains print statements.
And if run in command-line mode, it
will print out as usual. (This is
basically what I did in webGobbler,
too.)
Raymond Chen has written about this. In short, it's not possible directly under Windows but there are some workarounds.
I'd suggest shipping two executables - a CLI and GUI one.
I am working on a daemon where I need to embed a HTTP server. I am attempting to do it with BaseHTTPServer, which when I run it in the foreground, it works fine, but when I try and fork the daemon into the background, it stops working. My main application continues to work, but BaseHTTPServer does not.
I believe this has something to do with the fact that BaseHTTPServer sends log data to STDOUT and STDERR. I am redirecting those to files. Here is the code snippet:
# Start the HTTP Server
server = HTTPServer((config['HTTPServer']['listen'],config['HTTPServer']['port']),HTTPHandler)
# Fork our process to detach if not told to stay in foreground
if options.foreground is False:
try:
pid = os.fork()
if pid > 0:
logging.info('Parent process ending.')
sys.exit(0)
except OSError, e:
sys.stderr.write("Could not fork: %d (%s)\n" % (e.errno, e.strerror))
sys.exit(1)
# Second fork to put into daemon mode
try:
pid = os.fork()
if pid > 0:
# exit from second parent, print eventual PID before
print 'Daemon has started - PID # %d.' % pid
logging.info('Child forked as PID # %d' % pid)
sys.exit(0)
except OSError, e:
sys.stderr.write("Could not fork: %d (%s)\n" % (e.errno, e.strerror))
sys.exit(1)
logging.debug('After child fork')
# Detach from parent environment
os.chdir('/')
os.setsid()
os.umask(0)
# Close stdin
sys.stdin.close()
# Redirect stdout, stderr
sys.stdout = open('http_access.log', 'w')
sys.stderr = open('http_errors.log', 'w')
# Main Thread Object for Stats
threads = []
logging.debug('Kicking off threads')
while ...
lots of code here
...
server.serve_forever()
Am I doing something wrong here or is BaseHTTPServer somehow prevented from becoming daemonized?
Edit: Updated code to demonstrate the additional, previously missing code flow and that log.debug shows in my forked, background daemon I am hitting code after fork.
After a bit of googling I finally stumbled over this BaseHTTPServer documentation and after that I ended up with:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
server = ThreadedHTTPServer((config['HTTPServer']['listen'],config['HTTPServer']['port']), HTTPHandler)
server.serve_forever()
Which for the most part comes after I fork and ended up resolving my problem.
Here's how to do this with the python-daemon library:
from BaseHTTPServer import (HTTPServer, BaseHTTPRequestHandler)
import contextlib
import daemon
from my_app_config import config
# Make the HTTP Server instance.
server = HTTPServer(
(config['HTTPServer']['listen'], config['HTTPServer']['port']),
BaseHTTPRequestHandler)
# Make the context manager for becoming a daemon process.
daemon_context = daemon.DaemonContext()
daemon_context.files_preserve = [server.fileno()]
# Become a daemon process.
with daemon_context:
server.serve_forever()
As usual for a daemon, you need to decide how you will interact with the program after it becomes a daemon. For example, you might register a systemd service, or write a PID file, etc. That's all outside the scope of the question though.
In particular, it's outside the scope of the question to ask: once it's become a daemon process (necessarily detached from any controlling terminal), how do I stop the daemon process? That's up to you to decide, as part of defining the program's behaviour.
You start by instantiating a HTTPServer. But you don't actually tell it to start serving in any of the supplied code. In your child process try calling server.serve_forever().
See this for reference
A simple solution that worked for me was to override the BaseHTTPRequestHandler method log_message(), so we prevent any kind of writing in stdout and avoid problems when demonizing.
class CustomRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def log_message(self, format, *args):
pass
...
rest of custom class code
...
Just use daemontools or some other similar script instead of rolling your own daemonizing process. It is much better to keep this off your script.
Also, your best option: Don't use BaseHTTPServer. It is really bad. There are many good HTTP servers for python, i.e. cherrypy or paste. Both includes ready-to-use daemonizing scripts.
Since this has solicited answers since I originally posted, I thought that I'd share a little info.
The issue with the output has to do with the fact that the default handler for the logging module uses the StreamHandler. The best way to handle this is to create your own handlers. In the case where you want to use the default logging module, you can do something like this:
# Get the default logger
default_logger = logging.getLogger('')
# Add the handler
default_logger.addHandler(myotherhandler)
# Remove the default stream handler
for handler in default_logger.handlers:
if isinstance(handler, logging.StreamHandler):
default_logger.removeHandler(handler)
Also at this point I have moved to using the very nice Tornado project for my embedded http servers.