We Keep Coding

How to create a new console in Python to print message

I make a python script running in the console, and I want to create another console for printing important messages without running another python script to do that.
I first tried to use win32console.AllocConsole() directly, but it got Access is denied
(Seemingly because one process can attach to at most one console according to the docs).
So I tried creating a new process by using multiprocessing :
import sys, os
import win32api, win32con, win32console
import multiprocessing
def ShowConsole():
win32console.FreeConsole()
win32console.AllocConsole()
sys.stdout = open("CONOUT$", "w")
sys.stderr = open("CONOUT$", "w")
print("Test")
os.system("pause")
if __name__ == '__main__':
p = multiprocessing.Process(target=ShowConsole)
p.start()
But when I ran the code in Powershell, it exited directly with no message while no new console is created.
None of the possible solutions I found in stackoverflow works for me. What should I do?
Update: It turns out that it is because multiprocessing.Process fails to call ShowConsole function. I use multiprocessing.dummy.Process as the alternative and it works as expected.
The reason why multiprocessing.Process fails to call target is still unclear.

There's nothing wrong with your example above, it pops the console as shown below. I added a "hello" in the main section to differentiate.
But since you want to values from the first console to the second,
here's a better example. Utilize put/get to pass the information from the first console to the second console.
import win32console
import multiprocessing
import time
def secondconsole(output):
win32console.FreeConsole()
win32console.AllocConsole()
while True:
print(output.get())
if __name__ == "__main__":
output = multiprocessing.Queue()
multiprocessing.Process(target=secondconsole, args=[output]).start()
while True:
print("Hello World")
output.put("Hello to second console") #here you will provide the data to the second console
time.sleep(3) #sleep for 3 seconds just for testing

It looks like the issue might be with the way you are trying to open the console using sys.stdout and sys.stderr. Try using the following code instead:
import sys, os
import win32api, win32con, win32console
import multiprocessing
def ShowConsole():
win32console.FreeConsole()
win32console.AllocConsole()
os.dup2(win32console.GetStdHandle(win32console.STD_OUTPUT_HANDLE), sys.stdout.fileno())
os.dup2(win32console.GetStdHandle(win32console.STD_ERROR_HANDLE), sys.stderr.fileno())
print("Test")
os.system("pause")
if __name__ == '__main__':
p = multiprocessing.Process(target=ShowConsole)
p.start()

How to listen for a file change in urwid?

I want to remote control a python application which uses urwid for the user interface.
My idea was to create a file, pass it's name as command line argument to the application and whenever I write to the file the application reads from that file.
Urwid's event loop has a method watch_file(fd, callback).
This method is described as "Call callback() when fd has some data to read."
This sounds exactly like what I want to have, but it causes an infinite loop.
callback is executed as often as possible, despite the fact that the file is empty.
Even if I delete the file, callback is still called.
#!/usr/bin/env python3
import urwid
import atexit
def onkeypress(key, size=None):
if key == 'q':
raise urwid.ExitMainLoop()
text.set_text(key)
def onfilechange():
text.set_text(cmdfile.read())
# clear file so that I don't read already executed commands again
# and don't run into an infinite loop - but I am doing that anyway
with open(cmdfile.name, 'w') as f:
pass
cmdfile = open('/tmp/cmd', 'rt')
atexit.register(cmdfile.close)
text = urwid.Text("hello world")
filler = urwid.Filler(text)
loop = urwid.MainLoop(filler, unhandled_input=onkeypress)
loop.watch_file(cmdfile, onfilechange)
if __name__ == '__main__':
loop.run()
(My initial idea was to open the file only for reading instead of keeping it open all the time but fd has to be a file object, not a path.)
Urwid offers several different event loops.
By default, SelectEventLoop is used.
GLibEventLoop has the same behaviour, it runs into an infinite loop.
AsyncioEventLoop instead throws an "operation not permitted" exception.
TwistedEventLoop and TornadoEventLoop would need additional software to be installed.
I have considered using the independent watchdog library but it seems accessing the user interface from another thread would require to write a new loop, see this stack overflow question.
The answer to that question recommends polling instead which I would prefer to avoid.
If urwid specifically provides a method to watch a file I cannot believe that it does not work in any implementation.
So what am I doing wrong?
How do I react to a file change in a python/urwid application?
EDIT:
I have tried using named pipes (and removed the code to clear the file) but visually it has the same behaviour: the app does not start.
Audibly, however, there is a great difference: It does not go into the infinite loop until I write to the file.
Before I write to the file callback is not called but the app is not started either, it just does nothing.
After I write to the file, it behaves as described above for regular files.

I have found the following work around: read a named pipe in another thread, safe each line in a queue and poll in the UI thread to see if something is in the queue.
Create the named pipe with mkfifo /tmp/mypipe.
Then write to it with echo >>/tmp/mypipe "some text".
#!/usr/bin/env python3
import os
import threading
import queue
import urwid
class App:
POLL_TIME_S = .5
def __init__(self):
self.text = urwid.Text("hello world")
self.filler = urwid.Filler(self.text)
self.loop = urwid.MainLoop(self.filler, unhandled_input=self.onkeypress)
def watch_pipe(self, path):
self._cmd_pipe = path
self.queue = queue.Queue()
threading.Thread(target=self._read_pipe_thread, args=(path,)).start()
self.loop.set_alarm_in(0, self._poll_queue)
def _read_pipe_thread(self, path):
while self._cmd_pipe:
with open(path, 'rt') as pipe:
for ln in pipe:
self.queue.put(ln)
self.queue.put("!! EOF !!")
def _poll_queue(self, loop, args):
while not self.queue.empty():
ln = self.queue.get()
self.text.set_text(ln)
self.loop.set_alarm_in(self.POLL_TIME_S, self._poll_queue)
def close(self):
path = self._cmd_pipe
# stop reading
self._cmd_pipe = None
with open(path, 'wt') as pipe:
pipe.write("")
os.remove(path)
def run(self):
self.loop.run()
def onkeypress(self, key, size=None):
if key == 'q':
raise urwid.ExitMainLoop()
self.text.set_text(key)
if __name__ == '__main__':
a = App()
a.watch_pipe('/tmp/mypipe')
a.run()
a.close()

How to handle console exit and object destruction

Given this code:
from time import sleep
class TemporaryFileCreator(object):
def __init__(self):
print 'create temporary file'
# create_temp_file('temp.txt')
def watch(self):
try:
print 'watching tempoary file'
while True:
# add_a_line_in_temp_file('temp.txt', 'new line')
sleep(4)
except (KeyboardInterrupt, SystemExit), e:
print 'deleting the temporary file..'
# delete_temporary_file('temp.txt')
sleep(3)
print str(e)
t = TemporaryFileCreator()
t.watch()
during the t.watch(), I want to close this application in the console..
I tried using CTRL+C and it works:
However, if I click the exit button:
it doesn't work.. I checked many related questions about this but it seems that I cannot find the right answer..
What I want to do:
The console can be exited while the program is still running.. to handle that, when the exit button is pressed, I want to make a cleanup of the objects (deleting of created temporary files), rollback of temporary changes, etc..
Question:
how can I handle console exit?
how can I integrate it on object destructors (__exit__())
Is it even possible? (how about py2exe?)
Note: code will be compiled on py2exe.. "hopes that the effect is the same"

You may want to have a look at signals. When a *nix terminal is closed with a running process, this process receives a couple signals. For instance this code waits for the SIGHUB hangup signal and writes a final message. This codes works under OSX and Linux. I know you are specifically asking for Windows but you might want to give it a shot or investigate what signals a Windows command prompt is emitting during shutdown.
import signal
import sys
def signal_handler(signal, frame):
with open('./log.log', 'w') as f:
f.write('event received!')
signal.signal(signal.SIGHUP, signal_handler)
print('Waiting for the final blow...')
#signal.pause() # does not work under windows
sleep(10) # so let us just wait here
Quote from the documentation:
On Windows, signal() can only be called with SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, or SIGTERM. A ValueError will be raised in any other case.
Update:
Actually, the closest thing in Windows is win32api.setConsoleCtrlHandler (doc). This was already discussed here:
When using win32api.setConsoleCtrlHandler(), I'm able to receive shutdown/logoff/etc events from Windows, and cleanly shut down my app.
And if Daniel's code still works, this might be a nice way to use both (signals and CtrlHandler) for cross-platform purposes:
import os, sys
def set_exit_handler(func):
if os.name == "nt":
try:
import win32api
win32api.SetConsoleCtrlHandler(func, True)
except ImportError:
version = “.”.join(map(str, sys.version_info[:2]))
raise Exception(”pywin32 not installed for Python ” + version)
else:
import signal
signal.signal(signal.SIGTERM, func)
if __name__ == "__main__":
def on_exit(sig, func=None):
print "exit handler triggered"
import time
time.sleep(5)
set_exit_handler(on_exit)
print "Press to quit"
raw_input()
print "quit!"

If you use tempfile to create your temporary file, it will be automatically deleted when the Python process is killed.
Try it with:
>>> foo = tempfile.NamedTemporaryFile()
>>> foo.name
'c:\\users\\blah\\appdata\\local\\temp\\tmpxxxxxx'
Now check that the named file is there. You can write to and read from this file like any other.
Now kill the Python window and check that file is gone (it should be)
You can simply call foo.close() to delete it manually in your code.

PySide app seems to ignore SIGTERM during shutdown when run at startup using ~/.config/autostart

My PySide app seems to ignore the TERM signal sent during shutdown on Linux when it is automatically started by putting an entry in ~/.config/autostart.
I came to the conclusion that the issue is with PySide/Qt through the following experiment.
First, I created a simple Python script that catches the TERM signal and prints a confirmation then exits. Here is the code.
import signal
import sys
import time
def on_quit(*args):
print 'here', args
sys.exit(0)
for i in [x for x in dir(signal) if x.startswith('SIG')]:
try:
signum = getattr(signal, i)
if i == 'SIGCHILD' or i == 'SIGCHLD':
continue
signal.signal(signum, on_quit)
except Exception, m:
print 'Skipping %s' % i
time.sleep(1000000)
The for loop just registers the on_quit method on all possible signals.
Calling this in a terminal and sending it the TERM signal by killing it, I can confirm that it is being caught and processed by the script.
I then tried to run this script at startup by including an entry in ~/.config/autostart. The entry is similar to the following:
[Desktop Entry]
Version=1.0
Type=Application
Name=Test
GenericName=Test
Comment=Test
Exec=strace -T -f -tt -o run_test.strace run_test
Terminal=false
StartupNotify=false
I used strace to trace the system calls of the script to confirm in another way that it is catching the TERM signal. The options to strace tells it to include the time spent on the system call, to follow forks, to print the time of day with microseconds, and to send the output to run_test.strace, in that order.
run_test is just a bash script that calls the Python script; it is included because that is how my original PySide app is being called. The code for it is:
#!/bin/bash
python test.py &>out.test
I proceeded to test it by adding the desktop entry to ~/.config/autostart, restarting the computer, then shutting it down. run_test.strace contained the line "--- SIGTERM (Terminated) # 0 (0) ---" which confirms that SIGTERM was sent and caught. out.test also contains the expected output which further confirms it.
I then tried to test a simple PySide application with signal handling. The code is as follows.
import signal
import sys
import time
from PySide.QtGui import QApplication
from PySide.QtCore import QTimer
app = QApplication([])
timer = QTimer()
timer.start(500)
timer.timeout.connect(lambda: None)
def on_quit(*args):
print 'here', args
sys.exit(0)
for i in [x for x in dir(signal) if x.startswith('SIG')]:
try:
signum = getattr(signal, i)
if i == 'SIGCHILD' or i == 'SIGCHLD':
continue
signal.signal(signum, on_quit)
except Exception, m:
print 'Skipping %s' % i
app.exec_()
The QTimer is essential to allow the signal handler to execute.
Running the above script in a terminal and sending it the TERM signal, the output still confirms that the TERM signal is being caught and processed. Running it through strace also shows the line "--- SIGTERM (Terminated) # 0 (0) ---".
When running it during startup, however, the output shows that the on_quit method is not called nor is there an indication in the strace output that it was sent the TERM signal.
Could anyone help? I can provide the strace outputs if needed. System is Ubuntu 12.04 32bit.
edit: Further experimentation shows that it happens with QApplication but not with QCoreApplication. It's something probably GUI related then?

Urwid: how to see errors?

I am building application with interactive console interface (line htop, atop utilities) using urwid library, so my trouble is: as interface takes all the space in console window - I could not see python's errors, I tried to do that:
import sys
f = open("test_err", "w")
original_stderr = sys.stderr
sys.stderr = f
print a #a is undefined
sys.stderr = original_stderr
f.close()
It works when I dont use urwid, but not when I use it...

you could try redirecting errors to a file. after each time you run the program, you will need to refresh the file. most editors let you easily do this by pushing f5
def main():
#your code here
print someError #raises an error
try: #run main function
main()
except BaseException as err: #catch all errors
with open('errors.txt','a') as errors: #open a file to write the errors to
errors.write(err.message+'\n')#write the error
change the 'a' to 'w' in the open function if you only want to see one error in the file at a time (instead of having multiple error over a long period of time in one file).
if you want to see the error right when it happens, you can make the error catcher open a window that has the error on it.
def main():
#your code here
print someErr
try: #run main function
main()
except BaseException as err: #catch all errors
import Tkinter as tk #imports the ui module
root = tk.Tk() #creates the root of the window
#creates the text and attaches it to the root
window = tk.Label(root, text=err.message)
window.pack()
#runs the window
root.mainloop()
if you want to build your own window to catch errors, you can learn about Tkinter here. (it is built into python, you don't have to install anything)

Here's what I came up with. I'm taking advantage of unicode-rxvt (urxvt) feature to be passed in a file descriptor. Of course this means you need to be developing this in an X environment, and not a console.
from __future__ import print_function
import os
from datetime import datetime
_debugfile = None
def _close_debug(fo):
fo.close()
def DEBUG(*obj):
"""Open a terminal emulator and write messages to it for debugging."""
global _debugfile
if _debugfile is None:
import atexit
masterfd, slavefd = os.openpty()
pid = os.fork()
if pid: # parent
os.close(masterfd)
_debugfile = os.fdopen(slavefd, "w+", 0)
atexit.register(_close_debug, _debugfile)
else: # child
os.close(slavefd)
os.execlp("urxvt", "urxvt", "-pty-fd", str(masterfd))
print(datetime.now(), ":", ", ".join(map(repr, obj)), file=_debugfile)
This will open a new terminal window automatically when you call DEBUG for the first time and close it at exit. Then any messages passed to it are shown in this new window. This is your "debug window". So your main app works normally, without cluttering it up with messages, but you can still see debug output in this new terminal.