I'm asking the same question as this post, but in Python under Linux. I have a thread running in the background which output things constantly, but I would like to keep the input line at the bottom of the terminal without been flushed away. Can someone give me a code example? Suppose my code is like this:
from time import sleep
from multiprocessing import Process
def subproc():
while True:
print "something"
sleep(1)
def main():
p = Process(target=subproc)
p.start()
c = raw_input()
EDIT: by the way, I would like to keep the use of the raw_input function, if possible, because I want to have input history available.
unfortunately, there isn't any way of doing this that doesn't involve some serious or irritating consequences.
The way terminals scroll and handle stdin/stdout has not varied much in decades.
Take a look at terminal escape sequences. As long as the user terminal is based on the VT100 or ansi spec, these should work. You can use them to set the write location, clear lines, etc. But if a user scrolls, the illusion will break.
The problem of specifying a specific location on screen to output a character was addressed for early games like angband, rogue and nethack. The solution was the Curses library, and there happens to be a curses python module
By using curses, you can achieve what your after. But I know of no way of doing this in standard standard shells
Related
I want to run dialog --gauge from print()
Python file is like this
process1
print(30) #updates the gauge percentage to 30
process2
print(60) #updates the gauge percentage to 60
process3
print(100) # completes and exit the gauge
python run.py | dialog --gauge "Running program" 10 30 0
but the gauge is not updating or we can say python prints them at the last in an instant and not like they way they normally prints in terminal
How can I fix that, please tell
Not easy to answer if you don't provide the code.
But I am pretty sure you are missing a flush=True argument to your print.
import time
for i in range(100):
print(i, flush=True)
time.sleep(1)
works fine.
Some explanation about why it is needed: I/O are buffered, for obvious efficiency reason. That goes for print to. So there are different criteria to decide when to effectively do the print, that is when to empty (flush) the buffer and send the chars in it to whatever process is reading (for example, the terminal, in charge of rendering them, for an interactive process; or a process whose standard input is linked to our standard output through a pipe).
Obviously when output buffer is full. That is the most efficient, buffer size being optimized from different hardware optimums. Unless there are some applicative reasons not to do that
For interactive process (just python ran in a terminal), whenever a newline is printed. No need to bother the terminal char by char. Things are printed lines by lines. For most "line oriented" programs that is perfectly sufficient (oldie rant: don't forget that "terminals", which we should call "terminal emulators" are just an imitation of physical terminals, which, themselves, where some "paperless printer" replacing former printers. This is really "line oriented". There is a reason why CLI are not called "TCI" for "text command interface". Interaction with shells and terminal are mainly "line oriented").
But for non interactive process, there is no reason to treat newline differently. What is waiting for our output is neither a printer, nor anything imitating it. It is just another process, reading chars and processing them. There is no reason, a priori, to worry about interactivity.
Sometimes, tho, you need other rules that those default one. For example, because you are printing lines very slowly, and you want the terminal not to wait a newline before updating it. Or because you are feeding another process through a pipe, quite slowly too, and you don't want that other process to have to wait for your buffer to be full. Or, in your case, because you know that even tho your print are not directly sent to a terminal, but to another process, that is still an interactive application.
In those cases, you can force the flushing of the buffer after the print, with keyword argument flush=True
TL;DR, I'm trying to make a python-only single-module platform-nonspecific non-blocking keypress detector that works in IDLE.
Good luck.
So say I'm running a loop, in IDLE, for an arbitrary amount of time. I want a way to get user input without blocking the loop, on any system I want.
Example:
import keyboard #(is one module in same directory)(Mac, Windows, Linux, who knows?)
teststring = ''
while True:
if keyboard.isDown('a'):
teststring += 'a'
if keyboard.isDown('enter'):
if 'a' in testring:
break
... (neural network or whatever continues)
To put it a different way, I'm trying to make a universal event listener that works in IDLE. If one already exists, do let me know, but I have done some research and none work in one easy-to-use module.
Note:
You don't have to write this, I just want to know what pre-installed Python modules you would use to do this and a general idea of how to go about it.
I do not want to lose my sets if windows is about to shutdown/restart/log off/sleep, Is it possible to save it before shutdown? Or is there an alternative to save information without worring it will get lost on windows shutdown? JSON, CSV, DB? Anything?
s = {1,2,3,4}
with open("s.pick","wb") as f: # pickle it to file when PC about to shutdown to save information
pickle.dump(s,f)
I do not want to lose my sets if windows is about to shutdown/restart/log off/sleep, Is it possible to save it before shutdown?
Yes, if you've built an app with a message loop, you can receive the WM_QUERYENDSESSION message. If you want to have a GUI, most GUI libraries will probably wrap this up in their own way. If you don't need a GUI, your simplest solution is probably to use PyWin32. Somewhere in the docs there's a tutorial on creating a hidden window and writing a simple message loop. Just do that on the main thread, and do your real work on a background thread, and signal your background thread when a WM_QUERYENDSESSION message comes in.
Or, much more simply, as Evgeny Prokurat suggests, just use SetConsoleCtrlHandler (again through PyWin32). This can also catch ^C, ^BREAK, and the user closing your console, as well as the logoff and shutdown messages that WM_QUERYENDSESSION catches. More importantly, it doesn't require a message loop, so if you don't have any other need for one, it's a lot simpler.
Or is there an alternative to save information without worring it will get lost on windows shutdown? JSON, CSV, DB? Anything?
The file format isn't going to magically solve anything. However, a database could have two advantages.
First, you can reduce the problem by writing as often as possible. But with most file formats, that means rewriting the whole file as often as possible, which will be very slow. The solution is to streaming to a simpler "journal" file, packing that into the real file less often, and looking for a leftover journal at every launch. You can do that manually, but a database will usually do that for you automatically.
Second, if you get killed in the middle of a write, you end up with half a file. You can solve that by the atomic writing trick—write a temporary file, then replace the old file with the temporary—but this is hard to get right on Windows (especially with Python 2.x) (see Getting atomic writes right), and again, a database will usually do it for you.
The "right" way to do this is to create a new window class with a msgproc that dispatches to your handler on WM_QUERYENDSESSION. Just as MFC makes this easier than raw Win32 API code, win32ui (which wraps MFC) makes this easier than win32api/win32gui (which wraps raw Win32 API). And you can find lots of samples for that (e.g., a quick search for "pywin32 msgproc example" turned up examples like this, and searches for "python win32ui" and similar terms worked just as well).
However, in this case, you don't have a window that you want to act like a normal window, so it may be easier to go right to the low level and write a quick&dirty message loop. Unfortunately, that's a lot harder to find sample code for—you basically have to search the native APIs for C sample code (like Creating a Message Loop at MSDN), then figure out how to translate that to Python with the pywin32 documentation. Less than ideal, especially if you don't know C, but not that hard. Here's an example to get you started:
def msgloop():
while True:
msg = win32gui.GetMessage(None, 0, 0)
if msg and msg.message == win32con.WM_QUERYENDSESSION:
handle_shutdown()
win32api.TranslateMessage(msg)
win32api.DispatchMessage(msg)
if msg and msg.message == win32con.WM_QUIT:
return msg.wparam
worker = threading.Thread(real_program)
worker.start()
exitcode = msgloop()
worker.join()
sys.exit(exitcode)
I haven't shown the "how to create a minimal hidden window" part, or how to signal the worker to stop with, e.g., a threading.Condition, because there are a lot more (and easier-to-find) good samples for those parts; this is the tricky part to find.
you can detect windows shutdown/log off with win32api.setConsoleCtrlHandler
there is a good example How To Catch “Kill” Events with Python
I'm trying to write a simple Python IRC client. So far I can read data, and I can send data back to the client if it automated. I'm getting the data in a while True, which means that I cannot enter text while at the same time reading data. How can I enter text in the console, that only gets sent when I press enter, while at the same time running an infinite loop?
Basic code structure:
while True:
read data
#here is where I want to write data only if it contains '/r' in it
Another way to do it involves threads.
import threading
# define a thread which takes input
class InputThread(threading.Thread):
def __init__(self):
super(InputThread, self).__init__()
self.daemon = True
self.last_user_input = None
def run(self):
while True:
self.last_user_input = input('input something: ')
# do something based on the user input here
# alternatively, let main do something with
# self.last_user_input
# main
it = InputThread()
it.start()
while True:
# do something
# do something with it.last_user_input if you feel like it
What you need is an event loop of some kind.
In Python you have a few options to do that, pick one you like:
Twisted https://twistedmatrix.com/trac/
Asyncio https://docs.python.org/3/library/asyncio.html
gevent http://www.gevent.org/
and so on, there are hundreds of frameworks for this, you could also use any of the GUI frameworks like tkinter or PyQt to get a main event loop.
As comments have said above, you can use threads and a few queues to handle this, or an event based loop, or coroutines or a bunch of other architectures. Depending on your target platforms one or the other might be best. For example on windows the console API is totally different to unix ptys. Especially if you later need stuff like colour output and so on, you might want to ask more specific questions.
You can use a async library (see answer of schlenk) or use https://docs.python.org/2/library/select.html
This module provides access to the select() and poll() functions
available in most operating systems, epoll() available on Linux 2.5+
and kqueue() available on most BSD. Note that on Windows, it only
works for sockets; on other operating systems, it also works for other
file types (in particular, on Unix, it works on pipes). It cannot be
used on regular files to determine whether a file has grown since it
was last read.
There are alot of very similar questions to this but I can't find one that applies specifically to what I'm trying to do.
I have a simulation (written in SimPy) that I'm writing a GUI for, the main output of the simulation is text - to the console from 'print' statements. Now, I thought the simplest way would be to create a seperate module GUI.py, and import my simulation program into it:
import osi_model
I want all the print statements to be captured by the GUI and appear inside a Textctrl, which there's countless examples of on here, along these lines:
class MyFrame(wx.Frame):
def __init__(self, *args, **kwds):
<general frame initialisation stuff..>
redir=RedirectText(self.txtCtrl_1)
sys.stdout=redir
class RedirectText:
def __init__(self,aWxTextCtrl):
self.out=aWxTextCtrl
def write(self,string):
self.out.WriteText(string)
I am also starting my simulation from a 'Go' button:
def go_btn_click(self, event):
print 'GO'
self.RT = threading.Thread(target=osi_model.RunThis())
self.RT.start()
This all works fine, and the output from the simulation module is captured by the TextCtrl, except the GUI locks up and becomes unresponsive - I still need it to be accessible (at the very minimum to have a 'Stop' button). I'm not sure if this is a botched attempt at creating a new thread that I've done here, but I assume a new thread will be needed at some stage in this process.
People suggest using wx.CallAfter, but I'm not sure how to go about this considering the imported module doesn't know about wx, and also I can't realistically go through the entire simulation architecture and change all the print statements to wx.CallAfter, and any attempt to capture the shell from inside the imported simulation program leads to the program crashing.
Does anybody have any ideas about how I can best achieve this? So all I really need is for all console text to be captured by a TextCtrl while the GUI remains responsive, and all text is solely coming from an imported module.
(Also, secondary question regarding a Stop button - is it bad form to just kill the simulation thread?).
Thanks,
Duncan
I would suggest looking into this WX wiki article about long running tasks.
It specifically addresses the situation you're dealing with using a "start" button to being a long running process. Several different examples are given using different techniques like threads and idle handlers.
I think you would have to redirect stdout to a log file (or simple SQLite database?) and then use your thread to check the log file for updates which it would then pass along to the GUI using wx.CallAfter or similar. You might be able to use something like a socket server built in Python: http://wiki.wxpython.org/AsynchronousSockets. I think the wxPython Cookbook mentioned something about using an RPC server too (probably this one: http://docs.python.org/library/simplexmlrpcserver.html), but I don't remember the details.
You should also try asking on the official wxPython mailing list. They're very friendly over there.