Hey there Stack Overflow. I'm trying to build a testing script that should mix outputting changing characters (using curses) on multiple lines (creating them over time), creating new lines based on the thread number.
I have the below code:
# -*- coding: utf-8 -*-
import curses, time, threading
def threadedFunction(linePos):
stdscr = curses.initscr()
curses.noecho()
curses.cbreak()
try:
stdscr.clear()
for i in range(50):
stdscr.addstr(linePos, 0, "testing %s..." % i)
stdscr.refresh()
time.sleep(.1)
finally:
curses.echo()
curses.nocbreak()
curses.endwin()
pass
pass
if __name__ == "__main__":
for x in xrange(0, 4): # should produce 5 lines maximum
exec("process" + str(x) + " = threading.Thread(target = threadedFunction, args = (" + str(x) + ",))")
exec("process" + str(x) + ".start()")
I tried using the multithreading library before, but I had no hope with it. The threading library at least will display the numbers I want on a few lines before it goes crazy. Here's an example of what it does when I run it:
All I want is for the program to just simply start a new thread, and display a line that counts to 50 while adding new lines doing the same thing. How would I go about doing this?? Thanks in advance :)
Printing to the terminal from multiple threads will give you intermingled output like that. It is a very simple example of race condition. Use some kind of locking mechanism to coordinate writes to the terminal, or make sure to only write from one thread (for example, using a FIFO to pass message to the writing thread, which will write them to the terminal).
The weird numbers you see are part of the ANSI escape sequences that are used by programs to use special features of the terminal: writing \x1B[nF to the output will make your terminal move the cursor one line up, for example. Curses is outputting such codes for you, and because the terminal interprets them according to the ANSI meaning, you don't usually see them. But because of the multithreading issue, those become mingled and invalid, and part of them get printed to the screen.
Even if you only use curses in one thread, other processing-heavy threads can disrupt the escape sequences in the curses thread. The environment variable $ESCDELAY indicates how long (in ms) to wait after an escape code (0x1B) is sent; and if more than that time elapsed, a ^[ keystroke (ESC) is returned by get_wch().
Use stdscr.noutrefresh() instead of stdscr.refresh(), then call curses.doupdate() in a designated thread that handles the update. The idea is to do curses.doupdate() only in one thread.
Related
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
Considering the following curses console program in python. It should print the key variable on the terminal. However, there is also a window active so when print() invokes it really messes up the entire screen.
I would like the print to occur in a predictable place so at the end of the game I can output some results to the non-curses terminal.
What is the right way to use python print() and curses in single program?
import curses
screen = curses.initscr()
h, w = screen.getmaxyx()
window = curses.newwin(10, 20, 0, 0)
window.keypad(1)
curses.curs_set(0)
window.border(0)
window.timeout(100)
inputs = [];
while True:
key = window.getch()
inputs.append(key)
print(key) # <---- this messes things up
print(f"You typed: ${inputs}") # <---- this messes things up
In practice, the print() is for debug logging and other variable outputs that I need for the game.
Thank you
I tried using window.addstr() function to output the results. But I'm after how to utilise print for system output at the end of the game rather than modifying the console window.
curses' output stream and print stream write by default to the same device (e.g., your terminal), but are buffered independently (see release notes for ncurses 6.0 in 2015).
You can mix the two by temporarily suspending curses mode. For instance, you may find these helpful:
reset_shell_mode and reset_prog_mode
endwin and refresh
(getch does a refresh, so you should be able to use that instead of an explicit refresh).
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
I have a multithreaded program written in Python where I have a number of things happening at the same time:
reading raw data from an external source
organizing such data into different lists (parsing)
saving the post-parsed data into mass storage
some threads being used to flush some buffers from time to time and other optimizations
displaying parts of such data for real-time monitoring, using Curses.
The program is latency-sensitive, so I really need this to be multithreaded.
I got the curses thread to display correctly what I want to.
The problem is that while I had everything working without the curses thread, I had a "killswitch" in the main() function that terminated all activity at the press of a key.
I have this global variable called "killThreads" that goes into all functions who are called as threads, and all these functions only work as:
def oneThread():
while (not killThreads):
doStuff()
...
And then the main function defines the killThread as False, initializes all threads and turns the killThread as True after a raw_input():
killThreads=False
thisThread=threading.Thread(target=oneThread)
otherThread=threading.Thread(target=twoThread)
thisThread.setDaemon(True)
otherThread.setDaemon(True)
thisThread.start()
otherThread.start()
raw_input('Press to end the program')
killThreads=True
Everything ran fine until I ran a thread with the Curses module to display data.
It seems that while the Curses thread is on, it takes over all input commands. I tried to use getch() with no success. All I could do to keep everything running was to establish a timer within the Curses function:
def displayData():
screen=curses.initscr()
screen.nodelay(1)
timeKill=0
while (timeKill<80):
#stuff is drawn#
time.sleep(0.25)
timeKill+=1
Could anyone tell me how to go over Curses and get my keyboard input to "reach" the main function and kill all threads? Or do I always have to input to Curses and then make the Curses function alter the killThreads variable? If so, how do I do it (or where do I find the documentation for that)?
Thank you so much for your help.
Nice to meet you.
I'm trying to accomplish the same today. Look at this solution:
killThreads=False
thisThread=threading.Thread(target=oneThread)
otherThread=threading.Thread(target=twoThread)
thisThread.setDaemon(True)
otherThread.setDaemon(True)
thisThread.start()
otherThread.start()
raw_input('Press "q" to end the program')
key = ''
while key != ord('q'):
key = screen.getch()
killThreads=True
curses.nocbreak(); screen.keypad(0); curses.echo()
curses.endwin()
See, while will looping very fastly and waiting for q button be pressed, before switch your var killThreads to True.
It is pretty common practice. However, this while loop making thousands of idle loops in second, may be there can be more elegant way or better to embed into this while loop time.sleep(0.1) at least.
Most of questions related to this topics here in SO is as follows:
How to print some information on the same line without introducing a
new line
Q1 Q2.
Instead, my question is as follows:
I expect to see the following effect,
>> You have finished 10%
where the 10 keep increasing in the same time. I know how to do this in C++ but cannot
find a good solution in python.
import sys, time
for i in xrange(0, 101, 10):
print '\r>> You have finished %d%%' % i,
sys.stdout.flush()
time.sleep(2)
print
The \r is the carriage return. You need the comma at the end of the print statement to avoid automatic newline. Finally sys.stdout.flush() is needed to flush the buffer out to stdout.
For Python 3, you can use:
print("\r>> You have finished {}%".format(i), end='')
Python 3
You can use keyword arguments to print:
print('string', end='\r', flush=True)
end='\r' replaces the default end-of-line behavior with '\r'
flush=True flushes the buffer, making the printed text appear immediately.
Python 2
In 2.6+ you can use from __future__ import print_function at the start of the script to enable Python 3 behavior. Or use the old way:
Python's print puts a newline after each command, unless you suppress it with a trailing comma. So, the print command is:
print 'You have finished {0}%\r'.format(percentage),
Note the comma at the end.
Unfortunately, Python only sends the output to the terminal after a complete line. The above is not a complete line, so you need to flush it manually:
import sys
sys.stdout.flush()
On linux( and probably on windows) you can use curses module like this
import time
import curses
win = curses.initscr()
for i in range(100):
win.clear()
win.addstr("You have finished %d%%"%i)
win.refresh()
time.sleep(.1)
curses.endwin()
Benfit with curses as apposed to other simpler technique is that, you can draw on terminal like a graphics program, because curses provides moving to any x,y position e.g. below is a simple script which updates four views
import time
import curses
curses.initscr()
rows = 10
cols= 30
winlist = []
for r in range(2):
for c in range(2):
win = curses.newwin(rows, cols, r*rows, c*cols)
win.clear()
win.border()
winlist.append(win)
for i in range(100):
for win in winlist:
win.addstr(5,5,"You have finished - %d%%"%i)
win.refresh()
time.sleep(.05)
curses.endwin()
I had to combine a few answers above to make it work on Python 3.7 / Windows 10. The example runs on Spyder's console:
import sys, time
for i in range(0, 101, 5):
print("\r>> You have finished {}%".format(i), end='')
sys.stdout.flush()
time.sleep(.2)
The time.sleep(.2) is just used to simulates some time-consuming code.
using sys.stdout.write() instead of print works in both python 2 and 3 without any compromises.
The OP didn't specify Py2 or Py3. In Python 3 the 'import' of 'sys' and the 'sys.stdout' call can be replaced with 'flush=True':
import time
for i in range(0,101,25):
print("\r>>TESTING - {:0>3d}%".format(i), end='', flush=True)
time.sleep(.5)
print()
Thanks to Petr Viktorin for showing the "flush" parameter for Python 3 print(). I submit this because his Python 3 example doesn't include a 'format' specifier. It took me awhile to figure out that the additional parameters go after the 'format' specifier parentheses as shown in my example. I just picked an example format of 3 character integer 0 filled on the left. The best doc I found for Py3 format is: 6.1.3.1. Format Specification Mini-Language