strange terminal output when using colour/style formatting - python

I have the code:
import sys
for i in range(0, 20):
for j in range(0, 20):
sys.stdout.write('\x1b[1;32;40m' + ' ' + '\x1b[0m')
sys.stdout.write("\n")
which outputs 400 white squares in a 20x20 grid but also after about 180 squares outputs [1;32;40m in between some of the squares. It doesn't always output in the same place. Why is this happening, and how can it be fixed?

It's probably caused by some buffering bug in the terminal (I guess).
I can't replicate your issue, so here are some suggestions.
Output to sys.stdout is buffered. You could try flushing the output after each line of text ~
sys.stdout.flush()
But a better solution might be to add the bunch of escape codes to a string, then write it in a block ~
import sys
colour_line = '\x1b[1;32;40m' + (' ' * 20) + '\x1b[0m' + '\n'
for i in range(0, 20):
sys.stdout.write(colour_line)
This obviously simplifies the code, but this is getting away from your original design.

It's not ideal, but
time.sleep(0.0005)
or a different adjusted time between each print seems to work

Related

How to fully clear a single line in a command-line application? [duplicate]

If I had the following code:
for x in range(10):
print(x)
I would get the output of
1
2
etc..
What I would like to do is instead of printing a newline, I want to replace the previous value and overwrite it with the new value on the same line.
Simple Version
One way is to use the carriage return ('\r') character to return to the start of the line without advancing to the next line.
Python 3
for x in range(10):
print(x, end='\r')
print()
Python 2.7 forward compatible
from __future__ import print_function
for x in range(10):
print(x, end='\r')
print()
Python 2.7
for x in range(10):
print '{}\r'.format(x),
print
Python 2.0-2.6
for x in range(10):
print '{0}\r'.format(x),
print
In the latter two (Python 2-only) cases, the comma at the end of the print statement tells it not to go to the next line. The last print statement advances to the next line so your prompt won't overwrite your final output.
Line Cleaning
If you can’t guarantee that the new line of text is not shorter than the existing line, then you just need to add a “clear to end of line” escape sequence, '\x1b[1K' ('\x1b' = ESC):
for x in range(75):
print('*' * (75 - x), x, end='\x1b[1K\r')
print()
Since I ended up here via Google but am using Python 3, here's how this would work in Python 3:
for x in range(10):
print("Progress {:2.1%}".format(x / 10), end="\r")
Related answer here: How can I suppress the newline after a print statement?
#Mike DeSimone answer will probably work most of the time. But...
for x in ['abc', 1]:
print '{}\r'.format(x),
-> 1bc
This is because the '\r' only goes back to the beginning of the line but doesn't clear the output.
If POSIX support is enough for you, the following would clear the current line and leave the cursor at its beginning:
print '\x1b[2K\r',
It uses ANSI escape code to clear the terminal line. More info can be found in wikipedia and in this great talk.
Other approach
This other, (arguably worse) solution I have found looks like this:
last_x = ''
for x in ['abc', 1]:
print ' ' * len(str(last_x)) + '\r',
print '{}\r'.format(x),
last_x = x
-> 1
One advantage is that it will work on windows too.
I had the same question before visiting this thread. For me the sys.stdout.write worked only if I properly flush the buffer i.e.
for x in range(10):
sys.stdout.write('\r'+str(x))
sys.stdout.flush()
Without flushing, the result is printed only at the end out the script
Suppress the newline and print \r.
print 1,
print '\r2'
or write to stdout:
sys.stdout.write('1')
sys.stdout.write('\r2')
for x in range(10):
time.sleep(0.5) # shows how its working
print("\r {}".format(x), end="")
time.sleep(0.5) is to show how previous output is erased and new output is printed
"\r" when its at the start of print message , it gonna erase previous output before new output.
Try this:
import time
while True:
print("Hi ", end="\r")
time.sleep(1)
print("Bob", end="\r")
time.sleep(1)
It worked for me. The end="\r" part is making it overwrite the previous line.
WARNING!
If you print out hi, then print out hello using \r, you’ll get hillo because the output wrote over the previous two letters. If you print out hi with spaces (which don’t show up here), then it will output hi. To fix this, print out spaces using \r.
This works on Windows and python 3.6
import time
for x in range(10):
time.sleep(0.5)
print(str(x)+'\r',end='')
Here's a cleaner, more "plug-and-play", version of #Nagasaki45's answer. Unlike many other answers here, it works properly with strings of different lengths. It achieves this by clearing the line with just as many spaces as the length of the last line printed print. Will also work on Windows.
def print_statusline(msg: str):
last_msg_length = len(getattr(print_statusline, 'last_msg', ''))
print(' ' * last_msg_length, end='\r')
print(msg, end='\r')
sys.stdout.flush() # Some say they needed this, I didn't.
setattr(print_statusline, 'last_msg', msg)
Usage
Simply use it like this:
for msg in ["Initializing...", "Initialization successful!"]:
print_statusline(msg)
time.sleep(1)
This small test shows that lines get cleared properly, even for different lengths:
for i in range(9, 0, -1):
print_statusline("{}".format(i) * i)
time.sleep(0.5)
I couldn't get any of the solutions on this page to work for IPython, but a slight variation on #Mike-Desimone's solution did the job: instead of terminating the line with the carriage return, start the line with the carriage return:
for x in range(10):
print '\r{0}'.format(x),
Additionally, this approach doesn't require the second print statement.
The accepted answer is not perfect. The line that was printed first will stay there and if your second print does not cover the entire new line, you will end up with garbage text.
To illustrate the problem save this code as a script and run it (or just take a look):
import time
n = 100
for i in range(100):
for j in range(100):
print("Progress {:2.1%}".format(j / 100), end="\r")
time.sleep(0.01)
print("Progress {:2.1%}".format(i / 100))
The output will look something like this:
Progress 0.0%%
Progress 1.0%%
Progress 2.0%%
Progress 3.0%%
What works for me is to clear the line before leaving a permanent print. Feel free to adjust to your specific problem:
import time
ERASE_LINE = '\x1b[2K' # erase line command
n = 100
for i in range(100):
for j in range(100):
print("Progress {:2.1%}".format(j / 100), end="\r")
time.sleep(0.01)
print(ERASE_LINE + "Progress {:2.1%}".format(i / 100)) # clear the line first
And now it prints as expected:
Progress 0.0%
Progress 1.0%
Progress 2.0%
Progress 3.0%
I'm a bit surprised nobody is using the backspace character. Here's one that uses it.
import sys
import time
secs = 1000
while True:
time.sleep(1) #wait for a full second to pass before assigning a second
secs += 1 #acknowledge a second has passed
sys.stdout.write(str(secs))
for i in range(len(str(secs))):
sys.stdout.write('\b')
Here's my solution! Windows 10, Python 3.7.1
I'm not sure why this code works, but it completely erases the original line. I compiled it from the previous answers. The other answers would just return the line to the beginning, but if you had a shorter line afterwards, it would look messed up like hello turns into byelo.
import sys
#include ctypes if you're on Windows
import ctypes
kernel32 = ctypes.windll.kernel32
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
#end ctypes
def clearline(msg):
CURSOR_UP_ONE = '\033[K'
ERASE_LINE = '\x1b[2K'
sys.stdout.write(CURSOR_UP_ONE)
sys.stdout.write(ERASE_LINE+'\r')
print(msg, end='\r')
#example
ig_usernames = ['beyonce','selenagomez']
for name in ig_usernames:
clearline("SCRAPING COMPLETE: "+ name)
Output - Each line will be rewritten without any old text showing:
SCRAPING COMPLETE: selenagomez
Next line (rewritten completely on same line):
SCRAPING COMPLETE: beyonce
(Python3) This is what worked for me. If you just use the \010 then it will leave characters, so I tweaked it a bit to make sure it's overwriting what was there. This also allows you to have something before the first print item and only removed the length of the item.
print("Here are some strings: ", end="")
items = ["abcd", "abcdef", "defqrs", "lmnop", "xyz"]
for item in items:
print(item, end="")
for i in range(len(item)): # only moving back the length of the item
print("\010 \010", end="") # the trick!
time.sleep(0.2) # so you can see what it's doing
One more answer based on the prevous answers.
Content of pbar.py:
import sys, shutil, datetime
last_line_is_progress_bar=False
def print2(print_string):
global last_line_is_progress_bar
if last_line_is_progress_bar:
_delete_last_line()
last_line_is_progress_bar=False
print(print_string)
def _delete_last_line():
sys.stdout.write('\b\b\r')
sys.stdout.write(' '*shutil.get_terminal_size((80, 20)).columns)
sys.stdout.write('\b\r')
sys.stdout.flush()
def update_progress_bar(current, total):
global last_line_is_progress_bar
last_line_is_progress_bar=True
completed_percentage = round(current / (total / 100))
current_time=datetime.datetime.now().strftime('%m/%d/%Y-%H:%M:%S')
overhead_length = len(current_time+str(current))+13
console_width = shutil.get_terminal_size((80, 20)).columns - overhead_length
completed_width = round(console_width * completed_percentage / 100)
not_completed_width = console_width - completed_width
sys.stdout.write('\b\b\r')
sys.stdout.write('{}> [{}{}] {} - {}% '.format(current_time, '#'*completed_width, '-'*not_completed_width, current,
completed_percentage),)
sys.stdout.flush()
Usage of script:
import time
from pbar import update_progress_bar, print2
update_progress_bar(45,200)
time.sleep(1)
update_progress_bar(70,200)
time.sleep(1)
update_progress_bar(100,200)
time.sleep(1)
update_progress_bar(130,200)
time.sleep(1)
print2('some text that will re-place current progress bar')
time.sleep(1)
update_progress_bar(111,200)
time.sleep(1)
print('\n') # without \n next line will be attached to the end of the progress bar
print('built in print function that will push progress bar one line up')
time.sleep(1)
update_progress_bar(111,200)
time.sleep(1)
Better to overwrite the whole line otherwise the new line will mix with the old ones if the new line is shorter.
import time, os
for s in ['overwrite!', 'the!', 'whole!', 'line!']:
print(s.ljust(os.get_terminal_size().columns - 1), end="\r")
time.sleep(1)
Had to use columns - 1 on Windows.
This worked for me, using Python 3.7.9 within Spyder, in Windows:
from IPython.display import clear_output
from time import sleep
def print_and_overwrite(text):
'''Remember to add print() after the last print that you want to overwrite.'''
clear_output(wait=True)
print(text, end='\r')
for i in range(15):
#I print the results backwards (from 15 to 1), to test shorter strings
message = "Iteration %d out of 15" %(15-i)
print_and_overwrite(message)
sleep(0.5)
print() #This stops the overwriting
print("This will be on a new line")
Anyway if somebody wants to overprint (clear) a many lines previously printed in stdout, than this answer should be helpful for him. (Thanks Thijmen Dam for the nice article Overwrite Previously Printed Lines)
In ANSI console you can use special sequences:
\033[1A and \033[K
First of them lift up a cursor, second - erase a line entirely.
Example of the clearing a console (Python 3):
LINE_UP = '\033[1A'
LINE_CLEAR = '\033[K'
CONSOLE_HEIGHT = 24 #lines
def clear_console():
for a in range(CONSOLE_HEIGHT):
print(LINE_UP, end=LINE_CLEAR, flush=True)
or eventually simply (will clear screen and move cursor to 0,0):
print('\033[2J', end='', flush=True)
If you want just positioning cursor, then use this:
print('\033[<L>;<C>f', end='', flush=True)
where <L> and <C> are Line and Column correspondingly.
Handful reference for you ANSI escape sequences

python is there a way to delete from termimanl/console already printed result? [duplicate]

This question already has answers here:
Print in one line dynamically [duplicate]
(22 answers)
Closed 9 years ago.
The community reviewed whether to reopen this question 3 months ago and left it closed:
Original close reason(s) were not resolved
I was wondering if it was possible to remove items you have printed in Python - not from the Python GUI, but from the command prompt.
e.g.
a = 0
for x in range (0,3):
a = a + 1
b = ("Loading" + "." * a)
print (a)
so it prints
>>>Loading
>>>Loading.
>>>Loading..
>>>Loading...
But, my problem is I want this all on one line, and for it it remove it self when something else comes along. So instead of printing "Loading", "Loading.", "Loading... I want it to print "Loading.", then it removes what is on the line and replaces it with "Loading.." and then removes "Loading.." and replaces it (on the same line) with "Loading...". It's kind of hard to describe.
p.s I have tried to use the Backspace character but it doesn't seem to work ("\b")
Just use CR to go to beginning of the line.
import time
for x in range (0,5):
b = "Loading" + "." * x
print (b, end="\r")
time.sleep(1)
One way is to use ANSI escape sequences:
import sys
import time
for i in range(10):
print("Loading" + "." * i)
sys.stdout.write("\033[F") # Cursor up one line
time.sleep(1)
Also sometimes useful (for example if you print something shorter than before):
sys.stdout.write("\033[K") # Clear to the end of line
import sys
import time
a = 0
for x in range (0,3):
a = a + 1
b = ("Loading" + "." * a)
# \r prints a carriage return first, so `b` is printed on top of the previous line.
sys.stdout.write('\r'+b)
time.sleep(0.5)
print (a)
Note that you might have to run sys.stdout.flush() right after sys.stdout.write('\r'+b) depending on which console you are doing the printing to have the results printed when requested without any buffering.

Character Encoding Python 2.7.6 yQT Displaying chr(128)

I am running on a Windows system. Though I would like to be cross-platform. Right now I'll be happy to solve the question.
My test code is:
for n in range(32, 255):
if n%16 == 0:
xStr = xStr + '\n'
xStr = xStr + str(n) + ':' + chr(n) + '\t'
form.lstResponse.addItem(xStr) (This is a pyQT 4.8.5 QListWidget)
I am trying to use the symbol 'Arial' 128 which looks like a 'C" with two horizontal lines through it.
But, it appears that characters between 128 and 160 are missing?
How do I use the characters between 128 and 160?
All this Unicode stuff is quite baffling to me...
Thanks...
First, the
# -*- coding: utf-8 -*-
Has to be the first line. I have not tried all possibilities, But, moving it down the file kills it.
When the above code is included the following lines work as desired.
s = u'€'
xStr = QString(s)
form.lstResponse.addItem(xStr)
form.lstResponse.addItem(u'€')

python subprocess.call() not executing in loop

I have the following piece of code that generates plots with gnuplot:
import sys, glob, subprocess, os, time
for file in glob.glob('comb_*.out'):
fNameParts = file[5:].split('.')[0].split('-')
gPlotCmd = []
gPlotCmd = 'unset border; set xl "rotation period #"; set yl "T [K]"\n'
gPlotCmd += 'set key above\n'
gPlotCmd += 'set grid xtics ytics\n'
gPlotCmd += 'set term post eps enh color solid\n'
gPlotCmd += 'set xrange [20:876]\n'
gPlotCmd += 'set output "fig_TestRelax-' + fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3] + '-' + fNameParts[4] + '.eps"\n'
conf = fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3]
gPlotCmd += 'plot "' + file + '" using ($1/36000):($9-$3) w l lw 5 title "OP3-OP1 ' + conf + '", "' + file + '" using ($1/36000):($6-$3) w l lw 3 title "OP2-OP1 ' + conf + '", "' + file + '" using ($1/36000):($9-$6) w l lw 1 title "OP3-OP2 ' + conf + '"'
fw = open('temp.plt','w+')
fw.writelines(gPlotCmd)
subprocess.call(["gnuplot","temp.plt"])
print(file)
In the last loop execution the subprocess.call(["gnuplot","temp.plt"]) does not seem to be executed. At the end of the program, temp.plt exists with data from the last iteration. Also print(file) is executed during the last loop. Also if I plot the temp.plt left after the program I get the last plot (so there is no problem on the side of the data). Only the line subprocess.call(["gnuplot","temp.plt"]) is not executed. I also tried popen and monitor stdout and stderr but both were empty (as in all other iterations.
The checked the problem occurs both on linux and windows and in versions 3.3.5 and 2.7.3.
I conclude that there is something wrong with the script but I do not know what.
#lvc's and yours answer are correct; it is a buffering issue and fw.flush() should fix it. But you don't need the temporary file, you could pass the input commands to gnuplot directly without writing them to disk:
from subprocess import Popen, PIPE
p = Popen('gnuplot', stdin=PIPE)
p.communicate(input=gPlotCmd.encode('ascii'))
One likely error here is that the file temp.plt isn't actually written to disk at the time you run gnuplot. Python doesn't necessarily flush its buffers immediately after a call to writelines. This would mean that when gnuplot is launched from your script, it sees an empty file. It doesn't give an error, because an empty input isn't an error, and it has no way of knowing its expecting anything else. By the time you run it outside of your script, Python has exited and hence can't be holding anything in its own buffers anymore.
Use the with statement to ensure that fw is closed once you're done with it:
with open('temp.plt', 'w') as fw:
fw.writelines(gPlotCmd)
subprocess.call(["gnuplot","temp.plt"])
It seems I figured that out. I am missing fw.close(). The last lines of code should look:
fw = open('temp.plt','w+')
fw.writelines(gPlotCmd)
fw.close()
subprocess.call(["gnuplot","temp.plt"])
print(file)
Now the code produces the intended plots.

How can I overwrite/print over the current line in Windows command line?

On Unix, I can either use \r (carriage return) or \b (backspace) to overwrite the current line (print over text already visible) in the shell.
Can I achieve the same effect in a Windows command line from a Python script?
I tried the curses module but it doesn't seem to be available on Windows.
yes:
import sys
import time
def restart_line():
sys.stdout.write('\r')
sys.stdout.flush()
sys.stdout.write('some data')
sys.stdout.flush()
time.sleep(2) # wait 2 seconds...
restart_line()
sys.stdout.write('other different data')
sys.stdout.flush()
I know this is old, but i wanted to tell my version (it works on my PC in the cmd, but not in the idle) to override a line in Python 3:
>>> from time import sleep
>>> for i in range(400):
>>> print("\r" + str(i), end="")
>>> sleep(0.5)
EDIT:
It works on Windows and on Ubuntu
import sys
import time
for i in range(10):
print '\r', # print is Ok, and comma is needed.
time.sleep(0.3)
print i,
sys.stdout.flush() # flush is needed.
And if on the IPython-notebook, just like this:
import time
from IPython.display import clear_output
for i in range(10):
time.sleep(0.25)
print(i)
clear_output(wait=True)
http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/notebooks/Animations%20Using%20clear_output.ipynb
I just had this problem. You can still use \r, even in Windows Command Prompt, however, it only takes you back to the previous linebreak (\n).
If you do something like this:
cnt = 0
print str(cnt)
while True:
cnt += 1
print "\r" + str(cnt)
You'll get:
0
1
2
3
4
5
...
That's because \r only goes back to the last line. Since you already wrote a newline character with the last print statement, your cursor goes from the beginning of a new empty line to the beginning of the same new empty line.
To illustrate, after you print the first 0, your cursor would be here:
0
| # <-- Cursor
When you \r, you go to the beginning of the line. But you're already on the beginning of the line.
The fix is to avoid printing a \n character, so your cursor is on the same line and \r overwrites the text properly. You can do that with print 'text',. The comma prevents the printing of a newline character.
cnt = 0
print str(cnt),
while True:
cnt += 1
print "\r" + str(cnt),
Now it will properly rewrite lines.
Note that this is Python 2.7, hence the print statements.
Easy method:
import sys
from time import sleep
import os
#print("\033[y coordinate;[x coordinateH Hello")
os.system('cls')
sleep(0.2)
print("\033[1;1H[]")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H []")
sleep(0.2)
print("\033[1;1H[]")
sleep(0.2)
Simple way if you're just wanting to update the previous line:
import time
for i in range(20):
print str(i) + '\r',
time.sleep(1)
Easiest way is to use two \r - one at the beginning and one at the end
for i in range(10000):
print('\r'+str(round(i*100/10000))+'% Complete\r'),
It will go pretty quickly
On Windows (python 3), it seems to work (not using stdout directly):
import sys
for i in reversed(range(0,20)):
time.sleep(0.1)
if(i == 19):
print(str(i), end='', file=sys.stdout)
else:
print("\r{0:{width}".format(str(i), width = w, fill = ' ', align = 'right'), end='', file=sys.stdout)
sys.stdout.flush()
w = len(str(i))
The same line is updated everytime print function is called.
This algorithm can be improved, but it is posted to show what you can do. You can modify the method according to your needs.
Thanks for all the useful answers in here guys. I needed this :)
I found nosklo's answer particularly useful, but I wanted something fully contained within a function by passing the desired output as a parameter. Also, I didn't really need the timer, since I wanted the printing to take place after a specific event).
This is what did the trick for me, I hope someone else finds it useful:
import sys
def replace_cmd_line(output):
"""Replace the last command line output with the given output."""
sys.stdout.write(output)
sys.stdout.flush()
sys.stdout.write('\r')
sys.stdout.flush()
Yes, this question was asked 11 years ago but it's cool. I like to improvise.
Add-on to nosklo's answer:
import sys
import time
def restart_line():
sys.stdout.write("\r")
sys.stdout.flush()
string_one = "some data that is very long..."
sys.stdout.write(string_one)
sys.stdout.flush()
time.sleep(2)
restart_line()
string_two = "shorter data"
if len(string_two) < len(string_one):
string_two = string_two+(" "*int((len(string_one)-len(string_two))))
# This will overwrite the characters that would be left on the console
sys.stdout.write(string_two)
sys.stdout.flush()
Tested on PyCharm 2020.3 and Python version 3.9, to overwrite written printout I use the following:
from time import sleep
for x in range(10):
print(f'\r {x}', end='')
sleep(0.6)
That's the code I mainly use for my programs. Using end='\r' will overwrite the whole text for me, ignoring sleep.
In real scenario, I set it up as follows:
def progress_callback(progress):
print(f'\rDownloading File: {progress.dlable.file_name} Progress: ' + '{0:.2f}%'.format(progress.percent), end='')
# `return True` if the download should be canceled
return False
print('\nDownload complete!)
The print after the overwrite function has to be in new line, or the same line before will be overwritten.

Categories