I've been trying to print out the progress of a for loop in python2.7 using the following code:
for i in range(100):
if float(i) % 10.0 == 0:
print i, "\r",
The behaviour I'm after is the refreshing of the same line on std out rather than writing to a new line every time.
EDIT 1:
Testing in my console (Xfce Terminal 0.4.8), I actually don't get any output regardless of whether I include the if statement or not.
Why is there no output?
I originally said the behaviour of the stdout changed depending on the if statement being there or not because I simplified the code that produced the problem to its most simple form (only to produce the above mentioned effect). My apologies.
EDIT 2:
Thanks to senderle, this is solved. If you miss out the sleep() command, the prints and carriage return happen so quickly you can't see them.
EDIT 3:
One last thing. If you don't catch for the final number in range(100), i.e. 99, the number is cleared off the screen.
EDIT 4:
Note the comma after print i in senderle's answer.
I have found that using sys.stdout is a more system-independent way of doing this, for varions reasons having to do with the way print works. But you have to flush the buffer explicitly, so I put it in a function.
def carriage_return():
sys.stdout.write('\r')
sys.stdout.flush()
This is kind of a WAG. Let me know if it helps.
I tried this and it works for me. The time.sleep is just for dramatization.
import sys, time
def carriage_return():
sys.stdout.write('\r')
sys.stdout.flush()
for i in range(100):
if i % 10 == 0:
print i,
carriage_return()
time.sleep(1)
Finally, I have seen people do this as well. Using terminal control codes like this seems right in some ways, but it also seems more brittle to me. This works for me with the above code as well (on OS X).
def carriage_return():
if sys.platform.lower().startswith('win'):
print '\r'
else:
print chr(27) + '[A'
Testing your code as is, and just including a :colon: at the end of the first line, works just fine with Py2.7 32bit, Windows7 64-bit.
Do you have any out writes to stdout in your if or for block that could be causing the new-lines to be written out ?
Related
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
So i am currently trying to make something that will print . and remove it then print ..
and also remove it then print ... When i tried using sys module to remove the prevous text this was the output: lol [Ktest so it basically adds a [K to the next line.
I also tried using another method so instead of removing the prevous text it would just add onto it like:
import time
print("lol",end="")
time.sleep(1)
print("test")
it did work in IDLE but when i tried to use it by opening the file in the command promt it waited for 1 second and then just gave loltest without any delay between them. So nothing i found in the internet worked for me.
You may print with the keyword argument end to append the special character '\r' to the end of the line.
E.g.
import time
print(".", end='\r')
time.sleep(2)
print("..", end='\r')
time.sleep(2)
print("...", end='\r')
time.sleep(2)
'\r' is carriage return and will return to the start of the line in some terminals, from where you can overwrite the text you just printed. Note that the behaviour might differ between terminals though.
To print over the prvious print, you can use end="\r.
import time
print("lol", end="\r")
time.sleep(1)
print("test")
for i in range(4):
print("."*i, end="\r")
time.sleep(1)
You can use the os module to execute shell commands.
To clear the terminal, command required in windows is cls and for unix its clear
import os
os.system('cls' if os.name == 'nt' else 'clear')
If you don't want to clear previous terminal outputs you can use flexibility of print function or the carriage return as others denoted.
for _ in range(3):
print('.', end='')
time.sleep(1)
If you specifically want to print . then .. then ..., you don't need to remove the existing text; you can just print additional dots.
To make the dots actually appear one by one, you'll need to flush the buffers, using flush=True
import time
for _ in range(3):
print('.', end='', flush=True)
time.sleep(1)
print()
This has the advantage that it will work much more generally; almost any output method can do that, whereas ANSI codes or tricks with \r or clearing the screen depend on your hardware, operating system and various other things.
You can do it with ANSI escape codes, like this:
import sys, time
clear_line = '\x1b[1K\r'
print("lol", end="")
sys.stdout.flush() # to force printing the text above
time.sleep(1)
print(clear_line+"test") # Now lol replaced with test
Please note that ANSI codes you should use depend on the environment where the program is executing (platform, terminal, etc.).
Update: you may want to see the built-in curses module.
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
I'm using a script that runs for many hours, which prints statements to verify whether or not issues might have arisen (data is downloaded from the web, which sometimes gets distorted).
I've noticed a significant drop in performance after a while. I suspect that the many thousands of lines of print statements might be the reason.
It is commonly known that the terminal can be cleared of these print statements by the following line of code:
import os
os.system('cls') # for windows
Still, I suspect that this doesn't actually improve the performance speed and that it's merely a perceived improvement due to the fact that the screen is cleared. Is that true or not?
I've also considered suppressing certain print statements with the following code:
import sys
class NullWriter(object):
def write(self, arg):
pass
nullwrite = NullWriter()
oldstdout = sys.stdout
sys.stdout = oldstdout # enable output
print("text that I want to see")
sys.stdout = nullwrite # disable output
print("text I don't want to see")
My question: How can I improve the performance (speed) of my script, given that I still want to see the most recent print statements?
If you like you can just do a line feed without a carriage return and override the last line:
sys.stdout.write("\rDoing things")
sys.stdout.flush()
Printing over time shouldn't use any extra memory within python, but you might have your terminal's buffer set to high which can use a lot of memory. Or it's just taking time to flush the buffer because you're writing so fast to stdout.
You can also use the print function
Python 2.6+
From Python 2.6 you can import the print function from Python 3:
from __future__ import print_function
This allows you to use the Python 3 solution below.
Python 3
In Python 3, the print statement has been changed into a function. In Python 3, you can instead do:
print('.', end='')
I am new to Python. I am using Netbeans IDE 6.8. When I run the code below- using RUN FILE- it does not seem to produce any output. On the other hand when I debug the code, the output shows the value of counter- 6.
Is this a problem with the program below or one of quirks of Netbeans.
Here is the code:
class Counter:
pass
def cInit():
# Create counter
ctr = Counter()
ctr.value = 0
# Define and call a recursive function that modifies counter
def inner(n):
if (n > 0): inner(n-1)
ctr.value = ctr.value + 1
inner(5)
# Get counter
return ctr.value
if __name__ == "__main__":
print "Hello World";
d = cInit()
print d
This is a classic "bug" of netbeans and other IDEs. For terminal programs, they open a terminal, run the program under it, and then close it. This of course, means that your output window disappears.
There are two ways to fix it, depending on your IDE. Some IDEs have an option to wait for a key press after program completion, it'll be buried in your options panel somewhere. The other is to put a raw_input() command at the end of your code, so that the terminal pauses and waits for user input before closing. That may get very annoying for your end users if they run the thing on the command line, since they may not want it to pause in the middle of a pipeline.
There is nothing wrong with your code, it works fine when run under the Python REPL. This may be a Netbeans quirk -- does print work in other files?
As a side note -- if this is something to do with Netbeans, don't expect any official fix anytime soon -- Oracle killed Python support in Netbeans 7