We Keep Coding

Why is sys.stdout.write called twice?

I'm trying to log all my prints to a log file (and add a timestamp before the actual message, only in the log file), but I cannot figure out why sys.stdout.write is called twice.
import sys
from datetime import datetime
class Logger(object):
def __init__(self, filename="Default.log"):
self.stdout = sys.stdout
self.log = open(filename, "w")
def write(self, message):
self.stdout.write(message)
time_stamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')
self.log.write(f"{time_stamp} - {message}")
self.log.flush()
def flush(self):
pass
sys.stdout = Logger("log_file.txt")
print("Hello world !")
Output from terminal:
Hello World !
Output in log_file.txt:
2021-04-19 18:43:14.800691 - Hello world !2021-04-19 18:43:14.800735 -
What am I missing here? The write method is called again after the self.log.flush() has been called, but here message=''.
If I neglect the time_stamp variable it works like a charm, e.g calling self.log.write(message).
I could of course just check if message is empty, but I would really like to understand my problem here!
My solution
#iBug gave me the answer! I didn't know that print did work this way: one write call for the data and one write call for the end keyword which is \n as default.
My solution was to add a variable self._hidden_end to mthe Logger class that is initiated to 0 and then toggles everytime write is called to check it I should add a time stampe before writing to the log or not.
import sys
from datetime import datetime
class Logger(object):
def __init__(self, filename="Default.log"):
self.stdout = sys.stdout
self.log = open(filename, "w")
self._hidden_end = 0
def write(self, message):
self.stdout.write(message)
if not self._hidden_end:
time_stamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')
message = f"{time_stamp} - {message}"
self._hidden_end ^= 1
self.log.write(message)
self.log.flush()
def flush(self):
pass
sys.stdout = Logger("log_file.txt")
print("Hello world !")
Any thoughts on my solution above are welcome! :)

Your perceived results indicates that print() calls file.write twice: Once for the data and once for the "end content", which is a newline by default and can be overridden with the end keyword argument. Relevant source code lines confirm this speculation.
However, this is not something you should rely on. It's entirely implementation details, may be different for another implementation (e.g. PyPy) and may change at any time without notice, not to mention that overriding builtins is another bad practice. You should avoid this kind of practice without a compelling reason not to modify other parts of the code to use your custom logging facility.
Should you really need to do monkey-patching, it's safer to override the print() function since it has a defined, reliable interface. You can import the builtins module and assign your new print function to builtins.print, maintaining control over calls to your logging handler.

How to escape from the standard output redirection in Python

I want to process a live output of a thirdparty script, printing some lines that match a pattern but skipping others:
def thirdparty_code():
from random import choice
stuff = ['keep: important stuff', 'ignore: boring stuff']
while True:
chosen_line = choice(stuff)
print(chosen_line)
I use redirect_stdout (which passes lines to my dummy IO) and an extended StringIO (which serves as the IO but also calls my filtering function). However, when I call print() inside of my processing function I get a RecursionError - which is not unexpected:
from io import StringIO
from contextlib import redirect_stdout
class StringIOWithCallback(StringIO):
def __init__(self, callback, **kwargs):
super().__init__(**kwargs)
self.callback = callback
def write(self, s):
super().write(s)
self.callback(s)
def filter_boring_stuff_out(line):
if line.startswith('ignore'):
return
print(line)
my_io = StringIOWithCallback(filter_boring_stuff_out)
with redirect_stdout(my_io):
thirdparty_code()
I wonder if it is possible to escape from the redirect, e.g. specifying the file parameter in print() function so that it prints to the actual standard output. I know that I can easily use the standard error stream:
import sys
print(line, file=sys.stderr)
But I specifically want to use the standard output. Is there a nice, pythonic way to do that?

After having redirected the stdout, you can easily reset it thanks to __stdout__ which saves the original value.
sys.stdout = redirected_stdout
...
...
sys.stdout = sys.__stdout__
If you continually find yourself switching these output streams, you ought to create a function to output to the redirected stream:
def redirect(*args):
print(*args, file=redirected_file)
redirect(...) # redirect this output
print(...) # use standard stream

Once I have written my question I have realised that one simply needs to save the standard output object sys.stdout to a variable before invoking the redirect:
stdout = sys.stdout
def filter_boring_stuff_out(line):
if line.startswith('ignore'):
return
print(line, file=stdout)
But as always - I would be happy to learn about other possible solutions.

You can redirect stdout and ignore all messages that started with ignore. If you do it this way all prints will be intercepted. This will work better if you are trying to filter messages from code you don't have access to or don't want to change.
import sys
from contextlib import redirect_stdout
class Filter_Out:
def __init__(self, *_, start=None, anywhere=None, end=None):
self.last_ignore = False
self.start = start
self.anywhere = anywhere
self.end = end
self.terminal = sys.stdout
def write(self, txt):
if (self.start and txt.startswith(self.start)) or \
(self.end and txt.endswith(self.end)) or \
(self.anywhere and self.anywhere in txt):
self.last_ignore = True
return
if self.last_ignore and txt == '\n':
self.last_ignore = False
else:
self.terminal.write(txt)
def flush(self):
pass
with redirect_stdout(Filter_Out(start='ignore', anywhere='4')):
print("test")
print("test2")
print("ignore: test2") # will not print because it started with ignore
print("test1")
print("test42") # will not print because it had 4 in it

Supress calls to print that come from an import [duplicate]

Is there a way to stop a function from calling print?
I am using the pygame.joystick module for a game I am working on.
I created a pygame.joystick.Joystick object and in the actual loop of the game call its member function get_button to check for user input. The function does everything I need it to do, but the problem is that it also calls print, which slows down the game considerably.
Can I block this call to print?

Python lets you overwrite standard output (stdout) with any file object. This should work cross platform and write to the null device.
import sys, os
# Disable
def blockPrint():
sys.stdout = open(os.devnull, 'w')
# Restore
def enablePrint():
sys.stdout = sys.__stdout__
print 'This will print'
blockPrint()
print "This won't"
enablePrint()
print "This will too"
If you don't want that one function to print, call blockPrint() before it, and enablePrint() when you want it to continue. If you want to disable all printing, start blocking at the top of the file.

Use with
Based on #FakeRainBrigand solution I'm suggesting a safer solution:
import os, sys
class HiddenPrints:
def __enter__(self):
self._original_stdout = sys.stdout
sys.stdout = open(os.devnull, 'w')
def __exit__(self, exc_type, exc_val, exc_tb):
sys.stdout.close()
sys.stdout = self._original_stdout
Then you can use it like this:
with HiddenPrints():
print("This will not be printed")
print("This will be printed as before")
This is much safer because you can not forget to re-enable stdout, which is especially critical when handling exceptions.
Without with — Bad practice
The following example uses enable/disable prints functions that were suggested in previous answer.
Imagine that there is a code that may raise an exception. We had to use finally statement in order to enable prints in any case.
try:
disable_prints()
something_throwing()
enable_prints() # This will not help in case of exception
except ValueError as err:
handle_error(err)
finally:
enable_prints() # That's where it needs to go.
If you forgot the finally clause, none of your print calls would print anything anymore.
It is safer to use the with statement, which makes sure that prints will be reenabled.
Note: It is not safe to use sys.stdout = None, because someone could call methods like sys.stdout.write()

As #Alexander Chzhen suggested, using a context manager would be safer than calling a pair of state-changing functions.
However, you don't need to reimplement the context manager - it's already in the standard library. You can redirect stdout (the file object that print uses) with contextlib.redirect_stdout, and also stderr with contextlib.redirect_stderr.
import os
import contextlib
with open(os.devnull, "w") as f, contextlib.redirect_stdout(f):
print("This won't be printed.")

If you want to block print calls made by a particular function, there is a neater solution using decorators. Define the following decorator:
# decorater used to block function printing to the console
def blockPrinting(func):
def func_wrapper(*args, **kwargs):
# block all printing to the console
sys.stdout = open(os.devnull, 'w')
# call the method in question
value = func(*args, **kwargs)
# enable all printing to the console
sys.stdout = sys.__stdout__
# pass the return value of the method back
return value
return func_wrapper
Then just place #blockPrinting before any function. For example:
# This will print
def helloWorld():
print("Hello World!")
helloWorld()
# This will not print
#blockPrinting
def helloWorld2():
print("Hello World!")
helloWorld2()

If you are using Jupyter Notebook or Colab use this:
from IPython.utils import io
with io.capture_output() as captured:
print("I will not be printed.")

I have had the same problem, and I did not come to another solution but to redirect the output of the program (I don't know exactly whether the spamming happens on stdout or stderr) to /dev/null nirvana.
Indeed, it's open source, but I wasn't passionate enough to dive into the pygame sources - and the build process - to somehow stop the debug spam.
EDIT :
The pygame.joystick module has calls to printf in all functions that return the actual values to Python:
printf("SDL_JoystickGetButton value:%d:\n", value);
Unfortunately you would need to comment these out and recompile the whole thing. Maybe the provided setup.py would make this easier than I thought. You could try this...

A completely different approach would be redirecting at the command line. If you're on Windows, this means a batch script. On Linux, bash.
/full/path/to/my/game/game.py > /dev/null
C:\Full\Path\To\My\Game.exe > nul
Unless you're dealing with multiple processes, this should work. For Windows users this could be the shortcuts you're creating (start menu / desktop).

You can do a simple redirection, this seems a lot safer than messing with stdout, and doesn't pull in any additional libraries.
enable_print = print
disable_print = lambda *x, **y: None
print = disable_print
function_that_has_print_in_it(1) # nothing is printed
print = enable_print
function_that_has_print_in_it(2) # printing works again!
Note: this only works to disable the print() function, and would not disable all output if you're making calls to something else that is producing output. For instance if you were calling a C library that was producing it's own output to stdout, or if you were using intput().

No, there is not, especially that majority of PyGame is written in C.
But if this function calls print, then it's PyGame bug, and you should just report it.

The module I used printed to stderr. So the solution in that case would be:
sys.stdout = open(os.devnull, 'w')

"stop a function from calling print"
# import builtins
# import __builtin__ # python2, not test
printenabled = False
def decorator(func):
def new_func(*args,**kwargs):
if printenabled:
func("print:",*args,**kwargs)
return new_func
print = decorator(print) # current file
# builtins.print = decorator(builtins.print) # all files
# __builtin__.print = decorator(__builtin__.print) # python2
import sys
import xxxxx
def main():
global printenabled
printenabled = True
print("1 True");
printenabled = False
print("2 False");
printenabled = True
print("3 True");
printenabled = False
print("4 False");
if __name__ == '__main__':
sys.exit(main())
#output
print: 1 True
print: 3 True
https://stackoverflow.com/a/27622201

Change value of file object of print() function. By default it's sys.stdout, instead we can write to null device by open(os.devnull, 'w')
import os, sys
mode = 'debug' #'prod'
if mode == 'debug':
fileobj = sys.stdout
else:
fileobj = open(os.devnull,'w')
print('Hello Stackoverflow', file = fileobj)

Based on #Alexander Chzhen solution, I present here the way to apply it on a function with an option to suppress printing or not.
import os, sys
class SuppressPrints:
#different from Alexander`s answer
def __init__(self, suppress=True):
self.suppress = suppress
def __enter__(self):
if self.suppress:
self._original_stdout = sys.stdout
sys.stdout = open(os.devnull, 'w')
def __exit__(self, exc_type, exc_val, exc_tb):
if self.suppress:
sys.stdout.close()
sys.stdout = self._original_stdout
#implementation
def foo(suppress=True):
with SuppressPrints(suppress):
print("It will be printed, or not")
foo(True) #it will not be printed
foo(False) #it will be printed
I hope I can add my solution below answer of Alexander as a comment, but I don`t have enough (50) reputations to do so.

If you want to enable/disable print with a variable, you could call an auxiliary function instead print, something like printe(the name is just for convenience)
def printe(*what_to_print):
if prints_enable:
string = ""
for items in what_to_print:
string += str(items) + " "
print(string)

Define a new Print function where you enable print first. print your output next. And then disable print again.
def Print (*output):
enablePrint()
print (output)
disablePrint()
with one of the above "safe" enable / disable pair of function

Python: Write unittest for console print

Function foo prints to console. I want to test the console print. How can I achieve this in python?
Need to test this function, has NO return statement :
def foo(inStr):
print "hi"+inStr
My test :
def test_foo():
cmdProcess = subprocess.Popen(foo("test"), stdout=subprocess.PIPE)
cmdOut = cmdProcess.communicate()[0]
self.assertEquals("hitest", cmdOut)

You can easily capture standard output by just temporarily redirecting sys.stdout to a StringIO object, as follows:
import StringIO
import sys
def foo(inStr):
print "hi"+inStr
def test_foo():
capturedOutput = StringIO.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call unchanged function.
sys.stdout = sys.__stdout__ # Reset redirect.
print 'Captured', capturedOutput.getvalue() # Now works as before.
test_foo()
The output of this program is:
Captured hitest
showing that the redirection successfully captured the output and that you were able to restore the output stream to what it was before you began the capture.
Note that the code above in for Python 2.7, as the question indicates. Python 3 is slightly different:
import io
import sys
def foo(inStr):
print ("hi"+inStr)
def test_foo():
capturedOutput = io.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call function.
sys.stdout = sys.__stdout__ # Reset redirect.
print ('Captured', capturedOutput.getvalue()) # Now works as before.
test_foo()

This Python 3 answer uses unittest.mock. It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
import io
import unittest
import unittest.mock
from .solution import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
#unittest.mock.patch('sys.stdout', new_callable=io.StringIO)
def assert_stdout(self, n, expected_output, mock_stdout):
fizzbuzz(n)
self.assertEqual(mock_stdout.getvalue(), expected_output)
def test_only_numbers(self):
self.assert_stdout(2, '1\n2\n')
Note that the mock_stdout arg is passed automatically by the unittest.mock.patch decorator to the assert_stdout method.
A general-purpose TestStdout class, possibly a mixin, can in principle be derived from the above.
For those using Python ≥3.4, contextlib.redirect_stdout also exists, but it seems to serve no benefit over unittest.mock.patch.

If you happen to use pytest, it has builtin output capturing. Example (pytest-style tests):
def eggs():
print('eggs')
def test_spam(capsys):
eggs()
captured = capsys.readouterr()
assert captured.out == 'eggs\n'
You can also use it with unittest test classes, although you need to passthrough the fixture object into the test class, for example via an autouse fixture:
import unittest
import pytest
class TestSpam(unittest.TestCase):
#pytest.fixture(autouse=True)
def _pass_fixtures(self, capsys):
self.capsys = capsys
def test_eggs(self):
eggs()
captured = self.capsys.readouterr()
self.assertEqual('eggs\n', captured.out)
Check out Accessing captured output from a test function for more info.

You can also use the mock package as shown below, which is an example from
https://realpython.com/lessons/mocking-print-unit-tests.
from mock import patch
def greet(name):
print('Hello ', name)
#patch('builtins.print')
def test_greet(mock_print):
# The actual test
greet('John')
mock_print.assert_called_with('Hello ', 'John')
greet('Eric')
mock_print.assert_called_with('Hello ', 'Eric')

The answer of #Acumenus says:
It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
the bold part seems a big drawback, thus I would do the following instead:
# extend unittest.TestCase with new functionality
class TestCase(unittest.TestCase):
def assertStdout(self, expected_output):
return _AssertStdoutContext(self, expected_output)
# as a bonus, this syntactical sugar becomes possible:
def assertPrints(self, *expected_output):
expected_output = "\n".join(expected_output) + "\n"
return _AssertStdoutContext(self, expected_output)
class _AssertStdoutContext:
def __init__(self, testcase, expected):
self.testcase = testcase
self.expected = expected
self.captured = io.StringIO()
def __enter__(self):
sys.stdout = self.captured
return self
def __exit__(self, exc_type, exc_value, tb):
sys.stdout = sys.__stdout__
captured = self.captured.getvalue()
self.testcase.assertEqual(captured, self.expected)
this allows for the much nicer and much more re-usable:
# in a specific test case, the new method(s) can be used
class TestPrint(TestCase):
def test_print1(self):
with self.assertStdout("test\n"):
print("test")
by using a straight forward context manager. (It might also be desirable to append "\n" to expected_output since print() adds a newline by default. See next example...)
Furthermore, this very nice variant (for an arbitrary number of prints!)
def test_print2(self):
with self.assertPrints("test1", "test2"):
print("test1")
print("test2")
is possible now.

You can also capture the standard output of a method using contextlib.redirect_stdout:
import unittest
from contextlib import redirect_stdout
from io import StringIO
class TestMyStuff(unittest.TestCase):
# ...
def test_stdout(self):
with redirect_stdout(StringIO()) as sout:
my_command_that_prints_to_stdout()
# the stream replacing `stdout` is available outside the `with`
# you may wish to strip the trailing newline
retval = sout.getvalue().rstrip('\n')
# test the string captured from `stdout`
self.assertEqual(retval, "whatever_retval_should_be")
Gives you a locally scoped solution. It is also possible to capture the standard error using contextlib.redirect_stderr().

Another variant is leaning on the logging module rather than print(). This module also has a suggestion of when to use print in the documentation:
Display console output for ordinary usage of a command line script or program
PyTest has built-in support for testing logging messages.

Unable to restore stdout to original (only to terminal)

I need to output all my print statements to both terminal and file.
I've found a solution in this stackoverflow question
So i use
class Tee(object):
def __init__(self, name):
self.file = open(name, "a")
self.stdout = sys.stdout
sys.stdout = self
def __del__(self):
sys.stdout = self.stdout
self.file.close()
def write(self, data):
self.file.write(data)
self.stdout.write(data)
sys.stdout = Tee("log.txt")
That's works great ,
My problem is when i want to stop writing to file and go back to normal operation that's print only to terminal
I tried to use del(sys.stdout) to invoke the del method with no luck.
i even tried to change the last line to:
multiple_print = Tee("log.txt")
sys.stdout = multiple_print
and than use del(multiple_print) and failed
No matter what i tried , further uses of print still written to terminal and file both.
Any ideas?

You need to save a reference to the original file-descriptor before reassignment:
oldstdout = sys.stdout
and afterwards reassign it to sys.stdout!
The del statement doesn't call __del__ directly, but rather decreases the reference counter of your object. the __del__ method is called if the reference counter reaches zero and your object is about to get destroyed. So you can't force the reassignment, because it's up to the garbage collector, to call your __del__ method.
You need to directly reassign the sys.stdout variable:
Try:
sys.stdout = sys.__stdout__