We Keep Coding

External library uses print instead of return

I have a problem with an external library that I use in my script.
I execute a function from this library, but the function prints the output directly. However, I need the output to check if there is a specific string in it.
How can I ensure that the output of the function in the external library comes in a variable so that I can make the comparisons with the string?

If you really have no other choice, you could redirect stdout when you call the library. Here's a piece of code adapted from this answer :
def some_library():
print("Should probably return instead of print.")
import sys
from io import StringIO
class redirected_stdout:
def __init__(self):
self._stdout = None
self._string_io = None
def __enter__(self):
self._stdout = sys.stdout
sys.stdout = self._string_io = StringIO()
return self
def __exit__(self, type, value, traceback):
sys.stdout = self._stdout
#property
def string(self):
return self._string_io.getvalue()
with redirected_stdout() as out:
some_library()
result = out.string
print("return" in result)
# True
It would be much cleaner to modify the library, though.

You can exchange sys.stdout with your buffer temporarily, and then check the buffer.
def external_method():
print ("print something out, don't return")
class MyBuffer(object):
def __init__(self):
self.buffer = []
def write(self, *args, **kwargs):
self.buffer.append(args)
import sys
old_stdout = sys.stdout
sys.stdout = MyBuffer()
external_method()
my_buffer, sys.stdout = sys.stdout, old_stdout
print (my_buffer.buffer)

Decorator to modify output printed by a function in python

Suppose I have a function that prints five lines of text. I want to add prefix like 'asdf ' and suffix ' qwerty' for every line it prints on console. How to do this with decorator in python. Output generated can come from logging module or print statements in function that we want to decorate.

def my_decorator(func):
def wrapping_func(strlist):
for index, i in enumerate(strlist):
strlist[index] = 'wubalubadubdub' + i
return func(strlist)
return wrapping_func
#my_decorator
def fun(str_list):
for i in str_list:
print(i)
if __name__ == "__main__":
fun(['a', 'b'])
Duplicate question but anyways the above code is what you are looking for, the wrapping_func merely modifies the arguments that are given to the function i.e. adds a prefix and returns while calling the original function with the modified arguments with the my_decorator function just returning the wrapping_func.

Here is one sample code to demonstrate this problem.
Print statements output is customized, but not statements from logging module.
from contextlib import contextmanager
#contextmanager
def no_stdout():
import sys
old_stdout = sys.stdout
class CustomPrint():
def __init__(self, stdout):
self.old_stdout = stdout
def write(self, text):
if len(text.rstrip()):
self.old_stdout.write('custom Print--->'+ text)
sys.stdout = CustomPrint(old_stdout)
try:
yield
finally:
sys.stdout = old_stdout
def fun():
import logging
logger = logging.getLogger("tester")
logger.info("Test line from function")
print "BEFORE"
with no_stdout():
print "WHY HELLO!\n"
print "DING DONG!\n"
import logging
logger = logging.getLogger("test")
logger.info(" Hello world")
fun()
print "AFTER"
output:
BEFORE
custom Print--->WHY HELLO!
custom Print--->DING DONG!
2018-06-11 15:52:30,088 (42092) test INFO - Hello world
2018-06-11 15:52:30,092 (42092) tester INFO - Test line from function
AFTER
we see that logging modules output is not customized.

How to put all print result in a function into a variable?

I have a function in Python:
def f():
...
a lot of code
...
print "hello"
...
a lot of code
...
I want to call this function, however, the print result will be put into a variable instead print on the screen directly. How can I do this with Python?
ps:
please don't just return, sometimes I don't know where the print statement is.

Assuming that print is writing to sys.stdout, you can temporarily replace that with something like a StringIO object.
stdout = sys.stdout
sys.stdout = StringIO()
f()
x = sys.stdout.getvalue()
sys.stdout = stdout
Or, if you have a reference to the file handle that print is using, you can use that instead of sys.stdout.
If there are multiple uses of print from inside f, and you only want to capture some of them (say, only from a function g called from inside f), I'm afraid you are out of luck. The amount of introspection you would need to do would make it possible would allow you to simply re-implement the function to accumulate the desired output in a variable instead of using print.

Use a decorator like below
import sys
from StringIO import StringIO
s = StringIO()
def catch_stdout(user_method):
sys.stdout = s
def decorated(*args, **kwargs):
user_method(*args, **kwargs)
sys.stdout = sys.__stdout__
print 'printing result of all prints in one go'
s.seek(0, 0)
print s.read()
return decorated
#catch_stdout
def test():
print 'hello '
print 'world '
test()

You could also define your own context manager if you find you need to do this a lot so you can capture the output for a block of statements, eg:
import contextlib
from StringIO import StringIO
import sys
#contextlib.contextmanager
def capture_stdout():
old_stdout = sys.stdout
sys.stdout = StringIO()
yield sys.stdout, old_stdout
sys.stdout = old_stdout
Then use as follows:
def something():
print 'this is something'
# All prints that go to stdout inside this block either called
# directly or indirectly will be put into a StringIO object instead
# unless the original stdout is used directly...
with capture_print() as (res, stdout):
print 'hello',
print >> stdout, "I'm the original stdout!"
something()
print res.getvalue() + 'blah' # normal print to stdout outside with block
Gives you:
I'm the original stdout
hello this is something
blah

def f():
#code
variable = 'hello\n'
#code
variable += 'hello2\n'
#code
...
print(variable)
or
def f():
#code
variable = 'hello\n'
#code
variable += 'hello2\n'
#code
...
return(variable)
and then
print(f())

How to capture stdout output from a Python function call?

I'm using a Python library that does something to an object
do_something(my_object)
and changes it. While doing so, it prints some statistics to stdout, and I'd like to get a grip on this information. The proper solution would be to change do_something() to return the relevant information,
out = do_something(my_object)
but it will be a while before the devs of do_something() get to this issue. As a workaround, I thought about parsing whatever do_something() writes to stdout.
How can I capture stdout output between two points in the code, e.g.,
start_capturing()
do_something(my_object)
out = end_capturing()
?

Try this context manager:
from io import StringIO
import sys
class Capturing(list):
def __enter__(self):
self._stdout = sys.stdout
sys.stdout = self._stringio = StringIO()
return self
def __exit__(self, *args):
self.extend(self._stringio.getvalue().splitlines())
del self._stringio # free up some memory
sys.stdout = self._stdout
Usage:
with Capturing() as output:
do_something(my_object)
output is now a list containing the lines printed by the function call.
Advanced usage:
What may not be obvious is that this can be done more than once and the results concatenated:
with Capturing() as output:
print('hello world')
print('displays on screen')
with Capturing(output) as output: # note the constructor argument
print('hello world2')
print('done')
print('output:', output)
Output:
displays on screen
done
output: ['hello world', 'hello world2']
Update: They added redirect_stdout() to contextlib in Python 3.4 (along with redirect_stderr()). So you could use io.StringIO with that to achieve a similar result (though Capturing being a list as well as a context manager is arguably more convenient).

In python >= 3.4, contextlib contains a redirect_stdout decorator. It can be used to answer your question like so:
import io
from contextlib import redirect_stdout
f = io.StringIO()
with redirect_stdout(f):
do_something(my_object)
out = f.getvalue()
From the docs:
Context manager for temporarily redirecting sys.stdout to another file
or file-like object.
This tool adds flexibility to existing functions or classes whose
output is hardwired to stdout.
For example, the output of help() normally is sent to sys.stdout. You
can capture that output in a string by redirecting the output to an
io.StringIO object:
f = io.StringIO()
with redirect_stdout(f):
help(pow)
s = f.getvalue()
To send the output of help() to a file on disk, redirect the output to
a regular file:
with open('help.txt', 'w') as f:
with redirect_stdout(f):
help(pow)
To send the output of help() to sys.stderr:
with redirect_stdout(sys.stderr):
help(pow)
Note that the global side effect on sys.stdout means that this context
manager is not suitable for use in library code and most threaded
applications. It also has no effect on the output of subprocesses.
However, it is still a useful approach for many utility scripts.
This context manager is reentrant.

Here is an async solution using file pipes.
import threading
import sys
import os
class Capturing():
def __init__(self):
self._stdout = None
self._stderr = None
self._r = None
self._w = None
self._thread = None
self._on_readline_cb = None
def _handler(self):
while not self._w.closed:
try:
while True:
line = self._r.readline()
if len(line) == 0: break
if self._on_readline_cb: self._on_readline_cb(line)
except:
break
def print(self, s, end=""):
print(s, file=self._stdout, end=end)
def on_readline(self, callback):
self._on_readline_cb = callback
def start(self):
self._stdout = sys.stdout
self._stderr = sys.stderr
r, w = os.pipe()
r, w = os.fdopen(r, 'r'), os.fdopen(w, 'w', 1)
self._r = r
self._w = w
sys.stdout = self._w
sys.stderr = self._w
self._thread = threading.Thread(target=self._handler)
self._thread.start()
def stop(self):
self._w.close()
if self._thread: self._thread.join()
self._r.close()
sys.stdout = self._stdout
sys.stderr = self._stderr
Example usage:
from Capturing import *
import time
capturing = Capturing()
def on_read(line):
# do something with the line
capturing.print("got line: "+line)
capturing.on_readline(on_read)
capturing.start()
print("hello 1")
time.sleep(1)
print("hello 2")
time.sleep(1)
print("hello 3")
capturing.stop()

Also drawing on #kindall and #ForeveWintr's answers, here's a class that accomplishes this. The main difference from previous answers is that this captures it as a string, not as a StringIO object, which is much more convenient to work with!
import io
from collections import UserString
from contextlib import redirect_stdout
class capture(UserString, str, redirect_stdout):
'''
Captures stdout (e.g., from ``print()``) as a variable.
Based on ``contextlib.redirect_stdout``, but saves the user the trouble of
defining and reading from an IO stream. Useful for testing the output of functions
that are supposed to print certain output.
'''
def __init__(self, seq='', *args, **kwargs):
self._io = io.StringIO()
UserString.__init__(self, seq=seq, *args, **kwargs)
redirect_stdout.__init__(self, self._io)
return
def __enter__(self, *args, **kwargs):
redirect_stdout.__enter__(self, *args, **kwargs)
return self
def __exit__(self, *args, **kwargs):
self.data += self._io.getvalue()
redirect_stdout.__exit__(self, *args, **kwargs)
return
def start(self):
self.__enter__()
return self
def stop(self):
self.__exit__(None, None, None)
return
Examples:
# Using with...as
with capture() as txt1:
print('Assign these lines')
print('to a variable')
# Using start()...stop()
txt2 = capture().start()
print('This works')
print('the same way')
txt2.stop()
print('Saved in txt1:')
print(txt1)
print('Saved in txt2:')
print(txt2)
This is implemented in Sciris as sc.capture().

Based on kindall and ForeverWintr's answer.
I create redirect_stdout function for Python<3.4:
import io
from contextlib import contextmanager
#contextmanager
def redirect_stdout(f):
try:
_stdout = sys.stdout
sys.stdout = f
yield
finally:
sys.stdout = _stdout
f = io.StringIO()
with redirect_stdout(f):
do_something()
out = f.getvalue()

python: change sys.stdout print to custom print function

Im trying to understand how to create a custom print function.
(using python 2.7)
import sys
class CustomPrint():
def __init__(self):
self.old_stdout=sys.stdout #save stdout
def write(self, text):
sys.stdout = self.old_stdout #restore normal stdout and print
print 'custom Print--->' + text
sys.stdout= self # make stdout use CustomPrint on next 'print'
# this is the line that trigers the problem
# how to avoid this??
myPrint = CustomPrint()
sys.stdout = myPrint
print 'why you make 2 lines??...'
The code above prints this to console:
>>>
custom Print--->why you make 2 lines??...
custom Print--->
>>>
and i want to print only one line:
>>>
1custom Print--->why you make 2 lines??...
>>>
But cant figure out how to make this custom print work , i understand that there's some kind of recursion that triggers the second output to the console (i use self.write , to assign stdout to self.write himself !)
how can i make this work ? or is my approach just completely wrong...

It's not recursion. What happens is your write function is called twice, once with the text you expect, second time with just '\n'. Try this:
import sys
class CustomPrint():
def __init__(self):
self.old_stdout=sys.stdout
def write(self, text):
text = text.rstrip()
if len(text) == 0: return
self.old_stdout.write('custom Print--->' + text + '\n')
def flush(self):
self.old_stdout.flush()
What I do in the above code is I add the new line character to the text passed in the first call, and make sure the second call made by the print statement, the one meant to print new line, doesn't print anything.
Now try to comment out the first two lines and see what happens:
def write(self, text):
#text = text.rstrip()
#if len(text) == 0: return
self.old_stdout.write('custom Print--->' + text + '\n')

One solution may be to use a context manager if it's localised.
#!/usr/bin/env python
from __future__ import print_function
from contextlib import contextmanager
#############################
#contextmanager
def no_stdout():
import sys
old_stdout = sys.stdout
class CustomPrint():
def __init__(self, stdout):
self.old_stdout = stdout
def write(self, text):
if len(text.rstrip()):
self.old_stdout.write('custom Print--->' + text)
sys.stdout = CustomPrint(old_stdout)
try:
yield
finally:
sys.stdout = old_stdout
#############################
print("BEFORE")
with no_stdout():
print("WHY HELLO!\n")
print("DING DONG!\n")
print("AFTER")
The above produces:
BEFORE
custom Print--->WHY HELLO!
custom Print--->DING DONG!
AFTER
The code would need tidying up esp. around what the class should do WRT setting stdout back to what it was.

How about doing from __future__ import print_function. This way you will use Python3 print function instead of print statement from Python2. Then you can redefine the print function:
def print(*args, **kwargs):
__builtins__.print("Custom--->", *args, **kwargs)
There is a catch however, you will have to start using print function.