I would like to capture error messages which are generated from a python module A. A is written in C++ and SWIG, and so I cannot capture Python's sys.stderr.
>>> import A
>>> A.func() # this prints an error message by C++ std::cerr.
this is an error message from module A
What I want to do are to suppress the error message and to change my script behavior according to the error type. But A.func() does not return error numbers.
Assuming that my use of contextlib below is correct, it did not help.
>>> import io
>>> f = io.StringIO()
>>> import contextlib
>>> with contextlib.redirect_stderr(f):
... no_return = A.func()
ERROR MESSAGE HERE
>>> f.getvalue()
>>>
Thank you #PM2Ring
https://eli.thegreenplace.net/2015/redirecting-all-kinds-of-stdout-in-python/#id1
With minor modifications: replacing all the stdout with stderr and supporting macOS as shown below, it worked perfectly as expected.
if sys.platform == 'darwin':
c_stderr = ctypes.c_void_p.in_dll(libc, '__stderrp')
else:
c_stderr = ctypes.c_void_p.in_dll(libc, 'stderr')
Related
I have a program that uses the Python fileinput module, and I am trying to write unittests for the main() function. They work find when using an actual file, but raise OSError: reading from stdin while output is captured when I try to pass data via stdin. What is the correct way to mock the stdin input when using fileinput?
Example my_fileinput.py:
"""
$ echo "42" | python3.8 my_fileinput.py -
answer: 42
"""
import fileinput
import sys
def main():
for line in fileinput.input(sys.argv[1:]):
sys.stdout.write(f"answer #{fileinput.lineno()}: {line}")
if __name__ == "__main__":
main()
Example test_my_fileinput.py:
"""
$ python3.10 -m pytest test_my_fileinput.py
OSError: reading from stdin while output is captured
"""
import io
from unittest import mock
import my_fileinput
def test_stdin():
"""Test fileinput with stdin."""
with mock.patch.object(my_fileinput, "raw_input", create=True, return_value="42"):
with mock.patch("sys.stdout", new=io.StringIO()) as stdout:
with mock.patch("sys.argv", ["my_fileinput.py", "-"]):
# Raises OSError: reading from stdin while output is captured
my_fileinput.main()
assert stdout.getvalue() == "answer #1: 42"
I have tried various ways of mocking stdin, all with the same results. All result in the same OSError.
Update: A different version patching sys.stdin instead of inputfile.input
import io
from unittest import mock
import my_fileinput
def test_stdin():
"""Test fileinput with stdin."""
with mock.patch("sys.stdin", new=io.StringIO("42\n")):
with mock.patch("sys.stdout", new=io.StringIO()) as stdout:
with mock.patch("sys.argv", ["my_fileinput.py", "-"]):
my_fileinput.main()
assert stdout.getvalue() == "answer: 42\n"
Warning: The original answer gets rid of the OSError, but renders other functions in the inputfile module unusable (see comments).
Original:
Changing the first two arguments of mock.patch.object to fileinput and "input" seems to fix the OSError.
with mock.patch.object(fileinput, "input", create=True, return_value="42"):
The first argument is the target object you want to patch, which is the fileinput module. The second argument is the attribute to be changed in target object, which is input.
import io
import fileinput
from unittest import mock
import my_fileinput
def test_stdin():
"""Test fileinput with stdin."""
with mock.patch.object(fileinput, "input", create=True, return_value="42"):
with mock.patch("sys.stdout", new=io.StringIO()) as stdout:
with mock.patch("sys.argv", ["my_fileinput.py", "-"]):
my_fileinput.main()
assert stdout.getvalue() == "answer: 42\n"
It is not necessary for you to test fileinput itself, since that will be tested by CPython's own test suite: Lib/test/test_fileinput.py. The pytesthonic way to test your code would be like this, using fixtures:
import my_fileinput
def test_stdin(mocker, capsys):
mocker.patch("fileinput.lineno", return_value=1)
mocker.patch("fileinput.input", return_value=["42\n"])
my_fileinput.main()
out, err = capsys.readouterr()
assert out == "answer #1: 42\n"
The capsys fixture is included with pytest, and the mocker fixture is provided by plugin pytest-mock.
I am learning selenium and intend to check methods available.
browser = webdriver.Chrome()
browser.get(start_url)
help(browser)
The help document is too long so I'd like to copy them to a file.
In [19]: with open("webdriver.md", "w") as file:
...: file.write(help(browser))
...:
TypeError: write() argument must be str, not None
Either pydoc is not helpful
In [23]: pydoc.writedoc("browser")
No Python documentation found for 'browser'.
Use help() to get the interactive help utility.
Use help(str) for help on the str class.
How could I write help(browser) to a plain text file.
bulit-in help() is a wrapper around pydoc.Helper, it writes to stdout by default, you could temporarily redirecting sys.stdout to a file:
>>> import contextlib
>>> with contextlib.redirect_stdout(open('browser_help.txt', 'w')):
... help(browser)
or you could call pydoc.Helper directly, :
>>> import pydoc
>>> with open('browser_help.txt', 'w') as f:
... h = pydoc.Helper(output=f)
... h(browser)
You can try to change the current stdout:
import sys
sys.stdout = open('webdriver.md', 'w')
help(browser)
Starting from Python 3.4, you can also use contextlib.redirect_stdout:
from contextlib import redirect_stdout
with redirect_stdout(open('webdriver.md', 'w')):
help(browser)
I'm using python's ftplib to write a small FTP client, but some of the functions in the package don't return string output, but print to stdout. I want to redirect stdout to an object which I'll be able to read the output from.
I know stdout can be redirected into any regular file with:
stdout = open("file", "a")
But I prefer a method that doesn't uses the local drive.
I'm looking for something like the BufferedReader in Java that can be used to wrap a buffer into a stream.
from cStringIO import StringIO # Python3 use: from io import StringIO
import sys
old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()
# blah blah lots of code ...
sys.stdout = old_stdout
# examine mystdout.getvalue()
There is a contextlib.redirect_stdout() function in Python 3.4+:
import io
from contextlib import redirect_stdout
with io.StringIO() as buf, redirect_stdout(buf):
print('redirected')
output = buf.getvalue()
Here's a code example that shows how to implement it on older Python versions.
Just to add to Ned's answer above: you can use this to redirect output to any object that implements a write(str) method.
This can be used to good effect to "catch" stdout output in a GUI application.
Here's a silly example in PyQt:
import sys
from PyQt4 import QtGui
class OutputWindow(QtGui.QPlainTextEdit):
def write(self, txt):
self.appendPlainText(str(txt))
app = QtGui.QApplication(sys.argv)
out = OutputWindow()
sys.stdout=out
out.show()
print "hello world !"
A context manager for python3:
import sys
from io import StringIO
class RedirectedStdout:
def __init__(self):
self._stdout = None
self._string_io = None
def __enter__(self):
self._stdout = sys.stdout
sys.stdout = self._string_io = StringIO()
return self
def __exit__(self, type, value, traceback):
sys.stdout = self._stdout
def __str__(self):
return self._string_io.getvalue()
use like this:
>>> with RedirectedStdout() as out:
>>> print('asdf')
>>> s = str(out)
>>> print('bsdf')
>>> print(s, out)
'asdf\n' 'asdf\nbsdf\n'
Starting with Python 2.6 you can use anything implementing the TextIOBase API from the io module as a replacement.
This solution also enables you to use sys.stdout.buffer.write() in Python 3 to write (already) encoded byte strings to stdout (see stdout in Python 3).
Using StringIO wouldn't work then, because neither sys.stdout.encoding nor sys.stdout.buffer would be available.
A solution using TextIOWrapper:
import sys
from io import TextIOWrapper, BytesIO
# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)
# do something that writes to stdout or stdout.buffer
# get output
sys.stdout.seek(0) # jump to the start
out = sys.stdout.read() # read output
# restore stdout
sys.stdout.close()
sys.stdout = old_stdout
This solution works for Python 2 >= 2.6 and Python 3.
Please note that our new sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings.
This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes, which again often makes use of sys.stdout.buffer.
You can build a slight variation that accepts unicode and byte strings for write():
class StdoutBuffer(TextIOWrapper):
def write(self, string):
try:
return super(StdoutBuffer, self).write(string)
except TypeError:
# redirect encoded byte strings directly to buffer
return super(StdoutBuffer, self).buffer.write(string)
You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.
This method restores sys.stdout even if there's an exception. It also gets any output before the exception.
import io
import sys
real_stdout = sys.stdout
fake_stdout = io.BytesIO() # or perhaps io.StringIO()
try:
sys.stdout = fake_stdout
# do what you have to do to create some output
finally:
sys.stdout = real_stdout
output_string = fake_stdout.getvalue()
fake_stdout.close()
# do what you want with the output_string
Tested in Python 2.7.10 using io.BytesIO()
Tested in Python 3.6.4 using io.StringIO()
Bob, added for a case if you feel anything from the modified / extended code experimentation might get interesting in any sense, otherwise feel free to delete it
Ad informandum ... a few remarks from extended experimentation during finding some viable mechanics to "grab" outputs, directed by numexpr.print_versions() directly to the <stdout> ( upon a need to clean GUI and collecting details into debugging-report )
# THIS WORKS AS HELL: as Bob Stein proposed years ago:
# py2 SURPRISEDaBIT:
#
import io
import sys
#
real_stdout = sys.stdout # PUSH <stdout> ( store to REAL_ )
fake_stdout = io.BytesIO() # .DEF FAKE_
try: # FUSED .TRY:
sys.stdout.flush() # .flush() before
sys.stdout = fake_stdout # .SET <stdout> to use FAKE_
# ----------------------------------------- # + do what you gotta do to create some output
print 123456789 # +
import numexpr # +
QuantFX.numexpr.__version__ # + [3] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
QuantFX.numexpr.print_versions() # + [4] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
_ = os.system( 'echo os.system() redir-ed' )# + [1] via real_stdout + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
_ = os.write( sys.stderr.fileno(), # + [2] via stderr + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
b'os.write() redir-ed' )# *OTHERWISE, if via fake_stdout, EXC <_io.BytesIO object at 0x02C0BB10> Traceback (most recent call last):
# ----------------------------------------- # ? io.UnsupportedOperation: fileno
#''' ? YET: <_io.BytesIO object at 0x02C0BB10> has a .fileno() method listed
#>>> 'fileno' in dir( sys.stdout ) -> True ? HAS IT ADVERTISED,
#>>> pass; sys.stdout.fileno -> <built-in method fileno of _io.BytesIO object at 0x02C0BB10>
#>>> pass; sys.stdout.fileno()-> Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# io.UnsupportedOperation: fileno
# ? BUT REFUSES TO USE IT
#'''
finally: # == FINALLY:
sys.stdout.flush() # .flush() before ret'd back REAL_
sys.stdout = real_stdout # .SET <stdout> to use POP'd REAL_
sys.stdout.flush() # .flush() after ret'd back REAL_
out_string = fake_stdout.getvalue() # .GET string from FAKE_
fake_stdout.close() # <FD>.close()
# +++++++++++++++++++++++++++++++++++++ # do what you want with the out_string
#
print "\n{0:}\n{1:}{0:}".format( 60 * "/\\",# "LATE" deferred print the out_string at the very end reached -> real_stdout
out_string #
)
'''
PASS'd:::::
...
os.system() redir-ed
os.write() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.5
NumPy version: 1.10.4
Python version: 2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU? True
VML available? True
VML/MKL version: Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
>>>
EXC'd :::::
...
os.system() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.5
NumPy version: 1.10.4
Python version: 2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU? True
VML available? True
VML/MKL version: Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
Traceback (most recent call last):
File "<stdin>", line 9, in <module>
io.UnsupportedOperation: fileno
'''
In Python3.6, the StringIO and cStringIO modules are gone, you should use io.StringIO instead.So you should do this like the first answer:
import sys
from io import StringIO
old_stdout = sys.stdout
old_stderr = sys.stderr
my_stdout = sys.stdout = StringIO()
my_stderr = sys.stderr = StringIO()
# blah blah lots of code ...
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
// if you want to see the value of redirect output, be sure the std output is turn back
print(my_stdout.getvalue())
print(my_stderr.getvalue())
my_stdout.close()
my_stderr.close()
Use pipe() and write to the appropriate file descriptor.
https://docs.python.org/library/os.html#file-descriptor-operations
Here's another take on this. contextlib.redirect_stdout with io.StringIO() as documented is great, but it's still a bit verbose for every day use. Here's how to make it a one-liner by subclassing contextlib.redirect_stdout:
import sys
import io
from contextlib import redirect_stdout
class capture(redirect_stdout):
def __init__(self):
self.f = io.StringIO()
self._new_target = self.f
self._old_targets = [] # verbatim from parent class
def __enter__(self):
self._old_targets.append(getattr(sys, self._stream)) # verbatim from parent class
setattr(sys, self._stream, self._new_target) # verbatim from parent class
return self # instead of self._new_target in the parent class
def __repr__(self):
return self.f.getvalue()
Since __enter__ returns self, you have the context manager object available after the with block exits. Moreover, thanks to the __repr__ method, the string representation of the context manager object is, in fact, stdout. So now you have,
with capture() as message:
print('Hello World!')
print(str(message)=='Hello World!\n') # returns True
I am using the ctypes module to do some ptrace system calls on Linux, which actually works
pretty well. But if I get an error I wanna provide some useful information. Therefore I
do an get_errno() function call which returns the value of errno, but I didn't found
any function or something else which interprets the errno value and gives me an associated
error message.
Am I missing something?
Is there a ctypes based solution?
Here is my setup:
import logging
from ctypes import get_errno, cdll
from ctypes.util import find_library, errno
# load the c lib
libc = cdll.LoadLibrary(find_library("c"), use_errno=True)
...
Example:
return_code = libc.ptrace(PTRACE_ATTACH, pid, None, None)
if return_code == -1:
errno = get_errno()
error_msg = # here i wanna provide some information about the error
logger.error(error_msg)
This prints ENODEV: No such device.
import errno, os
def error_text(errnumber):
return '%s: %s' % (errno.errorcode[errnumber], os.strerror(errnumber))
print error_text(errno.ENODEV)
>>> import errno
>>> import os
>>> os.strerror(errno.ENODEV)
'No such device'
How can I tell the file (or tty) that is attached to my stdios?
Something like:
>>> import sys
>>> print sys.stdin.__path__
'/dev/tty1'
>>>
I could look in proc:
import os, sys
os.readlink('/proc/self/fd/%s' % sys.stdin.fileno())
But seems like there should be a builtin way?
The sys.std* objects are standard Python file objects, so they have a name attribute and a isatty method:
>>> import sys
>>> sys.stdout.name
'<stdout>'
>>> sys.stdout.isatty()
True
>>> anotherfile = open('/etc/hosts', 'r')
>>> anotherfile.name
'/etc/hosts'
>>> anotherfile.isatty()
False
Short of telling you exactly what TTY device you got, that's the extend of the API offered by Python.
Got it!
>>> import os
>>> import sys
>>> print os.ttyname(sys.stdin.fileno())
'/dev/pts/0'
>>>
It raises OSError: [Errno 22] Invalid argument if stdin isn't a TTY; but thats easy enough to test for with isatty()