We Keep Coding

how to verfify python log format in unittest?

Recently I am writting an python logging extension, and I want to add some tests for my extension to verify whether my extension work as expected.
However, I don't know how to capture the complete log and compare with my excepted result in unittest/pytest.
simplified sample:
# app.py
import logging
def create_logger():
formatter = logging.Formatter(fmt='%(name)s-%(levelname)s-%(message)s')
hdlr = logging.StreamHandler()
hdlr.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.setLevel('DEBUG')
logger.addHandler(hdlr)
return logger
app_logger = create_logger()
Here is my tests
Attempt 1: unittest
from app import app_logger
import unittest
class TestApp(unittest.TestCase):
def test_logger(self):
with self.assertLogs('', 'DEBUG') as cm:
app_logger.debug('hello')
# or some other way to capture the log output.
self.assertEqual('app-DEBUG-hello', cm.output)
expected behaviour:
cm.output = 'app-DEBUG-hello'
actual behaviour
cm.output = ['DEBUG:app:hello']
Attempt 2: pytest caplog
from app import app_logger
import pytest
def test_logger(caplog):
app_logger.debug('hello')
assert caplog.text == 'app-DEBUG-hello'
expected behaviour:
caplog.text = 'app-DEBUG-hello'
actual behaviour
caplog.text = 'test_logger.py 6 DEBUG hello'
Attempt 3: pytest capsys
from app import app_logger
import pytest
def test_logger(capsys):
app_logger.debug('hello')
out, err = capsys.readouterr()
assert err
assert err == 'app-DEBUG-hello'
expected behaviour:
err = 'app-DEBUG-hello'
actual behaviour
err = ''
Considering there will be many tests with different format, I don't want to check the log format manually. I have no idea how to get complete log as I see on the console and compare it with my expected one in the test cases. Hoping for your help, thx.

I know this is old but posting here since it pulled up in google for me...
Probably needs cleanup but it is the first thing that has gotten close for me so I figured it would be good to share.
Here is a test case mixin I've put together that lets me verify a particular handler is being formatted as expected by copying the formatter:
import io
import logging
from django.conf import settings
from django.test import SimpleTestCase
from django.utils.log import DEFAULT_LOGGING
class SetupLoggingMixin:
def setUp(self):
super().setUp()
logging.config.dictConfig(settings.LOGGING)
self.stream = io.StringIO()
self.root_logger = logging.getLogger("")
self.root_hdlr = logging.StreamHandler(self.stream)
console_handler = None
for handler in self.root_logger.handlers:
if handler.name == 'console':
console_handler = handler
break
if console_handler is None:
raise RuntimeError('could not find console handler')
formatter = console_handler.formatter
self.root_formatter = formatter
self.root_hdlr.setFormatter(self.root_formatter)
self.root_logger.addHandler(self.root_hdlr)
def tearDown(self):
super().tearDown()
self.stream.close()
logging.config.dictConfig(DEFAULT_LOGGING)
And here is an example of how to use it:
class SimpleLogTests(SetupLoggingMixin, SimpleTestCase):
def test_logged_time(self):
msg = 'foo'
self.root_logger.error(msg)
self.assertEqual(self.stream.getvalue(), 'my-expected-message-formatted-as-expected')

After reading the source code of the unittest library, I've worked out the following bypass. Note, it works by changing a protected member of an imported module, so it may break in future versions.
from unittest.case import _AssertLogsContext
_AssertLogsContext.LOGGING_FORMAT = 'same format as your logger'
After these commands the logging context opened by self.assertLogs will use the above format. I really don't know why this values is left hard-coded and not configurable.
I did not find an option to read the format of a logger, but if you use logging.config.dictConfig you can use a value from the same dictionary.

I know this doesn't completely answer the OP's question but I stumbled upon this post while looking for a neat way to capture logged messages.
Taking what #user319862 did, I've cleaned it and simplified it.
import unittest
import logging
from io import StringIO
class SetupLogging(unittest.TestCase):
def setUp(self):
super().setUp()
self.stream = StringIO()
self.root_logger = logging.getLogger("")
self.root_hdlr = logging.StreamHandler(self.stream)
self.root_logger.addHandler(self.root_hdlr)
def tearDown(self):
super().tearDown()
self.stream.close()
def test_log_output(self):
""" Does the logger produce the correct output? """
msg = 'foo'
self.root_logger.error(msg)
self.assertEqual(self.stream.getvalue(), 'foo\n')
if __name__ == '__main__':
unittest.main()

I new to python but have some experience in test/tdd in other languages, and found that the default way of "changing" the formatter is by adding a new streamhandler BUT in the case you already have a stream defined in your logger (i.e. using Azure functions or TestCase::assertLogs that add one for you) you end up logging twice one with your format and another with the "default" format.
If in the OP the function create_logger mutates the formatter of current StreamHandler, instead of adding a new StreamHandler (checks if exist and if doesn't creates a new one and all that jazz...)
Then you can call the create_logger after the with self.assertLogs('', 'DEBUG') as cm: and just assert the cm.output and it just works because you are mutating the Formatter of the StreamHandler that the assertLogs is adding.
So basically what's happening is that the execution order is not appropriate for the test.
The order of execution in OP is:
import stuff
Add stream to logger formatter
Run test
Add another stream to logger formatter via self.assertLogs
assert stuff in 2nd StreamHandler
When it should be
the order of execution is:
import stuff
Add stream with logger formatter (but is irrelevant)
Run test
Add another stream with logger formatter via self.assertLogs
Change current stream logger formatter
assert stuff in only and properly formatted StreamHandler

How can I add contextual information to a LoggerAdapter instance AFTER it has been instantiated?

Is it possible to inject contextual information into a LoggerAdapter instance (that already has a set of 'extra' parms) and have it persist throughout the lifetime of the object? I have a parameter I want to inject in a particular method, but I also need other methods that use the instance to have that injected parameter. I kinda tried to do logger.extra["foo"] = "FOO" but it didn't work. Any help is appreciated.

Here's a working example:
import logging
import logging.handlers
import sys
def main():
logger = logging.getLogger()
adapter = logging.LoggerAdapter(logger, extra={'foo': 'bar'})
sh = logging.StreamHandler()
sh.setFormatter(logging.Formatter('%(levelname)s %(message)s [%(foo)s]'))
logger.addHandler(sh)
adapter.warning('This is a test warning')
adapter.extra['foo'] = 'baz'
adapter.warning('This is another test warning')
logger.removeHandler(sh)
sh.close()
if __name__ == '__main__':
sys.exit(main())
When you run it, it prints
WARNING This is a test warning [bar]
WARNING This is another test warning [baz]
So as you can see, the info in the adapter can be updated after instantiation.

Python: Write unittest for console print

Function foo prints to console. I want to test the console print. How can I achieve this in python?
Need to test this function, has NO return statement :
def foo(inStr):
print "hi"+inStr
My test :
def test_foo():
cmdProcess = subprocess.Popen(foo("test"), stdout=subprocess.PIPE)
cmdOut = cmdProcess.communicate()[0]
self.assertEquals("hitest", cmdOut)

You can easily capture standard output by just temporarily redirecting sys.stdout to a StringIO object, as follows:
import StringIO
import sys
def foo(inStr):
print "hi"+inStr
def test_foo():
capturedOutput = StringIO.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call unchanged function.
sys.stdout = sys.__stdout__ # Reset redirect.
print 'Captured', capturedOutput.getvalue() # Now works as before.
test_foo()
The output of this program is:
Captured hitest
showing that the redirection successfully captured the output and that you were able to restore the output stream to what it was before you began the capture.
Note that the code above in for Python 2.7, as the question indicates. Python 3 is slightly different:
import io
import sys
def foo(inStr):
print ("hi"+inStr)
def test_foo():
capturedOutput = io.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call function.
sys.stdout = sys.__stdout__ # Reset redirect.
print ('Captured', capturedOutput.getvalue()) # Now works as before.
test_foo()

This Python 3 answer uses unittest.mock. It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
import io
import unittest
import unittest.mock
from .solution import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
#unittest.mock.patch('sys.stdout', new_callable=io.StringIO)
def assert_stdout(self, n, expected_output, mock_stdout):
fizzbuzz(n)
self.assertEqual(mock_stdout.getvalue(), expected_output)
def test_only_numbers(self):
self.assert_stdout(2, '1\n2\n')
Note that the mock_stdout arg is passed automatically by the unittest.mock.patch decorator to the assert_stdout method.
A general-purpose TestStdout class, possibly a mixin, can in principle be derived from the above.
For those using Python ≥3.4, contextlib.redirect_stdout also exists, but it seems to serve no benefit over unittest.mock.patch.

If you happen to use pytest, it has builtin output capturing. Example (pytest-style tests):
def eggs():
print('eggs')
def test_spam(capsys):
eggs()
captured = capsys.readouterr()
assert captured.out == 'eggs\n'
You can also use it with unittest test classes, although you need to passthrough the fixture object into the test class, for example via an autouse fixture:
import unittest
import pytest
class TestSpam(unittest.TestCase):
#pytest.fixture(autouse=True)
def _pass_fixtures(self, capsys):
self.capsys = capsys
def test_eggs(self):
eggs()
captured = self.capsys.readouterr()
self.assertEqual('eggs\n', captured.out)
Check out Accessing captured output from a test function for more info.

You can also use the mock package as shown below, which is an example from
https://realpython.com/lessons/mocking-print-unit-tests.
from mock import patch
def greet(name):
print('Hello ', name)
#patch('builtins.print')
def test_greet(mock_print):
# The actual test
greet('John')
mock_print.assert_called_with('Hello ', 'John')
greet('Eric')
mock_print.assert_called_with('Hello ', 'Eric')

The answer of #Acumenus says:
It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
the bold part seems a big drawback, thus I would do the following instead:
# extend unittest.TestCase with new functionality
class TestCase(unittest.TestCase):
def assertStdout(self, expected_output):
return _AssertStdoutContext(self, expected_output)
# as a bonus, this syntactical sugar becomes possible:
def assertPrints(self, *expected_output):
expected_output = "\n".join(expected_output) + "\n"
return _AssertStdoutContext(self, expected_output)
class _AssertStdoutContext:
def __init__(self, testcase, expected):
self.testcase = testcase
self.expected = expected
self.captured = io.StringIO()
def __enter__(self):
sys.stdout = self.captured
return self
def __exit__(self, exc_type, exc_value, tb):
sys.stdout = sys.__stdout__
captured = self.captured.getvalue()
self.testcase.assertEqual(captured, self.expected)
this allows for the much nicer and much more re-usable:
# in a specific test case, the new method(s) can be used
class TestPrint(TestCase):
def test_print1(self):
with self.assertStdout("test\n"):
print("test")
by using a straight forward context manager. (It might also be desirable to append "\n" to expected_output since print() adds a newline by default. See next example...)
Furthermore, this very nice variant (for an arbitrary number of prints!)
def test_print2(self):
with self.assertPrints("test1", "test2"):
print("test1")
print("test2")
is possible now.

You can also capture the standard output of a method using contextlib.redirect_stdout:
import unittest
from contextlib import redirect_stdout
from io import StringIO
class TestMyStuff(unittest.TestCase):
# ...
def test_stdout(self):
with redirect_stdout(StringIO()) as sout:
my_command_that_prints_to_stdout()
# the stream replacing `stdout` is available outside the `with`
# you may wish to strip the trailing newline
retval = sout.getvalue().rstrip('\n')
# test the string captured from `stdout`
self.assertEqual(retval, "whatever_retval_should_be")
Gives you a locally scoped solution. It is also possible to capture the standard error using contextlib.redirect_stderr().

Another variant is leaning on the logging module rather than print(). This module also has a suggestion of when to use print in the documentation:
Display console output for ordinary usage of a command line script or program
PyTest has built-in support for testing logging messages.

How can I simulate input to stdin for pyunit?

I'm trying to test a function that takes input from stdin, which I'm currently testing with something like this:
cat /usr/share/dict/words | ./spellchecker.py
In the name of test automation, is there any way that pyunit can fake input to raw_input()?

The short answer is to monkey patch raw_input().
There are some good examples in the answer to How to display the redirected stdin in Python?
Here is a simple, trivial example using a lambda that throws away the prompt and returns what we want.
System Under Test
cat ./name_getter.py
#!/usr/bin/env python
class NameGetter(object):
def get_name(self):
self.name = raw_input('What is your name? ')
def greet(self):
print 'Hello, ', self.name, '!'
def run(self):
self.get_name()
self.greet()
if __name__ == '__main__':
ng = NameGetter()
ng.run()
$ echo Derek | ./name_getter.py
What is your name? Hello, Derek !
Test case:
$ cat ./t_name_getter.py
#!/usr/bin/env python
import unittest
import name_getter
class TestNameGetter(unittest.TestCase):
def test_get_alice(self):
name_getter.raw_input = lambda _: 'Alice'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Alice')
def test_get_bob(self):
name_getter.raw_input = lambda _: 'Bob'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Bob')
if __name__ == '__main__':
unittest.main()
$ ./t_name_getter.py -v
test_get_alice (__main__.TestNameGetter) ... ok
test_get_bob (__main__.TestNameGetter) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK

Update -- using unittest.mock.patch
Since python 3.3 there is new submodule for unittest called mock that does exactly what you need to do. For those using python 2.6 or above there is a backport of mock found here.
import unittest
from unittest.mock import patch
import module_under_test
class MyTestCase(unittest.TestCase):
def setUp(self):
# raw_input is untouched before test
assert module_under_test.raw_input is __builtins__.raw_input
def test_using_with(self):
input_data = "123"
expected = int(input_data)
with patch.object(module_under_test, "raw_input", create=True,
return_value=expected):
# create=True is needed as raw_input is not in the globals of
# module_under_test, but actually found in __builtins__ .
actual = module_under_test.function()
self.assertEqual(expected, actual)
#patch.object(module_under_test, "raw_input", create=True)
def test_using_decorator(self, raw_input):
raw_input.return_value = input_data = "123"
expected = int(input_data)
actual = module_under_test.function()
self.assertEqual(expected, actual)
def tearDown(self):
# raw input is restored after test
assert module_under_test.raw_input is __builtins__.raw_input
if __name__ == "__main__":
unittest.main()
# where module_under_test.function is:
def function():
return int(raw_input("prompt> "))
Previous answer -- replacing sys.stdin
I think the sys module might be what you're looking for.
You can do something like
import sys
# save actual stdin in case we need it again later
stdin = sys.stdin
sys.stdin = open('simulatedInput.txt','r')
# or whatever else you want to provide the input eg. StringIO
raw_input will now read from simulatedInput.txt whenever it is called. If the contents of simulatedInput was
hello
bob
then the first call to raw_input would return "hello", the second "bob" and third would throw an EOFError as there was no more text to read.

You didn't describe what sort of code is in spellchecker.py, which gives me freedom to speculate.
Suppose it's something like this:
import sys
def check_stdin():
# some code that uses sys.stdin
To improve testability of check_stdin function, I propose to refactor it like so:
def check_stdin():
return check(sys.stdin)
def check(input_stream):
# same as original code, but instead of
# sys.stdin it is written it terms of input_stream.
Now most of your logic is in check function, and you can hand-craft whatever input you can imagine in order to test it properly, without any need to deal with stdin.
My 2 cents.

Replace sys.stdin with an instance of StringIO, and load the StringIO instance with the data you want returned via sys.stdin. Also, sys.__stdin__ contains the original sys.stdin object, so restoring sys.stdin after your test is as simple as sys.stdin = sys.__stdin__.
Fudge is a great python mock module, with convenient decorators for doing patching like this for you, with automatic cleanup. You should check it out.

If you are using mock module (written by Michael Foord), in order to mock raw_input function you can use syntax like:
#patch('src.main.raw_input', create=True, new=MagicMock(return_value='y'))
def test_1(self):
method_we_try_to_test(); # method or function that calls **raw_input**

How to test or mock "if __name__ == '__main__'" contents

Say I have a module with the following:
def main():
pass
if __name__ == "__main__":
main()
I want to write a unit test for the bottom half (I'd like to achieve 100% coverage). I discovered the runpy builtin module that performs the import/__name__-setting mechanism, but I can't figure out how to mock or otherwise check that the main() function is called.
This is what I've tried so far:
import runpy
import mock
#mock.patch('foobar.main')
def test_main(self, main):
runpy.run_module('foobar', run_name='__main__')
main.assert_called_once_with()

I will choose another alternative which is to exclude the if __name__ == '__main__' from the coverage report , of course you can do that only if you already have a test case for your main() function in your tests.
As for why I choose to exclude rather than writing a new test case for the whole script is because if as I stated you already have a test case for your main() function the fact that you add an other test case for the script (just for having a 100 % coverage) will be just a duplicated one.
For how to exclude the if __name__ == '__main__' you can write a coverage configuration file and add in the section report:
[report]
exclude_lines =
if __name__ == .__main__.:
More info about the coverage configuration file can be found here.
Hope this can help.

You can do this using the imp module rather than the import statement. The problem with the import statement is that the test for '__main__' runs as part of the import statement before you get a chance to assign to runpy.__name__.
For example, you could use imp.load_source() like so:
import imp
runpy = imp.load_source('__main__', '/path/to/runpy.py')
The first parameter is assigned to __name__ of the imported module.

Whoa, I'm a little late to the party, but I recently ran into this issue and I think I came up with a better solution, so here it is...
I was working on a module that contained a dozen or so scripts all ending with this exact copypasta:
if __name__ == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
print(__doc__)
else:
sys.exit(main())
Not horrible, sure, but not testable either. My solution was to write a new function in one of my modules:
def run_script(name, doc, main):
"""Act like a script if we were invoked like a script."""
if name == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
sys.stdout.write(doc)
else:
sys.exit(main())
and then place this gem at the end of each script file:
run_script(__name__, __doc__, main)
Technically, this function will be run unconditionally whether your script was imported as a module or ran as a script. This is ok however because the function doesn't actually do anything unless the script is being ran as a script. So code coverage sees the function runs and says "yes, 100% code coverage!" Meanwhile, I wrote three tests to cover the function itself:
#patch('mymodule.utils.sys')
def test_run_script_as_import(self, sysMock):
"""The run_script() func is a NOP when name != __main__."""
mainMock = Mock()
sysMock.argv = []
run_script('some_module', 'docdocdoc', mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_as_script(self, sysMock):
"""Invoke main() when run as a script."""
mainMock = Mock()
sysMock.argv = []
run_script('__main__', 'docdocdoc', mainMock)
mainMock.assert_called_once_with()
sysMock.exit.assert_called_once_with(mainMock())
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_with_help(self, sysMock):
"""Print help when the user asks for help."""
mainMock = Mock()
for h in ('-h', '--help'):
sysMock.argv = [h]
run_script('__main__', h*5, mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
sysMock.stdout.write.assert_called_with(h*5)
Blam! Now you can write a testable main(), invoke it as a script, have 100% test coverage, and not need to ignore any code in your coverage report.

Python 3 solution:
import os
from importlib.machinery import SourceFileLoader
from importlib.util import spec_from_loader, module_from_spec
from importlib import reload
from unittest import TestCase
from unittest.mock import MagicMock, patch
class TestIfNameEqMain(TestCase):
def test_name_eq_main(self):
loader = SourceFileLoader('__main__',
os.path.join(os.path.dirname(os.path.dirname(__file__)),
'__main__.py'))
with self.assertRaises(SystemExit) as e:
loader.exec_module(module_from_spec(spec_from_loader(loader.name, loader)))
Using the alternative solution of defining your own little function:
# module.py
def main():
if __name__ == '__main__':
return 'sweet'
return 'child of mine'
You can test with:
# Override the `__name__` value in your module to '__main__'
with patch('module_name.__name__', '__main__'):
import module_name
self.assertEqual(module_name.main(), 'sweet')
with patch('module_name.__name__', 'anything else'):
reload(module_name)
del module_name
import module_name
self.assertEqual(module_name.main(), 'child of mine')

I did not want to exclude the lines in question, so based on this explanation of a solution, I implemented a simplified version of the alternate answer given here...
I wrapped if __name__ == "__main__": in a function to make it easily testable, and then called that function to retain logic:
# myapp.module.py
def main():
pass
def init():
if __name__ == "__main__":
main()
init()
I mocked the __name__ using unittest.mock to get at the lines in question:
from unittest.mock import patch, MagicMock
from myapp import module
def test_name_equals_main():
# Arrange
with patch.object(module, "main", MagicMock()) as mock_main:
with patch.object(module, "__name__", "__main__"):
# Act
module.init()
# Assert
mock_main.assert_called_once()
If you are sending arguments into the mocked function, like so,
if __name__ == "__main__":
main(main_args)
then you can use assert_called_once_with() for an even better test:
expected_args = ["expected_arg_1", "expected_arg_2"]
mock_main.assert_called_once_with(expected_args)
If desired, you can also add a return_value to the MagicMock() like so:
with patch.object(module, "main", MagicMock(return_value='foo')) as mock_main:

One approach is to run the modules as scripts (e.g. os.system(...)) and compare their stdout and stderr output to expected values.

I found this solution helpful. Works well if you use a function to keep all your script code.
The code will be handled as one code line. It doesn't matter if the entire line was executed for coverage counter (though this is not what you would actually actually expect by 100% coverage)
The trick is also accepted pylint. ;-)
if __name__ == '__main__': \
main()

If it's just to get the 100% and there is nothing "real" to test there, it is easier to ignore that line.
If you are using the regular coverage lib, you can just add a simple comment, and the line will be ignored in the coverage report.
if __name__ == '__main__':
main() # pragma: no cover
https://coverage.readthedocs.io/en/coverage-4.3.3/excluding.html
Another comment by # Taylor Edmiston also mentions it

My solution is to use imp.load_source() and force an exception to be raised early in main() by not providing a required CLI argument, providing a malformed argument, setting paths in such a way that a required file is not found, etc.
import imp
import os
import sys
def mainCond(testObj, srcFilePath, expectedExcType=SystemExit, cliArgsStr=''):
sys.argv = [os.path.basename(srcFilePath)] + (
[] if len(cliArgsStr) == 0 else cliArgsStr.split(' '))
testObj.assertRaises(expectedExcType, imp.load_source, '__main__', srcFilePath)
Then in your test class you can use this function like this:
def testMain(self):
mainCond(self, 'path/to/main.py', cliArgsStr='-d FailingArg')

To import your "main" code in pytest in order to test it you can import main module like other functions thanks to native importlib package :
def test_main():
import importlib
loader = importlib.machinery.SourceFileLoader("__main__", "src/glue_jobs/move_data_with_resource_partitionning.py")
runpy_main = loader.load_module()
assert runpy_main()