Edit: Thanks #eemz for the idea to redesign the structure and use from unittest.mock import patch, but the problem persists.
So I just recently stumbled into unittest and I have a program which I normally start like this python run.py -config /path/to/config.file -y. I wanted to write a simple test in a separate test.py file: Execute the script, pass the mentioned arguments and get all of its output. I pass a prepared config file which is missing certain things, so the run.py will break and exactly log this error using logging.error: "xyz was missing in Config file!" (see example below). I'll get a few words from print() and then the logging instance kicks in and handles from there on. How do I get its output so I can check it? Feel free to rewrite this, as I'm still learning, please bear with me.
Simplified example:
run.py
import logging
def run(args):
< args.config = /path/to/config.file >
cnfg = Config(args.config)
cnfg.logger.info("Let's start with the rest of the code!") # This is NOT in 'output' of the unittest
< code >
if __name__ == "__main__":
print("Welcome! Starting execution.") # This is in 'output' of the unittest
< code to parse arguments 'args' >
run(args)
Config.py
import logging
class Config:
def __init__(self):
print("Creating logging instance, hold on ...") # This is in 'output' of the unittest
logger = logging.getLogger(__name__)
console_handler = logging.StreamHandler()
logger.addHandler(console_handler)
logger.info("Logging activated, let's go!") # This is NOT in 'output' of the unittest
self.logger = logger
if xyz not in config:
self.logger.error("xyz was missing in Config file!") # This is NOT in 'output' of the unittest
exit(1)
test.py
import unittest
from unittest.mock import patch
class TestConfigs(unittest.TestCase):
def test_xyz(self):
with patch('sys.stdout', new=StringIO()) as capture:
with self.assertRaises(SystemExit) as cm:
run("/p/to/f/missing/xyz/f", "", False, True)
output = capture.getvalue().strip()
self.assertEqual(cm.exception.code, 1)
# Following is working, because the print messages are in output
self.assertTrue("Welcome! Starting execution." in output)
# Following is NOT working, because the logging messages are not in output
self.assertTrue("xyz was missing in Config file!" in output)
if __name__ == "__main__":
unittest.main()
I would restructure run.py like this:
import logging
def main():
print("Welcome! Starting execution.")
Etc etc
if __name__ == "__main__":
main()
Then you can call the function run.main() in your unit test rather than forking a subprocess.
from io import StringIO
from unittest.mock import patch
import sys
import run
class etc etc
def test_run etc etc:
with patch('sys.stdout', new=StringIO()) as capture:
sys.argv = [‘run.py’, ‘-flag’, ‘-flag’, ‘-flag’]
run.main()
output = capture.getvalue().strip()
assert output == <whatever you expect it to be>
If you’re new to unit testing then you might not have seen mocks before. Effectively I am replacing stdout with a fake one to capture everything that gets sent there, so that I can pull it out later into the variable output.
In fact a second patch around sys.argv would be even better because what I’m doing here, an assignment to the real argv, will actually change it which will affect subsequent tests in the same file.
I ended up instantiating the logger of the main program with a specific name, so I could get the logger in the test.py again and assert that the logger was called with a specific text. I didn't know that I could just get the logger by using logging.getLogger("name") with the same name. Simplified example:
test.py
import unittest
from run import run
from unittest.mock import patch
main_logger = logging.getLogger("main_tool")
class TestConfigs(unittest.TestCase):
def test_xyz(self):
with patch('sys.stdout', new=StringIO()) as capture, \
self.assertRaises(SystemExit) as cm, \
patch.object(main_logger , "info") as mock_log1, \
patch.object(main_logger , "error") as mock_log2:
run("/path/to/file/missing/xyz.file")
output = capture.getvalue().strip()
self.assertTrue("Creating logging instance, hold on ..." in output)
mock_log1.assert_called_once_with("Logging activated, let's go!")
mock_log2.assert_called_once_with("xyz was missing in Config file!")
self.assertEqual(cm.exception.code, 1)
if __name__ == "__main__":
unittest.main()
run.py
def run(path: str):
cnfg = Config(path)
< code >
if __name__ == "__main__":
< code to parse arguments 'args' >
path = args.file_path
run(path)
Config.py
import logging
class Config:
def __init__(self, path: str):
print("Creating logging instance, hold on ...")
logger = logging.getLogger("main_tool")
console_handler = logging.StreamHandler()
logger.addHandler(console_handler)
logger.info("Logging activated, let's go!")
self.logger = logger
# Load file, simplified
config = load(path)
if xyz not in config:
self.logger.error("xyz was missing in Config file!")
exit(1)
This method seems to be very complicated and I got to this point by reading through a lot of other posts and the docs. Maybe some one knows a better way to achieve this.
Related
I am trying to figure out how to use the #patch.object to mock a __init__ and, log.write() for the Logger class, which is imported but isn't used inside the function of a module. The tuorial such as this, https://www.pythontutorial.net/python-unit-testing/python-patch/
, points to the patching needs to be at the target where it is used, not where it comes from. However, every example used shows the target to be mocked inside another function.
In the use case mentioned below, the logger is imported and used to write the log outside the scope of a function. Is there a way to mock the behavior both in main.py and routers.py?
src/apis/main.py
from utils.log import Logger
from routes import route
log = Logger(name="logger-1")
log.write("logger started")
def main():
log = Logger(name="logger-1")
log.write("inside main")
route()
if __name__ == "__main__":
import logging
logging.basicConfig(level=logging.INFO) # for demo
main()
In src/apis/routers/routes.py
from utils.log import Logger
log = Logger(name="logger-1")
log.write, message=f"Inside route")
def route():
log.write, message=f"Logging done.")
In utils/log/logging.py
import logging
Class Logger:
def __init__(self, name):
# needs to be mocked
def write(self, message):
# needs to be mocked to return None
When asking a question, it is very convenient to offer a Minimal Reproducible Example. So remove the unnecessary fastapi and starlette, and provide the code for the test you are trying to write.
Here it is :
# file: so74695297_main.py
from so74695297_log import Logger
from so74695297_routes import my_route
log = Logger(name="logger-1")
log.write("logger started")
def main(): # FAKE IMPL
log.write(message=f"in main()")
my_route()
if __name__ == "__main__":
import logging
logging.basicConfig(level=logging.INFO) # for demo
main()
# file: so74695297_routes.py
from so74695297_log import Logger
log = Logger(name="logger-1")
def my_route(): # FAKE IMPL
log.write(message=f"route")
# file: so74695297_log.py
import logging
class Logger:
def __init__(self, name):
self._logger = logging.getLogger(name) # FAKE IMPL
def write(self, message):
self._logger.info(message) # FAKE IMPL
when ran (the main.py file does something) :
INFO:logger-1:in main()
INFO:logger-1:route
Which is the expected output when the loggers don't fave any formatter.
Then adding a test :
# file: so74695297_test.py
import unittest
import unittest.mock as mock
from so74695297_routes import my_route
class TestMyRoute(unittest.TestCase):
def test__my_route_write_a_log(self):
spy_logger = mock.Mock()
with mock.patch("so74695297_log.Logger", new=spy_logger):
my_route()
assert spy_logger.assert_called()
if __name__ == "__main__":
unittest.main() # for demo
Ran 1 test in 0.010s
FAILED (failures=1)
Failure
Traceback (most recent call last):
File "/home/stack_overflow/so74695297_test.py", line 12, in test__my_route_write_a_log
assert spy_logger.assert_called()
File "/usr/lib/python3.8/unittest/mock.py", line 882, in assert_called
raise AssertionError(msg)
AssertionError: Expected 'mock' to have been called.
Now we have something to work with !
As #MrBeanBremen indicated, the fact that your logger is configured at import time (even when not being the "main" module) complicates things.
The problem is that, by the time the mock.patch line runs, the modules have already been imported and created their Logger. What we could do instead is mock the Logger.write method :
def test__my_route_writes_a_log(self):
with mock.patch("so74695297_log.Logger.write") as spy__Logger_write:
my_route()
spy__Logger_write.assert_called_once_with(message="route")
Ran 1 test in 0.001s
OK
If you prefer using the decorator form :
#mock.patch("so74695297_log.Logger.write")
def test__my_route_writes_a_log(self, spy__Logger_write):
my_route()
spy__Logger_write.assert_called_once_with(message="route")
Because we mocked the class's method, each Logger instance has a mock version of write :
# vvvv
#mock.patch("so74695297_main.Logger.write")
def test__main_writes_a_log(self, spy__Logger_write):
main()
# assert False, spy__Logger_write.mock_calls
spy__Logger_write.assert_any_call(message="in main()")
In the end, main.Logger.write is essentially the same thing as routes.Logger.write and as log.Logger.write, just a reference to the same "method" object. Mock from one way, mock for all the others too.
I wrote a Thing class that does logging using loguru. At the bottom of the class file I add a handler to the logger. This is what thing.py looks like.
from loguru import logger
class Thing:
def __init__(self):
logger.info("Thing created")
def __enter__(self):
logger.info("Thing entered")
return self
def __exit__(self, exc_type, exc_value, traceback):
logger.info("Thing exited")
logger.add("thing.log")
if __name__ == "__main__":
with Thing() as thing:
logger.info("In with block")
This works fine and it logs to thing.log as expected. What I would like to achieve is that it does not add the handler to thing.log when running tests.
This is my test file:
import pytest
from loguru import logger
from thing import Thing
#pytest.fixture
def thing(mocker):
mocker.patch("thing.logger", logger)
with Thing() as thing:
yield thing
def test_thing(thing, mocker):
mocker.patch("thing.logger", logger)
logger.info("In test")
assert isinstance(thing, Thing)
Now this test passes, but the logs are still written to thing.log (instead to only stdout, which is the default in for a loguru.logger).
How do I make sure that it only logs to the basic loguru.logger when running pytest?
What I tried:
Using monkeypatch instead of using mocker: monkeypatch.setattr("thing.logger", logger)
Patching in only one place (either in the fixture or in the test function)
Patching without replacement: mocker.patch("thing.logger") (so without a replacement logger)
Remove logger.add("thing.log") from thing.py!
You can either specify (as said in the docs) that you want to log to stdout: logger.add(sys.stdout) or just leave it out because the default for loguru.logger is in fact stdout!
The example provided in their docs:
logger.add(sys.stdout, format="{time} - {level} - {message}", filter="sub.module")
EDIT:
if __name__ == "__main__":
logger.add("thing.log")
if __name__ == "__main__":
with Thing() as thing:
#...
Now the logger will log to thing.log when the module is executed directly, but it will NOT add the file handler when the module is imported by another module (e.g. a test file).
Or you can use logger.remove(0) to stop logging when calling thing(mocker)!
Recently I am writting an python logging extension, and I want to add some tests for my extension to verify whether my extension work as expected.
However, I don't know how to capture the complete log and compare with my excepted result in unittest/pytest.
simplified sample:
# app.py
import logging
def create_logger():
formatter = logging.Formatter(fmt='%(name)s-%(levelname)s-%(message)s')
hdlr = logging.StreamHandler()
hdlr.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.setLevel('DEBUG')
logger.addHandler(hdlr)
return logger
app_logger = create_logger()
Here is my tests
Attempt 1: unittest
from app import app_logger
import unittest
class TestApp(unittest.TestCase):
def test_logger(self):
with self.assertLogs('', 'DEBUG') as cm:
app_logger.debug('hello')
# or some other way to capture the log output.
self.assertEqual('app-DEBUG-hello', cm.output)
expected behaviour:
cm.output = 'app-DEBUG-hello'
actual behaviour
cm.output = ['DEBUG:app:hello']
Attempt 2: pytest caplog
from app import app_logger
import pytest
def test_logger(caplog):
app_logger.debug('hello')
assert caplog.text == 'app-DEBUG-hello'
expected behaviour:
caplog.text = 'app-DEBUG-hello'
actual behaviour
caplog.text = 'test_logger.py 6 DEBUG hello'
Attempt 3: pytest capsys
from app import app_logger
import pytest
def test_logger(capsys):
app_logger.debug('hello')
out, err = capsys.readouterr()
assert err
assert err == 'app-DEBUG-hello'
expected behaviour:
err = 'app-DEBUG-hello'
actual behaviour
err = ''
Considering there will be many tests with different format, I don't want to check the log format manually. I have no idea how to get complete log as I see on the console and compare it with my expected one in the test cases. Hoping for your help, thx.
I know this is old but posting here since it pulled up in google for me...
Probably needs cleanup but it is the first thing that has gotten close for me so I figured it would be good to share.
Here is a test case mixin I've put together that lets me verify a particular handler is being formatted as expected by copying the formatter:
import io
import logging
from django.conf import settings
from django.test import SimpleTestCase
from django.utils.log import DEFAULT_LOGGING
class SetupLoggingMixin:
def setUp(self):
super().setUp()
logging.config.dictConfig(settings.LOGGING)
self.stream = io.StringIO()
self.root_logger = logging.getLogger("")
self.root_hdlr = logging.StreamHandler(self.stream)
console_handler = None
for handler in self.root_logger.handlers:
if handler.name == 'console':
console_handler = handler
break
if console_handler is None:
raise RuntimeError('could not find console handler')
formatter = console_handler.formatter
self.root_formatter = formatter
self.root_hdlr.setFormatter(self.root_formatter)
self.root_logger.addHandler(self.root_hdlr)
def tearDown(self):
super().tearDown()
self.stream.close()
logging.config.dictConfig(DEFAULT_LOGGING)
And here is an example of how to use it:
class SimpleLogTests(SetupLoggingMixin, SimpleTestCase):
def test_logged_time(self):
msg = 'foo'
self.root_logger.error(msg)
self.assertEqual(self.stream.getvalue(), 'my-expected-message-formatted-as-expected')
After reading the source code of the unittest library, I've worked out the following bypass. Note, it works by changing a protected member of an imported module, so it may break in future versions.
from unittest.case import _AssertLogsContext
_AssertLogsContext.LOGGING_FORMAT = 'same format as your logger'
After these commands the logging context opened by self.assertLogs will use the above format. I really don't know why this values is left hard-coded and not configurable.
I did not find an option to read the format of a logger, but if you use logging.config.dictConfig you can use a value from the same dictionary.
I know this doesn't completely answer the OP's question but I stumbled upon this post while looking for a neat way to capture logged messages.
Taking what #user319862 did, I've cleaned it and simplified it.
import unittest
import logging
from io import StringIO
class SetupLogging(unittest.TestCase):
def setUp(self):
super().setUp()
self.stream = StringIO()
self.root_logger = logging.getLogger("")
self.root_hdlr = logging.StreamHandler(self.stream)
self.root_logger.addHandler(self.root_hdlr)
def tearDown(self):
super().tearDown()
self.stream.close()
def test_log_output(self):
""" Does the logger produce the correct output? """
msg = 'foo'
self.root_logger.error(msg)
self.assertEqual(self.stream.getvalue(), 'foo\n')
if __name__ == '__main__':
unittest.main()
I new to python but have some experience in test/tdd in other languages, and found that the default way of "changing" the formatter is by adding a new streamhandler BUT in the case you already have a stream defined in your logger (i.e. using Azure functions or TestCase::assertLogs that add one for you) you end up logging twice one with your format and another with the "default" format.
If in the OP the function create_logger mutates the formatter of current StreamHandler, instead of adding a new StreamHandler (checks if exist and if doesn't creates a new one and all that jazz...)
Then you can call the create_logger after the with self.assertLogs('', 'DEBUG') as cm: and just assert the cm.output and it just works because you are mutating the Formatter of the StreamHandler that the assertLogs is adding.
So basically what's happening is that the execution order is not appropriate for the test.
The order of execution in OP is:
import stuff
Add stream to logger formatter
Run test
Add another stream to logger formatter via self.assertLogs
assert stuff in 2nd StreamHandler
When it should be
the order of execution is:
import stuff
Add stream with logger formatter (but is irrelevant)
Run test
Add another stream with logger formatter via self.assertLogs
Change current stream logger formatter
assert stuff in only and properly formatted StreamHandler
I am running Unit TestCases from Python Using XMLTestRunner.
This is my main file:
#LoggingService.py
class LoggerWriter:
# used to redirect console msgs to the log
def write(self, message):
log_event('warning', 'Console', message)
def direct_console_to_log():
global console_redirected
# This function can be called several times. Avoid redirecting more than once
if not console_redirected:
console_output = LoggerWriter()
sys.stderr = console_output
sys.stdout = console_output
console_redirected = True
def log_event(level, _id, event_details):
# log the event
pass
This is my one of the testing file:
#LoggingServiceTest.py
import unittest
import LoggingService
from LoggingService import direct_console_to_log
class Test(unittest.TestCase):
#patch('LoggingService.log_event')
def test_log_writer(self,log_mock):
direct_console_to_log()
print ('console msg')
self.assertTrue(log_mock.called)
if __name__ == "__main__":
unittest.main()
I used to run all the test file from XMLRunner.py:
#XMLRunner.py
from xmlrunner import XMLTestRunner
import unittest
test_modules = [file1.Test, file2.Test, LoggingServiceTest.py, .... file25.Test]
suite = unittest.TestSuite()
for test_module in test_modules:
tests = unittest.TestLoader().loadTestsFromTestCase(test_module)
suite.addTests(tests)
if __name__ == '__main__':
# import xmlrunner
# unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))
XMLTestRunner(output='test-reports').run(suite)
When I am trying to run LoggingServiceTest.py file from "XMLRunner.py", then the test report does not generate.But when I try without "LoggingServiceTest.py", then the report generates successfully.
And I am not sure, what is happening with the test case function
(test_log_writer(self,log_mock)) in LoggingServiceTest.py running.
Can anyone clear me
I use the following command to run the test case:
Python2.7.14\python.exe -m coverage run -p --branch --source=. --omit=Unit_Tests/*.py Unit_Tests/Testing.py
Finally I got the solution after debugging more.
When start test case from XMLRunner, It is creating some file objects for to write stderr and stdout and it is defined in sys.stderr, sys.stdout variable
And my test case is overwrite the sys.stderr, sys.stdout variable.
So XMLRunner try to fetching file objects from sys.stderr, sys.stdout, but it is not there, then it is not generating report.
I have the impression (but do not find the documentation for it) that unittest sets the logging level to WARNING for all loggers. I would like to:
be able to specify the logging level for all loggers, from the command line (when running the tests) or from the test module itself
avoid unittest messing around with the application logging level: when running the tests I want to have the same logging output (same levels) as when running the application
How can I achieve this?
I don't believe unittest itself does anything to logging, unless you use a _CapturingHandler class which it defines. This simple program demonstrates:
import logging
import unittest
logger = logging.getLogger(__name__)
class MyTestCase(unittest.TestCase):
def test_something(self):
logger.debug('logged from test_something')
if __name__ == '__main__':
# DEBUG for demonstration purposes, but you could set the level from
# cmdline args to whatever you like
logging.basicConfig(level=logging.DEBUG, format='%(name)s %(levelname)s %(message)s')
unittest.main()
When run, it prints
__main__ DEBUG logged from test_something
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
showing that it is logging events at DEBUG level, as expected. So the problem is likely to be related to something else, e.g. the code under test, or some other test runner which changes the logging configuration or redirects sys.stdout and sys.stderr. You will probably need to provide more information about your test environment, or better yet a minimal program that demonstrates the problem (as my example above shows that unittest by itself doesn't cause the problem you're describing).
See below example for logging in Python. Also you can change LOG_LEVEL using 'setLevel' method.
import os
import logging
logging.basicConfig()
logger = logging.getLogger(__name__)
# Change logging level here.
logger.setLevel(os.environ.get('LOG_LEVEL', logging.INFO))
logger.info('For INFO message')
logger.debug('For DEBUG message')
logger.warning('For WARNING message')
logger.error('For ERROR message')
logger.critical('For CRITICAL message')
This is in addition to #Vinay's answer above. It does not answer the original question. I wanted to include command line options for modifying the log level. The intent was to get detailed loggin only when I pass a certain parameter from the command line. This is how I solved it:
import sys
import unittest
import logging
from histogram import Histogram
class TestHistogram(unittest.TestCase):
def test_case2(self):
h = Histogram([2,1,2])
self.assertEqual(h.calculateMaxAreaON(), 3)
if __name__ == '__main__':
argv = len(sys.argv) > 1 and sys.argv[1]
loglevel = logging.INFO if argv == '-v' else logging.WARNING
logging.basicConfig(level=loglevel)
unittest.main()
The intent is to get more verbose logging. I know it does not answer the question, but I'll leave it here in case someone comes looking for a similar requirement such as this.
this worked for me:
logging.basicConfig(level=logging.DEBUG)
And if I wanted a specific format:
logging.basicConfig(
level=logging.DEBUG,
datefmt="%H:%M:%S",
format="%(asctime)s.%(msecs)03d [%(levelname)-5s] %(message)s",
)
Programmatically:
Put this line of code in each test function defined in your class that you want to set the logging level:
logging.getLogger().setLevel(logging.INFO)
Ex. class:
import unittest
import logging
class ExampleTest(unittest.TestCase):
def test_method(self):
logging.getLogger().setLevel(logging.INFO)
...
Command Line:
This example just shows how to do it in a normal script, not specific to unittest example. Capturing the log level via command line, using argparse for arguments:
import logging
import argparse
...
def parse_args():
parser = argparse.ArgumentParser(description='...')
parser.add_argument('-v', '--verbose', help='enable verbose logging', action='store_const', dest="loglevel", const=logging.INFO, default=logging.WARNING)
...
def main():
args = parse_args()
logging.getLogger().setLevel(args.loglevel)