I have some unittests that I am running on a class, ClassA. This class has methods that call ClassB. ClassA and ClassB are in separate modules, class_A.py and class_B.py. Both of these modules have logger functions that are available globally within that module.
During my test of the methods of ClassA I want to suppress the logging messages. Using mock.patch it is straightforward to do this for the logger in class_A.py. But I have not found a way to patch the logger in class_B.py.
Here is some sample code:
unittest_file.py
import unittest
import class_A
class TestClassA(unittest.TestCase):
def setUp(self):
pass
def test_method(self):
my_class_A = class_A.ClassA()
my_class_A.run_method()
def tearDown(self):
pass
class_A.py
import logging
from class_B import ClassB
logger = logging.getLogger(__name__)
class ClassA:
run_method(self):
my_class_B = ClassB()
my_class_B.do_something()
logger.info("Running method on Class A")
class_B.py
import logging
logger = logging.getLogger(__name__)
class ClassB:
do_something(self):
logger.info("Doing something on Class B")
To patch the logger in class_A.py I can add #patch('__main__.unittest_file.ClassA.logger', autospec=True) as a decorator to the test_method() method.
But I don't have direct access to the logger in class_B.py to patch it as well.
The closest I have come to a solution is this:
#patch(eval('__main__.unittest_file.class_A.ClassB.__module__')['logger'])
but the eval() function applied to the module name doesn't preserve the import location, so it doesn't patch the logger in the right namespace.
I know I could easily solve this by changing the import statement in class_A.py to import class_B instead of from class_B import ClassB, but I don't want to edit the code in class_A.py or class_B.py, so that's not the answer I'm looking for.
I also know there are ways around this by disabling logging, as in this question but I am more interested in being able to fine-tune the control I have over different parts of the code to be tested, so that's not the answer I'm looking for either.
My question is:
Is there a way that I can navigate from an imported object to other objects in its parent module, while staying in the same namespace?
For me simply patching class_A.logger and class_B.logger works, e.g.
import unittest
from unittest import mock
class TestCaseA(unittest.TestCase):
#mock.patch('class_A.logger')
#mock.patch('class_B.logger')
def test_method(self, mockedA, mockedB):
my_class_A = ClassA()
my_class_A.run_method()
Both loggers don't output log messsages anymore.
Related
I am trying to mock a function that should get called when calling a class method:
# SomeClass.py
from some_module import some_function
class SomeClass:
def some_method(self, *a, **kw):
...
some_function()
...
# tests.py
from mock import patch
from some_package.SomeClass import SomeClass
class TestSomeClass:
#patch('SomeClass.some_function') # <- error
def test__some_function__called(self, mocked_function):
...
SomeClass().some_method()
mocked_function.assert_called()
However, I keep getting an error saying that SomeClass does not have a method called some_function.
The issue is due to the fact that the module name and the class name are the same, i.e. SomeClass. This is causing patch some confusion.
The solution: a combination of importlib and overriding the imported modules function with Mock():
# tests.py
import importlib
from mock import Mock
from some_package.SomeClass import SomeClass
class TestSomeClass:
def test__some_function__called(self):
some_class_module = importlib('some_package.SomeClass') # this is the actual module
some_class_module.some_function = Mock()
...
SomeClass().some_method()
some_class_module.some_function.assert_called() # <- no more error - tests pass
This is how I solved this issue - if anyone has an alternative solution it would be great to hear it. Hope this helps anyone with a similar issue.
I have been trying to create a new class of Logger by subclassing logging.Logger . Python version is 3.5
I have several modules in my application and I configure the logging only in the main module where I set the logger class using logging.setLoggerClass(...)
However when I retrieve the same Logger instance from some other module, it still creates a new instance of the Logger class and not the child class instance that I defined.
For example my code is :
# module 1
import logging
class MyLoggerClass(logging.getLoggerClass()):
def __init__(name):
super(MyLoggerClass, self).__init__(name)
def new_logger_method(...):
# some new functionality
if __name__ == "__main__":
logging.setLoggerClass(MyLoggerClass)
mylogger = logging.getLogger("mylogger")
# configuration of mylogger instance
# module 2
import logging
applogger = logging.getLogger("mylogger")
print(type(applogger))
def some_function():
applogger.debug("in module 2 some_function")
When this code is executed, I expect the applogger in module 2 to be of type MyLoggerClass. I intend to use the new_logger_method for some new functionality.
However since applogger is turning out to be of type logging.Logger, when the code is run it throws Logger has no attribute named new_logger_method.
Has anyone ever faced this issue?
Thanks in advance for any help!
Pranav
Instead of attempting to affect the global logger by changing the default logger factory, if you want your module to play nicely with any environment you should define a logger just for your module (and its children) and use it as a main logger for everything else deeper in your module structure. The trouble is that you explicitly want to use a different logging.Logger class than the default/globally defined one and the logging module doesn't provide an easy way to do context-based factory switching so you'll have to do it yourself.
There are many ways to do that but my personal preference is to be as explicit as possible and define your own logger module which you'll then import in your other modules in your package whenever you need to obtain a custom logger. In your case, you can create logger.py at the root of your package and do something like:
import logging
class CustomLogger(logging.Logger):
def __init__(self, name):
super(CustomLogger, self).__init__(name)
def new_logger_method(self, caller=None):
self.info("new_logger_method() called from: {}.".format(caller))
def getLogger(name=None, custom_logger=True):
if not custom_logger:
return logging.getLogger(name)
logging_class = logging.getLoggerClass() # store the current logger factory for later
logging._acquireLock() # use the global logging lock for thread safety
try:
logging.setLoggerClass(CustomLogger) # temporarily change the logger factory
logger = logging.getLogger(name)
logging.setLoggerClass(logging_class) # be nice, revert the logger factory change
return logger
finally:
logging._releaseLock()
Feel free to include other custom log initialization logic in it if you so desire. Then from your other modules (and sub-packages) you can import this logger and use its getLogger() to obtain a local, custom logger. For example, all you need in module1.py is:
from . import logger # or `from package import logger` for external/non-relative use
log = logger.getLogger(__name__) # obtain a main logger for this module
def test(): # lets define a function we can later call for testing
log.new_logger_method("Module 1")
This covers the internal use - as long as you stick to this pattern in all your modules/sub-modules you'll have the access to your custom logger.
When it comes to external use, you can write an easy test to show you that your custom logger gets created and that it doesn't interfere with the rest of the logging system therefore your package/module can be declared a good citizen. Under the assumption that your module1.py is in a package called package and you want to test it as a whole from the outside:
import logging # NOTE: we're importing the global, standard `logging` module
import package.module1
logging.basicConfig() # initialize the most rudimentary root logger
root_logger = logging.getLogger() # obtain the root logger
root_logger.setLevel(logging.DEBUG) # set root log level to DEBUG
# lets see the difference in Logger types:
print(root_logger.__class__) # <class 'logging.RootLogger'>
print(package.module1.log.__class__) # <class 'package.logger.CustomLogger'>
# you can also obtain the logger by name to make sure it's in the hierarchy
# NOTE: we'll be getting it from the standard logging module so outsiders need
# not to know that we manage our logging internally
print(logging.getLogger("package.module1").__class__) # <class 'package.logger.CustomLogger'>
# and we can test that it indeed has the custom method:
logging.getLogger("package.module1").new_logger_method("root!")
# INFO:package.module1:new_logger_method() called from: root!.
package.module1.test() # lets call the test method within the module
# INFO:package.module1:new_logger_method() called from: Module 1.
# however, this will not affect anything outside of your package/module, e.g.:
test_logger = logging.getLogger("test_logger")
print(test_logger.__class__) # <class 'logging.Logger'>
test_logger.info("I am a test logger!")
# INFO:test_logger:I am a test logger!
test_logger.new_logger_method("root - test")
# AttributeError: 'Logger' object has no attribute 'new_logger_method'
I'm trying to run unit tests on a class that imports two other modules, and I'm trying to patch out those modules using mock. One of these modules is instantiated in the class I'm testing, and I have not been able to patch it out. It appears that I have managed to patch out the other one.
What is the best way to patch out the sequence.processor module in this code?
Directory structure
logger.py
parser/
__init__.py
docparser.py
sequence/
__init__.py
processor.py
tests/
testdocparser.py
/parser/docparser.py
import logger
from sequence.processor import Processor
class DocParser(object):
def __init__(self, reader_writer):
self.processor = Processor(reader_writer)
def write_and_parse(self, products):
logger.log(products)
self.processor.process(products)
/tests/testdocparser.py
import unittest
from mock import MagicMock, patch
from parser import docparser
class DocParserTests(unittest.TestCase):
def setUp(self):
self.mock_writer = MagicMock()
self.docparser = docparser.DocParser(self.mock_writer)
#patch("parser.docparser.logger") # This seems to be patched properly
#patch("parser.docparser.Processor") # This is not patched
def test_write_and_parse(self, mock_logger, mock_proc):
products = "products"
self.docparser.write_and_parse(products)
You patch Processor in test_write_and_parse() but it's instantiated in DocParser.__init__() which is called from setUp().
This should work, though I haven't tested it:
class DocParserTests(unittest.TestCase):
def setUp(self):
self.mock_writer = MagicMock()
with patch('parser.docparser.Processor'):
self.docparser = docparser.DocParser(self.mock_writer)
#patch("parser.docparser.logger")
def test_write_and_parse(self, mock_logger):
products = "products"
self.docparser.write_and_parse(products)
I've used context manager instead of decorator to avoid changing setUp() signature (adding an argument).
Also the order of mock arguments for test_write_and_parse() is incorrect in your code. Here's an excerpt from mock docs:
When you nest patch decorators the mocks are passed in to the decorated function in the same order they applied (the normal python order that decorators are applied). This means from the bottom up...
Correct order:
#patch("parser.docparser.logger") # This seems to be patched properly
#patch("parser.docparser.Processor") # This is not patched
def test_write_and_parse(self, mock_proc, mock_logger):
# ...
Of cource, it doesn't really matter in your particular case because mock_proc and mock_logger are not used later.
I have this code that I want to test:
log = logging.getLogger(__name__)
class A(object):
def __init__(self):
log.debug('Init')
but I cannot figure out how to assert that log.debug was called with 'Init'
I tried patching logger but inspecting it I only found a getLogger mock.
I'm sure its simple, but I just cant figure it!
Thanks in advance for any and all help!
You can use patch.object() on the actual logging object. that lets you verify that you're using the correct logger too:
logger = logging.getLogger('path.to.module.under.test')
with mock.patch.object(logger, 'debug') as mock_debug:
run_code_under_test()
mock_debug.assert_called_once_with('Init')
Alternatively, if you're using Pytest, then it already has a fixture that captures logs for you:
def test_bar(caplog):
with caplog.at_level(logging.DEBUG):
run_code_under_test()
assert "Init" in caplog.text
# or, if you really need to check the log-level
assert caplog.records[-1].message == "Init"
assert caplog.records[-1].levelname == "DEBUG"
More info in the pytest docs on logging
Assuming log is a global variable in a module mymod, you want to mock the actual instance that getLogger returned, which is what invokes debug. Then, you can check if log.debug was called with the correct argument.
with mock.patch('mymod.log') as log_mock:
# test code
log_mock.debug.assert_called_with('Init')
I am late for this question but another of way to achieve it is:
#patch('package_name.module_name.log')
def test_log_in_A(self, mocked_log):
a = A()
mocked_log.debug.assert_called_once_with('Init')
This also works:
from unittest.mock import patch
class Test(TestCase):
def test_logger(self):
with patch('logging.Logger.warning') as mocked_logger:
call_func()
mocked_logger.assert_called_once_with('log')
Here is a complete example
"""
Source to test
"""
import logging
logger = logging.getLogger("abc")
def my_fonction():
logger.warning("Oops")
"""
Testing part
"""
import unittest
from unittest.mock import patch, MagicMock
abc_logger = logging.getLogger("abc")
class TestApp(unittest.TestCase):
#patch.object(abc_logger, "warning", MagicMock())
def test_my_fonction(self):
# When
my_fonction()
# Then
abc_logger.warning.assert_called_once()
I was wondering what the standard set up is for performing logging from within a Python app.
I am using the Logging class, and I've written my own logger class that instantiates the Logging class. My main then instantiates my logger wrapper class. However, my main instantiates other classes and I want those other classes to also be able to write to he log file via the logger object in the main.
How do I make that logger object such that it can be called by other classes? It's almost like we need some sort of static logger object to get this to work.
I guess the long and short of the question is: how do you implement logging within your code structure such that all classes instantiated from within main can write to the same log file? Do I just have to create a new logging object in each of the classes that points to the same file?
I don't know what you mean by the Logging class - there's no such class in Python's built-in logging. You don't really need wrappers: here's an example of how to do logging from arbitrary classes that you write:
import logging
# This class could be imported from a utility module
class LogMixin(object):
#property
def logger(self):
name = '.'.join([__name__, self.__class__.__name__])
return logging.getLogger(name)
# This class is just there to show that you can use a mixin like LogMixin
class Base(object):
pass
# This could be in a module separate from B
class A(Base, LogMixin):
def __init__(self):
# Example of logging from a method in one of your classes
self.logger.debug('Hello from A')
# This could be in a module separate from A
class B(Base, LogMixin):
def __init__(self):
# Another example of logging from a method in one of your classes
self.logger.debug('Hello from B')
def main():
# Do some work to exercise logging
a = A()
b = B()
with open('myapp.log') as f:
print('Log file contents:')
print(f.read())
if __name__ == '__main__':
# Configure only in your main program clause
logging.basicConfig(level=logging.DEBUG,
filename='myapp.log', filemode='w',
format='%(name)s %(levelname)s %(message)s')
main()
Generally it's not necessary to have loggers at class level: in Python, unlike say Java, the unit of program (de)composition is the module. However, nothing stops you from doing it, as I've shown above. The script, when run, displays:
Log file contents:
__main__.A DEBUG Hello from A
__main__.B DEBUG Hello from B
Note that code from both classes logged to the same file, myapp.log. This would have worked even with A and B in different modules.
Try using logging.getLogger() to get your logging object instance:
http://docs.python.org/3/library/logging.html#logging.getLogger
All calls to this function with a given name return the same logger instance. This means that logger instances never need to be passed between different parts of an application.
UPDATE:
The recommended way to do this is to use the getLogger() function and configure it (setting a handler, formatter, etc...):
# main.py
import logging
import lib
def main():
logger = logging.getLogger('custom_logger')
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler('test.log'))
logger.info('logged from main module')
lib.log()
if __name__ == '__main__':
main()
# lib.py
import logging
def log():
logger = logging.getLogger('custom_logger')
logger.info('logged from lib module')
If you really need to extend the logger class take a look at logging.setLoggerClass(klass)
UPDATE 2:
Example on how to add a custom logging level without changing the Logging class:
# main.py
import logging
import lib
# Extend Logger class
CUSTOM_LEVEL_NUM = 9
logging.addLevelName(CUSTOM_LEVEL_NUM, 'CUSTOM')
def custom(self, msg, *args, **kwargs):
self._log(CUSTOM_LEVEL_NUM, msg, args, **kwargs)
logging.Logger.custom = custom
# Do global logger instance setup
logger = logging.getLogger('custom_logger')
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler('test.log'))
def main():
logger = logging.getLogger('custom_logger')
logger.custom('logged from main module')
lib.log()
if __name__ == '__main__':
main()
Note that adding custom level is not recommended: http://docs.python.org/2/howto/logging.html#custom-levels
Defining a custom handler and maybe using more than one logger may do the trick for your other requirement: optional output to stderr.