I'm writing some unit tests for a server program which catches most exceptions, but logs them, and would like to make assertions on the logged output. I've found the testfixtures package useful to this end; for example:
import logging
import testfixtures
with testfixtures.LogCapture() as l:
logging.info('Here is some info.')
l.check(('root', 'INFO', 'Here is some info.'))
Following the documentation, the check method will raise an error if either the logger name, level, or message is not as expected.
I would like to perform a more 'flexible' kind of test in which I make assertions on the message using a wildcard for the other elements of the tuple. This less stringent assertion would look something like
l.check((*, *, 'Here is some info.'))
but this is not valid syntax. Is there any way to specify a 'wildcard' in the check method of the testfixtures.logcapture.LogCapture class?
The way to check messages only (which, as pointed out to me by the author, is actually described in the documentation) is to use the records attribute of the LogCapture class, which is a list of logging.LogRecord objects. So the appropriate assertion is:
assert l.records[-1].getMessage() == 'Here is some info.'
Related
In Java, Junit has some "assert" methods that, when they fail, will tell you something like "Assertion failed. Expected x but saw y".
What's an equivalent library in Python for that, as opposed to "assert x" producing "assertion failed" but not further information?
When you use the assert keyword you can add customised error messages:
assert your_statement_here, "Custom error message here"
Within your custom error message error you can format any variables or data you need to show to debug.
However, if you need something more powerful, as #Asocia said, you will need unittest, specifically assert-methods is what I think you are looking for, and the official documentation can help with that:
https://docs.python.org/3/library/unittest.html#assert-methods
The TestCase class provides several assert methods to check for and
report failures.
assertEqual(a, b) checks: a == b
assertNotEqual(a, b) checks: a != b
... and so on
Main Question
I am using a module that relies on logging instead of raising error messages. How can I catch logged errors from within Python to react to them (without dissecting the log file)?
Minimal Example
Suppose logging_module.py looks like this:
import logging
import random
def foo():
logger = logging.getLogger("quz")
if random.choice([True,False]):
logger.error("Doooom")
If this module used exceptions, I could do something like this:
from logging_module import foo, Doooom
try:
foo()
except Doooom:
bar()
Assuming that logging_module is written the way it is and I cannot change it, this is impossible. What can I do instead?
What I considered so far
I went through the logging documentation (though I did not read every word), but the only way to access what is logged seems to be dissecting the actual log, which seems overly tedious to me (but I may misunderstand this).
You can add a filter to the logger that the module uses to inspect every log. The documentation has this to say on using filters for something like that:
Although filters are used primarily to filter records based on more
sophisticated criteria than levels, they get to see every record which
is processed by the handler or logger they’re attached to: this can be
useful if you want to do things like counting how many records were
processed by a particular logger or handler
The code below assumes that you are using the logging_module that you showed in the question and tries to emulate what the try-except does: that is, when an error happens inside a call of foo the function bar is called.
import logging
from logging_module import foo
def bar():
print('error was logged')
def filt(r):
if r.levelno == logging.ERROR:
bar()
return True
logger = logging.getLogger('quz')
logger.addFilter(filt)
foo() # bar will be called if this logs an error
I have created python function to search through an ldap object as below:
def my_search(l, baseDN, searchScope=ldap.SCOPE_ONELEVEL, searchFilter="objectClass=*", retrieveAttributes=None):
logger.console("Reachedhere")
try:
logger.console("Reachedhereinsidetry\n")
ldap_result_id = l.search_s(baseDN,searchScope,searchFilter,retrieveAttributes)
logger.console("Gotresult\n")
So I invoke this keyword now in a Robot Testcase as so:
*** Settings ***
Documentation This testsuite checks the LDAP functionalities of DB nodes.
Resource ../../COMMON/Libraries/SDL-DB-COMMON-LIB.txt
Library ../../COMMON/Libraries/pythonldap.py
*** Test Cases ***
Perform Ldap Operations
${ldapObj} ldapopen ${DB_1_EXT_APP_IP}
Log to Console ${ldapObj}
${SearchReturn} my_search ${ldapObj} "uid=5000000,ds=CRIBER,o=D,dc=CN" ldap.SCOPE_ONELEVEL "objectClass=*" None
When I run this TC, it throws me an error in the search like so:
TypeError: an integer is required
The error is definitely in "ldap_result_id = l.search_s(baseDN,searchScope,searchFilter,retrieveAttributes)" line, since I'm able to print the earlier comments.
What is the issue here?
The issue here is the scope level which cannot be passed as above from Robot. The changes I did was :
def my_search(l, baseDN, searchScopeLevel, searchFilter="objectClass=*", retrieveAttributes=None):
try:
if searchScopeLevel == 'ONE':
searchScope=ldap.SCOPE_ONELEVEL
elif searchScopeLevel == 'BASE':
searchScope=ldap.SCOPE_BASE
elif searchScopeLevel == 'SUB':
searchScope=ldap.SCOPE_SUBTREE
ldap_result_id = l.search(baseDN,searchScope,searchFilter,retrieveAttributes)
Robot TC Changes :
*** Test Cases ***
Perform Ldap Operations
${ldapObj} ldapopen ${DB_1_EXT_APP_IP}
${SearchReturn} my_search ${ldapObj} uid=205000000,ds=CRIBER,o=DEFT,dc=C ONE objectClass=*
And the issue is resolved. :)
Presuming the exception is raised in the my_search method - by default the arguments to methods in RF are casted to string. Thus this call:
${SearchReturn} my_search ${ldapObj} "uid=2620105000000,ds=SUBSCRIBER,o=DEFAULT,dc=C-NTDB" ldap.SCOPE_ONELEVEL "objectClass=*" None
Has a number of issues:
the baseDN argument will have an actual value "uid=2620105000000,ds=SUBSCRIBER,o=DEFAULT,dc=C-NTDB" - i.e. with the quotation marks included, thus probably not what you're aiming for; remove them
the same for the searchFilter - remove the quotes in the call
the searchScope, which probably is your problem, will receive the value ldap.SCOPE_ONELEVEL - a string with this content. This most probably is a constant defined in your ldap module; the safest bet that it'll work is to provide the integer value of that const - integers are given in the format ${1}, but that's hardly sustainable. Perhaps you could export it and the other constants in the COMMON/Libraries/pythonldap.py library, and use it in the test cases
finally, the retrieveAttributes argument will receive the string literal "None", not the None datatype you probably want; to get it, use this RF builtin variable - ${None}
HTH, and again - provide more details to receive on-the-spot answers.
I'd like to a log some information to a file/database every time assert is invoked. Is there a way to override assert or register some sort of callback function to do this, every time assert is invoked?
Regards
Sharad
Try overload the AssertionError instead of assert. The original assertion error is available in exceptions module in python2 and builtins module in python3.
import exceptions
class AssertionError:
def __init__(self, *args, **kwargs):
print("Log me!")
raise exceptions.AssertionError
I don't think that would be possible. assert is a statement (and not a function) in Python and has a predefined behavior. It's a language element and cannot just be modified. Changing the language cannot be the solution to a problem. Problem has to be solved using what is provided by the language
There is one thing you can do though. Assert will raise AssertionError exception on failure. This can be exploited to get the job done. Place the assert statement in Try-expect block and do your callbacks inside that block. It isn't as good a solution as you are looking for. You have to do this with every assert. Modifying a statement's behavior is something one won't do.
It is possible, because pytest is actually re-writing assert expressions in some cases. I do not know how to do it or how easy it is, but here is the documentation explaining when assert re-writing occurs in pytest:
https://docs.pytest.org/en/latest/assert.html
By default, if the Python version is greater than or equal to 2.6, py.test rewrites assert statements in test modules.
...
py.test rewrites test modules on import. It does this by using an
import hook to write a new pyc files.
Theoretically, you could look at the pytest code to see how they do it, and perhaps do something similar.
For further information, Benjamin Peterson wrote up Behind the scenes of py.test’s new assertion rewriting [ at http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html ]
I suggest to use pyhamcrest. It has very beatiful matchers which can be simply reimplemented. Also you can write your own.
I like the fact that the unit test class will print out the nice summary about what pass and fail; although I can't see on the official documentation, how do you customize that output.
I see the msg parameter for the assert, which means that I can print a descriptive message when the assert trigger (test fail), but what if you want to include a summary on success?