We Keep Coding

How to spec a Python mock which defaults to AttributeError() without explicit assignment?

I have a question related to Python unittest.mock.Mock and spec_set functionalities.
My goal is to create a Mock with the following functionalities:
It has a spec of an arbitrary class I decide at creation time.
I must be able to assign on the mock only attributes or methods according to the spec of point 1
The Mock must raise AttributeError in the following situations:
I try to assign an attribute that is not in the spec
I call or retrieve a property that is either missing in the spec_set, or present in the spec_set but assigned according to the above point.
Some examples of the behavior I would like:
class MyClass:
property: int = 5
def func() -> int:
pass
# MySpecialMock is the Mock with the functionalities I am dreaming about :D
mock = MyMySpecialMock(spec_set=MyClass)
mock.not_existing # Raise AttributeError
mock.func() # Raise AttributeError
mock.func = lambda: "it works"
mock.func() # Returns "it works"
I have tried multiple solutions without any luck, or without being explicitly verbose. The following are some examples:
Using Mock(spec_set=...), but it does not raise errors in case I call a specced attribute which I did not explicitly set
Using Mock(spec_set=...) and explicitly override every attribute with a function with an Exception side effect, but it is quite verbose since I must repeat all the attributes...
My goal is to find a way to automatize 2, but I have no clean way to do so. Did you ever encounter such a problem, and solve it?
For the curious ones, the goal is being able to enhance the separation of unit testings; I want to be sure that my mocks are called only on the methods I explicitly set, to avoid weird and unexpected side effects.
Thank you in advance!

spec_set defines a mock object which is the same as the class, but then doesn't allow any changes to be made to it, since it defines special __getattr__ and __setattr__. This means that the first test (calling a non-existent attr) will fail as expected, but then so will trying to set an attr:
from unitest import mock
class X:
pass
m = mock.Mock(spec_set=X)
m.func()
# __getattr__: AttributeError: Mock object has no attribute 'func'
m.func = lambda: "it works"
# __setattr__: AttributeError: Mock object has no attribute 'func'
Instead, you can use create_autospec() which copies an existing function, and adds the mock functions to it, but without affecting __setattr__:
n = mock.create_autospec(X)
n.func()
# __getattr__: AttributeError: Mock object has no attribute 'func'
n.func = lambda: "it works"
n.func()
# 'it works'

I think I found a satisfying answer to my problem, by using the dir method.
To create the Mock with the requirements I listed above, it should be enough to do the following:
def create_mock(spec: Any) -> Mock:
mock = Mock(spec_set=spec)
attributes_to_override = dir(spec)
for attr in filter(lambda name: not name.startswith("__"), attributes_to_override):
setattr(mock, attr, Mock(side_effect=AttributeError(f"{attr} not implemented")))
return mock

Patching in pytest before import

Given a module
some_module.py
_foo = Foo(config)
def do_foo():
return _foo.foo()
Foo.__init__ raises an exception, so I am not able to write a test since the exception is already raised at import time in some_module.py.
I want to patch the module to replace _foo with an object of another type, say TestFoo. I tried using mock.patch, but this does not work because the exception is already raised before the patching is applied.
I am coming from java where I would have injected through constructor while testing, which is easy. What are my options in Python?

Asserting an exception generated by a #propertyname.setter

The Source Code
I have a bit of code requiring that I call a property setter to test wether or not locking functionaliy of a class is working (some functions of the class are async, requiring that a padlock boolean be set during their execution). The setter has been written to raise a RuntimeError if the lock has been set for the instance.
Here is the code:
#filename.setter
def filename(self, value):
if not self.__padlock:
self.__filename = value
else:
self.__events.on_error("data_store is locked. you should be awaiting safe_unlock if you wish to "
"change the source.")
As you can see here, if self.__padlock is True, a RuntimeError is raised. The issue arises when attempting to assert the setter with python unittest .
The Problem
It appears that unittest lacks functionality needed to assert wether or not a property setter raises an exception.
Attempting to use assertRaises doesn't obviously work:
# Non-working concept
self.assertRaises(RuntimeError, my_object.filename, "testfile.txt")
The Question
How does one assert that a class's property setter will raise a given exception in a python unittest.TestCase method?

You need to actually invoke the setter, via an assignment. This is simple to do, as long as you use assertRaises as a context manager.
with self.assertRaises(RuntimeError):
my_object.filename = "testfile.txt"
If you couldn't do that, you would have to fall back to an explicit try statement (which gets tricky, because you need to handle both "no exception" and "exception other than RuntimeError" separately.
try:
my_object.filename = "testfile.txt"
except RuntimeError:
pass
except Exception:
raise AssertionError("something other than RuntimeError")
else:
raise AssertionError("no RuntimeError")
or (more) explicitly invoke the setter:
self.assertRaises(RuntimeError, setattr, myobject, 'filename', 'testfile.txt')
or worse, explicitly invoke the setter:
self.assertRaises(RuntimeError, type(myobject).filename.fset, myobject, 'testfile.txt')
In other words, three cheers for context managers!

You can use the setattr method like so:
self.assertRaises(ValueError, setattr, p, "name", None)
In the above example, we will try to set p.name equal to None and check if there is a ValueError raised.

Custom exceptions in unittests

I have created my custom exceptions as such within errors.py
mapper = {
'E101':
'There is no data at all for these constraints',
'E102':
'There is no data for these constraints in this market, try changing market',
'E103':
'There is no data for these constraints during these dates, try changing dates',
}
class DataException(Exception):
def __init__(self, code):
super().__init__()
self.msg = mapper[code]
def __str__(self):
return self.msg
Another function somewhere else in the code raises different instances of DataException if there is not enough data in a pandas dataframe. I want to use unittest to ensure that it returns the appropriate exception with its corresponding message.
Using a simple example, why does this not work:
from .. import DataException
def foobar():
raise DataException('E101')
import unittest
with unittest.TestCase.assertRaises(DataException):
foobar()
As suggested here: Python assertRaises on user-defined exceptions
I get this error:
TypeError: assertRaises() missing 1 required positional argument: 'expected_exception'
Or alternatively:
def foobar():
raise DataException('E101')
import unittest
unittest.TestCase.assertRaises(DataException, foobar)
results in:
TypeError: assertRaises() arg 1 must be an exception type or tuple of exception types
Why is it not recognizing DataException as an Exception? Why does the linked stackoverflow question answer work without supplying a second argument to assertRaises?

You are trying to use methods of the TestCase class without creating an instance; those methods are not designed to be used in that manner.
unittest.TestCase.assertRaises is an unbound method. You'd use it in a test method on a TestCase class you define:
class DemoTestCase(unittest.TestCase):
def test_foobar(self):
with self.assertRaises(DataException):
foobar()
The error is raised because unbound methods do not get self passed in. Because unittest.TestCase.assertRaises expects both self and a second argument named expected_exception you get an exception as DataException is passed in as the value for self.
You do now have to use a test runner to manage your test cases; add
if __name__ == '__main__':
unittest.main()
at the bottom and run your file as a script. Your test cases are then auto-discovered and executed.
It is technically possible to use the assertions outside such an environment, see Is there a way to use Python unit test assertions outside of a TestCase?, but I recommend you stick to creating test cases instead.
To further verify the codes and message on the raised exception, assign the value returned when entering the context to a new name with with ... as <target>:; the context manager object captures the raised exception so you can make assertions about it:
with self.assertRaises(DataException) as context:
foobar()
self.assertEqual(context.exception.code, 'E101')
self.assertEqual(
context.exception.msg,
'There is no data at all for these constraints')
See the TestCase.assertRaises() documentation.
Last but not least, consider using subclasses of DataException rather than use separate error codes. That way your API users can just catch one of those subclasses to handle a specific error code, rather than having to do additional tests for the code and re-raise if a specific code should not have been handled there.

Python, rasing an exception without arguments

I would like to know the best practice about raising an exception without arguments.
In the official python documentation, you can see this :
try:
raise KeyboardInterrupt
(http://docs.python.org/tutorial/errors.html chap. 8.6)
and in some differents code, like Django or Google code, you can see this :
def AuthenticateAndRun(self, username, password, args):
raise NotImplementedError()
(http://code.google.com/p/neatx/source/browse/trunk/neatx/lib/auth.py)
The exception is instanciate before being raised while there is no argument.
What is the purpose to instanciate an exception without arguments ? When I should use the first case or the second case ?
Thanks in advance
Fabien

You can use whichever form you like. There is no real difference and both are legal in Python 2 and 3. Python style guide does not specify which one is recommended.
A little more information on the "class form" support:
try:
raise KeyboardInterrupt
This form is perfectly legal in both Python 2 and 3.
Excerpt from pep-3109:
raise EXCEPTION is used to raise a new exception. This form has two
sub-variants: EXCEPTION may be an exception class or an instance of an
exception class; valid exception classes are BaseException and its
subclasses [5]. If EXCEPTION is a subclass, it will be called with no
arguments to obtain an exception instance.
It is also described in Python documentation:
... If it is a class, the exception instance will be obtained when needed
by instantiating the class with no arguments.

Raising an exception class instead of an exception instance is deprecated syntax and should not be used in new code.
raise Exception, "This is not how to raise an exception..."

In languages like C++ you can raise any object, not just Exceptions. Python is more constrained. If you try :
raise 1
You get:
Traceback (most recent call last):
...
TypeError: exceptions must be old-style classes or derived from BaseException, not int
In python programming model you can usually use a class by itself instead of an instance (this is handy for fast creation of unique instances, just define a class). Hence no wonder you can raise an exception class instead of an exception instance.
However like Ignacio said, that is deprecated.
Also, some side-note that is not the direct question:
You could also see raise alone by itself in some code. Without any object class or anything. It is used in exception handlers to re-raise the current exception and will keep the initial traceback.