We Keep Coding

Cannot set side_effect on a mocked method

I'm trying to write a test for a function that uses a class as a dependency and calls this class method(s).
Let's assume the function is
def store_username_and_password(**kwargs) -> Tuple[str, StorageResult]:
storage = MyDependency(param1, param2)
try:
storage.read_data(mountpoint, path)
except InvalidPathException:
storage.write_data(data, mountpoint, path)
return (f"Stored successfully {some_params}", StorageResult(some_params))
In the test I'm trying to patch MyDependency like this:
input = {....}
with patch("my.application.namespace.MyDependency") as mock_storage:
mock_storage.read_data.side_effect = InvalidPathException("the data does not exist yet")
with raises(InvalidPathException) as e:
store_username_and_password(**input)
However, when I debug it and step inside the function call from the test above and proceed to the storage.read_data(mountpoint, path) call, I see in the debugger that there is no side_effect set. So it never raises that exception I want on read_data call.
See below:

Anthony's comment helped me but I thought I'd add an explanation...
mock_storage.read_data.side_effect will work, but only if you call MyDependency.read_data(). What happens is the call to MyDependency(param1, param2) creates a new MagicMock that doesn't have the side_effect set on it. The new mock can be accessed with mock_storage.return_value.
So, if you have:
storage = MyDependency(param1, param2)
storage.read_data(mountpoint, path)
Then to mock that method you need:
with patch("my.application.namespace.MyDependency") as mock_storage:
mock_storage.return_value.read_data.side_effect = Exception

Using mock.patch.object with "new" parameter

I am currently using unittest and unittest.mock to test my code. For example, when I want to overwrite a certain method's behaviour I do it as following:
def mock_get(address):
class Response():
def __init__(self):
self.status_code = 200
def json(self):
dict= {"key":value}
return json.dumps(dict)
r = Response()
return r
with mock.patch.object(requests, 'get', new=mock_get) as get:
#call function that uses requests.get
get.assert_called_with(address)#this throws an exception saying that a function() type does not have an attribute called "assert_called_with"
In this example, the same could be achieved without the "new" parameter, however there are cases where I want to write a new function to replace the original one, yet be able to access the mock instance like you would when you patch a function without the "new" parameter like this:
with mock.patch("path.to.requests.get") as get:
get.status_code = 200
get.json.return_value = {"test_key": "test_value"}
# call function that uses requests.get
get.assert_called_with(address)#this works

How to group decorators in Python

In Flask I'm using a set of decorators for each route, but the code is "ugly":
#app.route("/first")
#auth.login_required
#crossdomain(origin='*')
#nocache
def first_page:
....
#app.route("/second")
#auth.login_required
#crossdomain(origin='*')
#nocache
def second_page:
....
I would prefer to have a declaration that groups all of them with a single decorator:
#nice_decorator("/first")
def first_page:
....
#nice_decorator("/second")
def second_page:
....
I tried to follow the answer at Can I combine two decorators into a single one in Python? but I cannot make it working:
def composed(*decs):
def deco(f):
for dec in reversed(decs):
f = dec(f)
return f
return deco
def nice_decorator(route):
composed(app.route(route),
auth.login_required,
crossdomain(origin="*"),
nocache)
#nice_decorator("/first")
def first_page:
....
because of this error that I don't understand:
#nice_decorator("/first")
TypeError: 'NoneType' object is not callable
Following one of the comments I defined it with another form that works but without the possibility to specify the route parameter:
new_decorator2 = composed(app.route("/first"),
auth.login_required,
crossdomain(origin="*"),
nocache)
Is it possible to define a single decorator with parameters?

You're not defining the composition correctly. You need to change the definition of nice_decorator to something like this:
def nice_decorator(route):
return composed(
app.route(route),
auth.login_required,
crossdomain(origin="*"),
nocache
)
Your previous version never actually returned anything. Python isn't like Ruby or Lisp where the last expression is the return value.

Is it possible to create a dynamic localized scope in Python?

I have a scenario where I'm dynamically running functions at run-time and need to keep track of a "localized" scope. In the example below, "startScope" and "endScope" would actually be creating levels of "nesting" (in reality, the stuff contained in this localized scope isn't print statements...it's function calls that send data elsewhere and the nesting is tracked there. startScope / endScope just set control flags that are used to start / end the current nesting depth).
This all works fine for tracking the nested data, however, exceptions are another matter. Ideally, an exception would result in "falling out" of the current localized scope and not end the entire function (myFunction in the example below).
def startScope():
#Increment our control object's (not included in this example) nesting depth
control.incrementNestingDepth()
def endScope():
#Decrement our control object's (not included in this example) nesting depth
control.decrementNestingDepth()
def myFunction():
print "A"
print "B"
startScope()
print "C"
raise Exception
print "D"
print "This print statement and the previous one won't get printed"
endScope()
print "E"
def main():
try:
myFunction()
except:
print "Error!"
Running this would (theoretically) output the following:
>>> main()
A
B
C
Error!
E
>>>
I'm quite certain this isn't possible as I've written it above - I just wanted to paint a picture of the sort of end-result I'm trying to achieve.
Is something like this possible in Python?
Edit: A more relevant (albeit lengthy) example of how this is actually being used:
class Log(object):
"""
Log class
"""
def __init__(self):
#DataModel is defined elsewhere and contains a bunch of data structures / handles nested data / etc...
self.model = DataModel()
def Warning(self, text):
self.model.put("warning", text)
def ToDo(self, text):
self.model.put("todo", text)
def Info(self, text):
self.model.put("info", text)
def StartAdvanced(self):
self.model.put("startadvanced")
def EndAdvanced(self):
self.model.put("endadvanced")
def AddDataPoint(self, data):
self.model.put("data", data)
def StartTest(self):
self.model.put("starttest")
def EndTest(self):
self.model.put("endtest")
def Error(self, text):
self.model.put("error", text)
#myScript.py
from Logger import Log
def test_alpha():
"""
Crazy contrived example
In this example, there are 2 levels of nesting...everything up to StartAdvanced(),
and after EndAdvanced() is included in the top level...everything between the two is
contained in a separate level.
"""
Log.Warning("Better be careful here!")
Log.AddDataPoint(fancyMath()[0])
data = getSerialData()
if data:
Log.Info("Got data, let's continue with an advanced test...")
Log.StartAdvanced()
#NOTE: If something breaks in one of the following methods, then GOTO (***)
operateOnData(data)
doSomethingCrazy(data)
Log.ToDo("Fill in some more stuff here later...")
Log.AddDataPoint(data)
Log.EndAdvanced()
#(***) Ideally, we would resume here if an exception is raised in the above localized scope
Log.Info("All done! Log some data and wrap everything up!")
Log.AddDataPoint({"data": "blah"})
#Done
#framework.py
import inspect
from Logger import Log
class Framework(object):
def __init__(self):
print "Framework init!"
self.tests = []
def loadTests(self, file):
"""
Simplifying this for the sake of clarity
"""
for test in file:
self.tests.append(test)
def runTests(self):
"""
Simplifying this for the sake of clarity
"""
#test_alpha() as well as any other user tests will be run here
for test in self.tests:
Log.StartTest()
try:
test()
except Exception,e :
Log.Error(str(e))
Log.EndTest()
#End

You can achieve a similar effect with a context manager using a with statement. Here I use the contextlib.contextmanager decorator:
#contextlib.contextmanager
def swallower():
try:
yield
except ZeroDivisionError:
print("We stopped zero division error")
def foo():
print("This error will be trapped")
with swallower():
print("Here comes error")
1/0
print("This will never be reached")
print("Merrily on our way")
with swallower():
print("This error will propagate")
nonexistentName
print("This won't be reached")
>>> foo()
This error will be trapped
Here comes error
We stopped zero division error
Merrily on our way
This error will propagate
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
foo()
File "<pyshell#3>", line 10, in foo
nonexistentName
NameError: global name 'nonexistentName' is not defined
It cannot be done with an ordinary function call as in your example. In your example, the function startScope returns before the rest of the body of myFunction executes, so startScope can't have any effect on it. To handle exceptions, you need some kind of explicit structure (either a with statement or a regular try/except) inside myFunction; there's no way to make a simple function call magically intercept exceptions that are raised in its caller.
You should read up on context managers as they seem to fit what you're trying to do. The __enter__ and __exit__ methods of the context manager would correspond to your startScope and endScope. Whether it will do exactly what you want depends on exactly what you want those "manager" functions to do, but you will probably have more luck doing it with a context manager than trying to do it with simple function calls.

Exceptions that reflect error codes of a remote service

I'm working with an external service which reports errors by code.
I have the list of error codes and the associated messages. Say, the following categories exist: authentication error, server error.
What is the smartest way to implement these errors in Python so I can always lookup an error by code and get the corresponding exception object?
Here's my straightforward approach:
class AuthError(Exception):
pass
class ServerError(Exception):
pass
map = {
1: AuthError,
2: ServerError
}
def raise_code(code, message):
""" Raise an exception by code """
raise map[code](message)
Would like to see better solutions :)

Your method is correct, except that map should be renamed something else (e.g. ERROR_MAP) so it does not shadow the builtin of the same name.
You might also consider making the function return the exception rather than raising it:
def error(code, message):
""" Return an exception by code """
return ERROR_MAP[code](message)
def foo():
raise error(code, message)
By placing the raise statement inside foo, you'd raise the error closer to where the error occurred and there would be one or two less lines to trace through if the stack trace is printed.

Another approach is to create a polymorphic base class which, being instantiated, actually produces a subclass that has the matching code.
This is implemented by traversing __subclasses__() of the parent class and comparing the error code to the one defined in the class. If found, use that class instead.
Example:
class CodeError(Exception):
""" Base class """
code = None # Error code
def __new__(cls, code, *args):
# Pick the appropriate class
for E in cls.__subclasses__():
if E.code == code:
C = E
break
else:
C = cls # fall back
return super(CodeError, cls).__new__(C, code, *args)
def __init__(self, code, message):
super(CodeError, self).__init__(message)
# Subclasses with error codes
class AuthError(CodeError):
code = 1
class ServerError(CodeError):
code = 2
CodeError(1, 'Wrong password') #-> AuthError
CodeError(2, 'Failed') #-> ServerError
With this approach, it's trivial to associate error message presets, and even map one class to multiple codes with a dict.