We Keep Coding

Mocking a class used in a with statement

I have a class which has an __exit__ and __enter__ function so that I can use it in a with statement, e.g.:
with ClassName() as c:
c.do_something()
I am now trying to write a unit test to test this. Basically, I am trying to test that do_something() has only been called once.
An example (which I called testmocking1):
class temp:
def __init__(self):
pass
def __enter__(self):
pass
def __exit__(self, exc_type, exc_val, exc_tb):
pass
def test_method(self):
return 1
def fun():
with temp() as t:
return t.test_method()
And my test:
import unittest
import test_mocking1
from test_mocking1 import fun
import mock
from mock import patch
class MyTestCase(unittest.TestCase):
#patch('test_mocking1.temp', autospec = True)
def test_fun_enter_called_once(self, mocked_object):
fun()
mocked_object.test_method.assert_called_once()
if __name__ == '__main__':
unittest.main()
So I would expect this to pass, because the test_method has been called exactly once in the function fun(). But the actual result that I get is:
======================================================================
FAIL: test_fun_enter_called_once (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<path_to_virtual_env>\lib\site-packages\mock\mock.py", line 1305, in patched
return func(*args, **keywargs)
File "<File_with_test>", line 11, in test_fun_enter_called_once
mocked_object.test_method.assert_called_once()
File "<path_to_virtual_env>\lib\site-
packages\mock\mock.py", line 915, in assert_called_once
raise AssertionError(msg)
AssertionError: Expected 'test_method' to have been called once. Called 0 times.
How do I test whether a function in a class which is created using a with statement has been called (either once or multiple times), and (related) how do I set the results of those calls (using .side_effect or .return_value)?

The with statement takes whatever __enter__ returns to bind to the name in the as <name> part. You bound it to t:
with temp() as t:
t.test_method()
Note that temp() is called, so the with statement starts with temp.return_value. t is not temp.return_value either, it is whatever temp().__enter__() returns, so you need to use the return value for that call:
entered = mocked_object.return_value.__enter__.return_value
entered.test_method.assert_called_once()
Extending on this, if you want to alter what test_method() returns, do so on the return value of mocked_object.return_value.__enter__.return_value.
You can always print out the mock_calls() attribute of your object to see what has happened to it:
>>> from test_mocking1 import fun
>>> from mock import patch
>>> with patch('test_mocking1.temp', autospec = True) as mocked_object:
... fun()
...
>>> print(mocked_object.mock_calls)
[call(),
call().__enter__(),
call().__enter__().test_method(),
call().__exit__(None, None, None)]
>>> mocked_object.return_value.__enter__.return_value.test_method.called
True
>>> mocked_object.return_value.__enter__.return_value.test_method.call_count
1
Note that your actual implementation of temp.__enter__() returns None, so without mocking your fun() function fails with an attribute error.

Importing and using a decorator in another file

I have a simple decorator like this below. However when I import the python file it immediately runs and I can't call the function again. How are decorators supposed to be used?
def plain_decorator(func):
def decorated_func():
print "Decorating"
func()
print "Decorated"
return decorated_func()
#plain_decorator
def hw():
print "Hello Decorators!"
>>> import decorator_ex2 as d
Decorating
Hello Decorators!
Decorated
>>> d.hw()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable
>>>

try using this, its because you are calling your inner(decorated_func) function when you return from outer(plain_decorator) function
def plain_decorator(func):
def decorated_func():
print "Decorating"
func()
print "Decorated"
return decorated_func
#plain_decorator
def hw():
print "Hello Decorators!"

python patch with side_effect on Object's method is not called with self

I encounter a surprising behaviour of the side_effect parameter in patch.object where the function replacing the original does not receive self
class Animal():
def __init__(self):
self.noise = 'Woof'
def make_noise(self):
return self.noise
def loud(self):
return self.noise.upper() + '!!'
from unittest.mock import patch
dog = Animal()
dog.make_noise()
with patch.object(Animal, 'make_noise', side_effect=loud):
dog.make_noise()
This give:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 902, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 968, in _mock_call
ret_val = effect(*args, **kwargs)
TypeError: loud() missing 1 required positional argument: 'self'
If I change the loud function to
def loud(*args, **kwargs):
print(args)
print(kwargs)
I get the following output:
()
{}
Is there a way to replace a function from an object and still receive self?

self is only supplied for bound methods (because functions are descriptors). A Mock object is not such a method, and the side_effect function is not bound, so self is indeed not going to be passed in.
If you must have access the instance in a side_effect object, you'll have to patch the function on the class with an actual function:
with patch.object(Animal, 'make_noise', new=loud):
Now make_noise is replaced by the loud function on the Animal class, so it'll be bound:
>>> with patch.object(Animal, 'make_noise', new=loud):
... dog.make_noise()
...
'WOOF!!'
The alternative is to set autospec=True, at which point mock will use a real function to mock out make_noise():
>>> with patch.object(Animal, 'make_noise', autospec=True, side_effect=loud):
... dog.make_noise()
...
'WOOF!!'
Also see the Mocking Unbound Methods section in the mock getting started section.

Calling lambda method defined in class scope (as a class attribute)

class _GhostLink(object):
toGhost = lambda filename: False
class _Mod_AllowGhosting_All(_GhostLink):
def _loop(self):
# ...
if self.__class__.toGhost(fileName) != oldGhost:...
produces:
Traceback (most recent call last):
File "bash\basher\mod_links.py", line 592, in Execute
changed = self._loop()
File "bash\basher\mod_links.py", line 587, in _loop
if self.__class__.toGhost(fileName) != oldGhost:
TypeError: unbound method <lambda>() must be called with _Mod_AllowGhosting_All instance as first argument (got Path instance instead)
while passing an instance as in if self.toGhost(fileName) != ... results in:
Traceback (most recent call last):
File "bash\basher\mod_links.py", line 592, in Execute
changed = self._loop()
File "bash\basher\mod_links.py", line 587, in _loop
if self.toGhost(fileName) != oldGhost:
TypeError: <lambda>() takes exactly 1 argument (2 given)
How come toGhost behaves as a classmethod instance method ?
EDIT: I know the difference of class,static etc methods - this is a syntactic question

Looks like you want a static method:
class _GhostLink(object):
toGhost = staticmethod(lambda filename: False)
or:
class _GhostLink(object):
#staticmethod
def toGhost(filename):
return False

The reason this happens is fundamentally that lambda and def do the same thing, except that def also assigns a variable, That is, both constructs produce a function.
The binding of a function (whether from lambda or def) into an instance method happens because functions are also descriptors; remember, in every single case:
foo = lambda (...): (...)
is identical to:
def foo(...):
return (...)
so when you say:
class _GhostLink(object):
toGhost = lambda filename: False
It's the same as if you had said:
class _GhostLink(object):
def toGhost(filename):
return False
So the moral of the story is that you should probably never use lambda as the right side of an assignment; it's not "better" or even different from using def. All it does is confuse.

Pythonic way of getting method arguments into a dict

I'm writing an api and was wondering what's the most pythonic way to do the following.
I'm writing a bunch of methods to do various web calls, the arguments mostly translate into post data keys and values.
The way I've been writing it so far is mostly like this;
def doSomething(self,param1,param2,param3):
payload={"param1":param1,
"param2":param2,
"param3":param3}
return self.request("do/something",payload)
This already has the draw back of having to repeat the parameter names which are subject to change, but this pattern isn't too bad.
The following case is what got me trying to think of a better way. In this case there are 4 optional arguments for the call
def doSomethingElse(self,param1,param2=None,param3=None,param4=None,param5=None):
payload= {"param1":param1}
if param2:
payload["param2"]= param2
if param3:
payload["param3"]= param3
# ... etc ...
self.request("do/something/else",payload)
My first thought was to do this:
def doSomethingElse(self,param1,**params):
payload = {"param1":param1}
payload.update(params)
self.request("do/something/else",payload)
or even:
def doSomethingElse(self,**payload):
self.request("do/something/else",payload)
Although the second one is nice and simple, the method can be called without the non-default argument. But in both cases I lose the method signature when using the api and the user won't know what the parameters are (I know I could write the expected signature in a docstring but I'd rather prevent misspelt keywords getting sent).
I'm thinking there must be a nice pythonic solution to do this, any ideas?
EDIT
I think a key point which I didn't make clear enough is that the arguments are getting sent in post data in a call, and I want to make sure only those keys can get sent, like in the first example of doSomethingElse, you can't send anything other than those 5 named parameters.

The Pythonic way is to name the parameters when you call the function, not in the function signature:
def doSomething(self, **kwargs):
self.request("do/something/else", kwargs)
doSomething(param1=3, param2='one', param3=4)

How about simply
def get_payload(ldict):
return {k:v for k,v in ldict.iteritems() if k != 'self' and v is not None}
class fred(object):
some_class_var = 17
def method(self, a, b=2):
payload = get_payload(locals())
print payload
which gives
>>> f = fred()
>>> f.method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() takes at least 2 arguments (1 given)
>>> f.method(2)
{'a': 2, 'b': 2}
>>> f.method(2, b=3)
{'a': 2, 'b': 3}
>>> f.method(5, b=None)
{'a': 5}
>>> f.method(2, b=3, c=19)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() got an unexpected keyword argument 'c'
>>> help(f.method)
Help on method method in module __main__:
method(self, a, b=2) method of __main__.fred instance
which I think matches your criteria. The next step would be to use a decorator (probably with either wraps or the decorator module to preserve the signature) so that payload was computed and then passed, but I don't know if #payload would be all that much better than payload = get_payload(locals()). Note that using locals() this way, it needs to be done at the start.
I second the feeling that this isn't exactly the best way to prevent unwanted nuclear attacks, though.

Something like this, perhaps:
def doSomethingElse(self, param1, **params):
payload = {"param1": param1}
for name, value in params.items():
if value is not None:
payload[name] = value
self.request("do/something/else", payload)

If you have several such functions, you can do as following:
class Requester(object):
def __init__(self, tobecalled, *allowed):
self.tobecalled = tobecalled
self.allowed = set(allowed)
def __call__(self, otherobj, **k):
for kw in k.iterkeys():
if kw not in self.allowed:
raise ValueError("unknown argument(s) given: %s" % kw)
otherobj.request(self.tobecalled, **k)
def __get__(self, outside, outsideclass):
return lambda **k: self(outside, **k)
class Outside(object):
def request(self, method, **k):
print method, k
do_one_thing = Requester("do/one/thing", 'param1', 'param2')
do_nonsense = Requester("do/nonsense", 'param3')
simple = Requester("simple")
o = Outside()
o.do_one_thing(param1=1, param2=2)
o.do_nonsense(param3=12)
o.simple()
try: o.do_one_thing(rparam1=1, param2=2)
except ValueError, e: print e
try: o.do_nonsense(gparam3=12)
except ValueError, e: print e
try: o.simple(whatever=12)
except ValueError, e: print e
What happens here? We create a Requester object which plays the role of a method: if we put it in another class (here: Outside), it can be called in a way that it also gets a reference of an object which it is called on. What I call outside here is "the outer self", as I call it now. And then, it returns a lambda which calls the object itself, just like a function does. And there, the arguments are checked for validity, and if that passes, we do the call on the "outside"'s request() method.