I'm trying to use an abstract factory in Python, minimally reproduced with the following 3 files:
test_factory.py
from factory import Factory
def test_factory():
factory = Factory.makeFactory()
product = factory.makeProduct('Hi there')
print(product)
if __name__ == '__main__':
test_factory()
factory.py
from abc import ABCMeta, abstractmethod
from product import ConcreteProduct
class Factory(metaclass=ABCMeta):
#classmethod
#abstractmethod
def makeProduct(cls):
pass
#classmethod
def makeFactory(cls):
return ConcreteFactory()
class ConcreteFactory(Factory):
#classmethod
def makeProduct(cls, message):
return ConcreteProduct(message)
product.py
class ConcreteProduct(object):
def __init__(self, message):
self._message = message
def __str__(self):
return self._message
What I'm having trouble figuring out is how to mock this code to verify that ConcreteProduct.__init__ is invoked with an appropriate value. Since the test file never sees product.py, I'm not sure how to accomplish this, or if it's even possible. I suspect that there is something more fundamentally wrong with my design here.
The simplest way is patch factory.ConcreteProduct module reference.
So your test can be (not tested):
from factory import Factory
from unittest.mock import *
#patch("factory.ConcreteProduct")
def test_factory(mock_product_factory):
mock_product = mock_product_factory.return_value
factory = Factory.makeFactory()
product = factory.makeProduct('Hi there')
self.assertIs(product, mock_product)
mock_product_factory.assert_called_with('Hi there')
if __name__ == '__main__':
test_factory()
If (and only if) ConcreteProduct reference in factory module desn't exist in your test environment you can use create=True patch's attribute to inject it.
I would like to point out that ConcreteProduct reference in factory module is already a factory. Class in Python are factories yet, this is not a typed language and the factory concept is less rigid than java. I'm coming from Java background and I still use factories even in python but they become real useful when you should manipulate the input to create a correct object, if your factory method is just an arguments pass through to a class reference consider to remove the man in the middle.
So far I've ended up with a solution that I was led to by #Michele dAmico's answer, and it is very close to his.
test_factory.py becomes:
from factory import Factory
from unittest.mock import patch
#patch('product.ConcreteProduct.__init__', return_value=None)
def test_factory(mock_init):
factory = Factory.makeFactory()
product = factory.makeProduct('Hi there')
mock_init.assert_called_with('Hi there')
if __name__ == '__main__':
test_factory()
Note that I'm patching product., not factory., so I'm basically sidestepping factory.py and mocking what I know it's going to import. I don't know how I feel about breaking encapsulation this way, but honestly that's the way I feel about mocking in general.
I preferred this over the other answer because it's a little shorter, and because according to the mock docs it can be dangerous:
By default patch will fail to replace attributes that don’t exist. If
you pass in create=True, and the attribute doesn’t exist, patch will
create the attribute for you when the patched function is called, and
delete it again afterwards. This is useful for writing tests against
attributes that your production code creates at runtime. It is off by
default because it can be dangerous. With it switched on you can write
passing tests against APIs that don’t actually exist!
I would certainly be interested in further discussion as I still have a sense that I can learn some better design methods to make this cleaner.
Related
I'm building a Python library magic_lib in which I need to instantiate a Python class (let's call it SomeClass) which is defined in the Python application that would import magic_lib.
What's the appropriate way to use/work on SomeClass when I develop magic_lib, since I don't have SomeClass in the magic_lib repo?
I'm thinking to create a dummy SomeClass like this. During packaging, I then exclude this class.
from typing import Any
class SomeClass:
def __init__(self, *arg: Any, **kwargs: Any):
pass
I'm wondering if this is the right approach. If not, any suggestions how I could approach this problem.
Thanks.
Additional thoughts: maybe I could use importlib like this?
my_module = importlib.import_module('some.module.available.in.production')
some_class = my_module.SomeClass()
Here is a more specific example:
Let's say I have two repos: workflows and magic_lib. Within workflows, it has defined a class named Task. Generally, we define tasks directly within the workflows repo. Everything works just fine. Now, let's say, I want to use magic_lib to programmatically define tasks in the workflows repo. Something like the following in the workflows repo:
from magic_lib import Generator
tasks: List[Task] = Generator().generate_tasks()
In order to do that, within magic_lib, I need to somehow have access to the class Task so that I can have it returned through the function generate_tasks(). I cannot really import Task defined in workflows from magic_lib. My question is how I can have access to Task within magic_lib.
Original question:
In python, there are decorators:
from <MY_PACKAGE> import add_method
#add_method
class MyClass:
def old_method(self):
print('Old method')
Decorators are functions which take classes/functions/... as argument:
def add_method(cls):
class Wrapper(cls):
def new_method(self):
print('New method')
return Wrapper
MyClass is passed as the cls argument to the add_method decorator function. The function can return a new class which inherits from MyClass
x = MyClass()
x.old_method()
x.new_method()
We can see that the method has been added. YAY !
So to recap, decorators are a great way to pass your user's custom class to your library. Decorators are just functions so they are easy to handle.
Modified question:
Classes can be passed to functions and methods as arguments
from magic_lib import generate_five_instances
tasks: List[Task] = generate_five_instances(Task)
def generate_five_instances(cls):
return [cls() for _ in range(5)]
If you come from another language, you might find this weird, but classes are FIRST CLASS CITIZENS in Python. That means you can assign them to variables and pass them as arguments.
I am trying to get documentation to build on a ReadTheDocs installation via Sphinx. The classes I am documenting inherit from a larger Framework, which I cannot easily install and thus would like to mock. However, Mock seems to be overly greedy in mocking also the classes I would actually like to document. The code in question is as follows:
# imports that need to be mocked
from big.framework import a_class_decorator, ..., SomeBaseClass
#a_class_decorator("foo", "bar")
class ToDocument(SomeBaseClass):
""" Lots of nice documentation here
which will never appear
"""
def a_function_that_is_being_documented():
""" This will show up on RTD
"""
I hit the point where I make sure I don't blindly mock the decorator, but am instead explicit in my Sphinx conf.py. Otherwise, I follow RTD suggestions for mocking modules:
class MyMock(MagicMock):
#classmethod
def a_class_decorator(cls, classid, version):
def real_decorator(theClass):
print(theClass)
return theClass
return real_decorator
#classmethod
def __getattr__(cls, name):
return MagicMock()
sys.modules['big.framework'] = MyMock()
Now I would expect that for the printout I get something referring to my to be documented class, e.g. <ToDocument ...>.
However, I always get a Mock for that class as well, <MagicMock spec='str' id='139887652701072'>, which of course does not have any of the documentation I am trying to build. Any ideas?
Turns out the problem was inheritance from a mocked class. Begin explicit about the base class and creating an empty class
class SomeBaseClass:
pass
to patch in in the conf.py solved the problem
Python 3.6
I'm trying to modify the behavior of a third party library.
I don't want to directly change the source code.
Considering this code below:
class UselessObject(object):
pass
class PretendClassDef(object):
"""
A class to highlight my problem
"""
def do_something(self):
# Allot of code here
result = UselessObject()
return result
I'd like to substitute my own class for UselessObject
I'd like to know if using a metaclass in my module to intercept the creation of UselessObject is a valid idea?
EDIT
This answer posted by Ashwini Chaudhary on the same question, may be of use to others. As well as the below answer.
P.S. I also discovered that 'module' level __metaclass__ does't work in python 3. So my initial question of it 'being a valid idea' is False
FWIW, here's some code that illustrates Rawing's idea.
class UselessObject(object):
def __repr__(self):
return "I'm useless"
class PretendClassDef(object):
def do_something(self):
return UselessObject()
# -------
class CoolObject(object):
def __repr__(self):
return "I'm cool"
UselessObject = CoolObject
p = PretendClassDef()
print(p.do_something())
output
I'm cool
We can even use this technique if CoolObject needs to inherit UselessObject. If we change the definition of CoolObject to:
class CoolObject(UselessObject):
def __repr__(self):
s = super().__repr__()
return "I'm cool, but my parent says " + s
we get this output:
I'm cool, but my parent says I'm useless
This works because the name UselessObject has its old definition when the CoolObject class definition is executed.
This is not a job for metaclasses.
Rather, Python allows you to do this through a technique called "Monkeypatching", in which you, at run time, substitute one object for another in run time.
In this case, you'd be changing the thirdyparty.UselessObject for your.CoolObject before calling thirdyparty.PretendClassDef.do_something
The way to do that is a simple assignment.
So, supposing the example snippet you gave on the question is the trirdyparty module, on the library, your code would look like:
import thirdyparty
class CoolObject:
# Your class definition here
thirdyparty.UselesObject = Coolobject
Things you have to take care of: that you change the object pointed by UselessObject in the way it is used in your target module.
If for example, your PretendedClassDef and UselessObject are defined in different modules, you have to procees in one way if UselessObject is imported with from .useless import UselessObject (in this case the example above is fine), and import .useless and later uses it as useless.UselessObject - in this second case, you have to patch it on the useless module.
Also, Python's unittest.mock has a nice patch callable that can properly perform a monkeypatching and undo it if by some reason you want the modification to be valid in a limited scope, like inside a function of yours, or inside a with block. That might be the case if you don't want to change the behavior of the thirdyparty module in other sections of your program.
As for metaclasses, they only would be of any use if you would need to change the metaclass of a class you'd be replacing in this way - and them they only could have any use if you'd like to insert behavior in classes that inherit from UselessObject. In that case it would be used to create the local CoolObject and you'd still perform as above, but taking care that you'd perform the monkeypatching before Python would run the class body of any of the derived classes of UselessObject, taking extreme care when doing any imports from the thirdparty library (that would be tricky if these subclasses were defined on the same file)
This is just building on PM 2Ring's and jsbueno's answers with more contexts:
If you happen to be creating a library for others to use as a third-party library (rather than you using the third-party library), and if you need CoolObject to inherit UselessObject to avoid repetition, the following may be useful to avoid an infinite recursion error that you might get in some circumstances:
module1.py
class Parent:
def __init__(self):
print("I'm the parent.")
class Actor:
def __init__(self, parent_class=None):
if parent_class!=None: #This is in case you don't want it to actually literally be useless 100% of the time.
global Parent
Parent=parent_class
Parent()
module2.py
from module1 import *
class Child(Parent):
def __init__(self):
print("I'm the child.")
class LeadActor(Actor): #There's not necessarily a need to subclass Actor, but in the situation I'm thinking, it seems it would be a common thing.
def __init__(self):
Actor.__init__(self, parent_class=Child)
a=Actor(parent_class=Child) #prints "I'm the child." instead of "I'm the parent."
l=LeadActor() #prints "I'm the child." instead of "I'm the parent."
Just be careful that the user knows not to set a different value for parent_class with different subclasses of Actor. I mean, if you make multiple kinds of Actors, you'll only want to set parent_class once, unless you want it to change for all of them.
I am bit lost while writing the test case for UserCompanyRateLimitValidation class. I am finding difficulty in mocking the class which is being instantiated from inside this class.
class UserCompanyRateLimitValidation:
def __init__(self, user_public_key):
self.adapter = UserAdapter(user_public_key)
container = self.adapter.get_user_company_rate_limit()
super(UserCompanyRateLimitValidation, self).__init__(container,\
UserCompanyRateLimitValidation.TYPE)
I have to test this class. I have written test case something like this. I have tried to mock the UserAdapter class but I am not able to do so completely.
def test_case_1():
self.user_public_key = 'TEST_USER_PUBLIC_KEY_XXXXXX1234567890XXXXX'
UserAdapter_mock = mock(UserAdapter)
when(UserAdapter_mock).get_user_company_rate_limit().\
thenReturn(get_fake_container_object())
self.test_obj = UserCompanyRateLimitValidation(self.user_public_key)
Here if you see I have mocked get_user_company_rate_limit() call from the testable function, container = self.adapter.get_user_company_rate_limit()
but I am still not able to figure out the way in which I can mock this call,
self.adapter = UserAdapter(user_public_key)
It is quite simple if you know the trick.
Creating an object in Python is very much like a function call to the class object. UserCompanyRateLimitValidation is 'invoking' UserAdapter(user_public_key). You want to stub the return value of that 'call' to return UserAdapter_mock.
You can stub this like you would stub a function in a module. The line you're missing is:
when(module_declaring_UserAdapter)\
.UserAdapter(self.user_public_key)\
.thenReturn(UserAdapter_mock)
After that, calling module_declaring_UserAdapter.UserAdapter(self.user_public_key) will return UserAdapter_mock.
Here's the link to the section in the manual: https://code.google.com/p/mockito-python/wiki/Stubbing#Modules
You have to be careful to choose the right module_declaring_UserAdapter, due to the way the from ... import ... statement works. From your code, I'd say you have to pick the module in which UserCompanyRateLimitValidation is declared.
Here is another way of looking at it. Say I have this code in which I would like to mock MyClass:
from some.module import MyClass
class AnotherClass:
def __init__(self):
self.my_class = MyClass()
One would typically call the imports as shown above. With some slight modification of the import, we can get it into a state where MyClass it can be mocked using mockito:
from some import module
class AnotherClass:
def __init__(self):
self.my_class = module.MyClass()
Then the mocking would work like so:
from some import module
when(module).MyClass().thenReturn(mock())
I am testing a class that inherits from another one very complex, with DB connection methods and a mess of dependences. I would like to mock its base class so that I can nicely play with the method defined in the subclass, but in the moment I inherit from a mocked class, the object itself turns a mock and loses all its methods.
How can I mock a superclass?
More or less the situation can be summed up in this:
import mock
ClassMock = mock.MagicMock()
class RealClass(ClassMock):
def lol(self):
print 'lol'
real = RealClass()
real.lol() # Does not print lol, but returns another mock
print real # prints <MagicMock id='...'>
This is a simplified case. What is actually happening is that RealClass extends AnotherClass, but I managed to intercept the AnotherClass and replace it with a mock.
This is something I've been struggling with for a long time, but I think I've finally found a solution.
As you already noticed, if you try to replace the base class with a Mock, the class you're attempting to test simply becomes the mock, which defeats your ability to test it. The solution is to mock only the base class's methods rather than the entire base class itself, but that's easier said than done: it can be quite error prone to mock every single method one by one on a test by test basis.
What I've done instead is created a class that scans another class, and assigns to itself Mock()s that match the methods on the other class. You can then use this class in place of the real base class in your testing.
Here is the fake class:
class Fake(object):
"""Create Mock()ed methods that match another class's methods."""
#classmethod
def imitate(cls, *others):
for other in others:
for name in other.__dict__:
try:
setattr(cls, name, Mock())
except (TypeError, AttributeError):
pass
return cls
So for example you might have some code like this (apologies this is a little bit contrived, just assume that BaseClass and SecondClass are doing non-trivial work and contain many methods and aren't even necessarily defined by you at all):
class BaseClass:
def do_expensive_calculation(self):
return 5 + 5
class SecondClass:
def do_second_calculation(self):
return 2 * 2
class MyClass(BaseClass, SecondClass):
def my_calculation(self):
return self.do_expensive_calculation(), self.do_second_calculation()
You would then be able to write some tests like this:
class MyTestCase(unittest.TestCase):
def setUp(self):
MyClass.__bases__ = (Fake.imitate(BaseClass, SecondBase),)
def test_my_methods_only(self):
myclass = MyClass()
self.assertEqual(myclass.my_calculation(), (
myclass.do_expensive_calculation.return_value,
myclass.do_second_calculation.return_value,
))
myclass.do_expensive_calculation.assert_called_once_with()
myclass.do_second_calculation.assert_called_once_with()
So the methods that exist on the base classes remain available as mocks you can interact with, but your class does not itself become a mock.
And I've been careful to ensure that this works in both python2 and python3.
This should work for you.
import mock
ClassMock = mock.MagicMock # <-- Note the removed brackets '()'
class RealClass(ClassMock):
def lol(self):
print 'lol'
real = RealClass()
real.lol() # Does not print lol, but returns another mock
print real # prints <MagicMock id='...'>
You should'nt pass an instance of the class as you did. mock.MagicMock is a class, so you pass it directly.
In [2]: inspect.isclass(mock.MagicMock)
Out[2]: True
I was facing a similar problem and was able to do this via #patch.object. See examples for patch decorators in the official python doc.
class MyTest(unittest.TestCase):
#patch.object(SomeClass, 'inherited_method')
def test_something(self, mock_method):
SomeClass.static_method()
mock_method.assert_called_with()
Just exemplifying #Akash's answer, which was the one that in fact solved my inheritance mock challenge:
#patch.object(SomeClassInheritingAnother, "inherited_method")
def test_should_test_something(self, mocked_inherited_method, mocker, caplog):
#Mocking an HTTP result status code
type(mocked_inherited_method.return_value).status_code = mocker.PropertyMock(return_value=200)
#Calling the inherited method, that should end up using the mocked method
SomeClassInheritingAnother.inherited_method()
#Considering that the request result is being logged as 'Request result: {response.status_code}'
assert "Request result: 200" in caplog.text