We Keep Coding

unittest unable to import class from pickle (AttributeError: Can't get attribute...)

I need a unittest to load a previously saved class in a pickle. However, when I load the pickle in the unittest (out of unittest works), it raises the error:
AttributeError: Can't get attribute 'Foo' on <module
'unittest.main' from '...\unittest\main.py'>
Code example to save the class (I save this code in run_and_save_class.py):
from pickle import dump
from pickle import load
from pickle import HIGHEST_PROTOCOL
class Foo(object):
def __init__(self):
self.bar = None
self.file_out = "./out.pkl"
def save_class(self):
with open(self.file_out, "wb") as file_out:
dump(self, file_out, protocol=HIGHEST_PROTOCOL)
def load_class(self):
with open(self.file_out, "rb") as file_out:
cls = load(file_out)
return cls
if __name__ == "__main__":
cls = Foo()
cls.bar = "saving a bar"
cls.save_class()
Code to test the class (I save this code in unittest_class.py):
import unittest
from run_and_save_class import Foo
class ClassValidation(unittest.TestCase):
def __init__(self, *args, **kwargs):
print("init")
self.cls = Foo
self.instance = Foo().load_class()
print("class loaded")
unittest.TestCase.__init__(self, *args, **kwargs)
def test_anything(self):
pass
I run in Anaconda Prompt:
python run_and_save_class.py
python -m unittest -v unittest_class.py
The latter is the one that raises the error.
However, this works in a notebook.
from run_and_save_class import Foo
cls = Foo().load_class()
I don't understand why it doesn't in a unittest.

The problem is that pickle saves the object relative to __main__, where dump was called (via save_class). To load the same object, you have to provide the same environment - a workaround is to add the class to __main__ in your test, so that pickle can find it:
import __main__
class ClassValidation(unittest.TestCase):
def __init__(self, *args, **kwargs):
__main__.Foo = Foo
self.cls = Foo
self.instance = Foo().load_class()
unittest.TestCase.__init__(self, *args, **kwargs)
def test_anything(self):
self.assertEqual("saving a bar", self.instance.bar)

Try to instanciate the Foo object in the ClassValidation in this way : self.cls = Foo()

Mock a Class, but not one of its functions

When I import MyApp from app.py, a instance of the SerialConnection class
is created immediately. I want to mock the SerialConnection class but I still need a function from within this SerialConnection class.
app.py
# A module that creates strings that is sent via SerialConnection
from myserial import SerialConnection
class MyApp():
global ser
ser = SerialConnection() # ---> Needs to be mocked
def __init__(self):
pass
#ser.decorator # ---> Needs to be by-passed
def myfunc(self):
return 'testing1234'
myserial.py
# A module to write and read to a serial/UART buffer
from functools import wraps
import serial
class SerialConnection():
def __init__(self):
""" Initilize the Serial instance. """
self.ser = serial.Serial(port=2, baudrate=9600)
def decorator(self, func):
#wraps(func)
def wrapper_func(*args):
return func(*args)
return wrapper_func
test_app.py
# This file tests the function "myfunc" from "MyApp" class.
from patch import mock
#patch('myserial.SerialConnection')
def test_myfunc(mock_class):
# I can now import MyApp successfuly...
from app import MyApp
# ..but I need to bypass the decorator myserial.SerialConnection.decorator
# do I add a by-pass decorator to the "mock_class"?
# myfunc() turns out to be mocked and not a real function
assert MyApp().myfunc() == 'testing1234'

Python decorator staticmethod object not callable

Hi I'm trying to create a decorator but I'm getting an error of staticmethod object not callable below is my code
from db.persistence import S3Mediator
from sqlalchemy.ext.declarative import declarative_base
import logging
from functools import wraps
Base = declarative_base()
def s3(func):
#wraps(func)
def wrapper(*args, **kwargs):
try:
s3client = S3Mediator.get_s3_connection()
kwargs["s3client"] = s3client
retval = func(*args, **kwargs) #### an error is raised here
except Exception as e:
raise e
return retval
return wrapper
And here is the mediator that instantiate the s3 object
import boto3
import logging
class S3Mediator(object):
s3_client = None
def __init__(self, host, access_key, secret):
self.client = boto3.client(
's3',
aws_access_key_id= access_key,
aws_secret_access_key= secret
)
S3Mediator.s3_client = self.client
#staticmethod
def get_s3_connection():
return S3Mediator.s3_client
Now S3Mediator is already instantiate at the app.py now I'm trying to use this decorator as
#s3
#staticmethod
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here
Any idea why its returning a staticmethod object not callable and how to fix this

Ok found the cause of the problem. I put the #staticmethod below my decorator that's why my decorator is thinking that all method of the decorator is static. I just change the
#s3
#staticmethod
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here
to this
#staticmethod
#s3
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here

Python mock a dependancy object

I have a python class (MyClass) I am unit testing. It has a external dependency which creates a database, I would like to mock that object and check it was called.
from entities_database import DBCreator
class MyClass():
def __init__(self):
self.db = DBCreator()
def create(self, name):
value = self.db.create_db(name)
I would like to test the init function, so see that a mock DBCreator called. And then to test the create function to check that create_db() was called with "name", and it returned "name" to value.
I am not sure how to go about this, so far I have this:
from entities_database import DBCreator
from test_unit import MyClass
import unittest
import mock
class MyClassTest(unittest.TestCase):
#mock.patch('entities_database.DBCreator', autospec=True)
def test_myclass_init(self, dbcreator_mock):
creator = mock.create_autospec(DBCreator)
dbcreator_mock.return_value = creator
myclass = MyClass()
assert(dbcreator_mock.called)
Which results in:
F
======================================================================
FAIL: test_myclass_init (unit_test.MyClassTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/paul/Data/TSC/Code/parltrack_eu/venv/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/Users/paul/Data/TSC/Code/parltrack_eu/unit_test.py", line 32, in test_myclass_init
assert(dbcreator_mock.called)
AssertionError
----------------------------------------------------------------------
Ran 1 test in 0.036s
FAILED (failures=1)
How can I fix my code?
---- UPDATE ----
Apart from the solution below with dependency injection, one can also do the following with patch, as suggested below #Goyo
from entities_database import DBCreator
from test_unit import MyClass
import unittest
import mock
class MyClassTest(unittest.TestCase):
#mock.patch('test_unit.DBCreator', autospec=True)
def test_myclass_init(self, mock_db):
'''
Test the initialization of MyClass
Test that DBCreator is initialized
'''
creator = mock.create_autospec(DBCreator)
mock_db.return_value = creator
# Assert that the DBCreator is initialised
myclass = MyClass()
assert(mock_db.called)
#mock.patch('test_unit.DBCreator', autospec=True)
def test_myclass_create(self, mock_db):
'''
Test the myclass.create() function
and assert it calls db.create_db() with the correct
argument
'''
name = 'unittest'
myclass = MyClass()
myclass.create(name)
# Assert that create_db was called with name
myclass.db.create_db.assert_called_with(name)

Patching is tricky. You have to patch the object in the same namespace where the SUT will look it up. In this case probably you want #mock.patch('test_unit.DBCreator', autospec=True).
Using dependency injection avoids this kind of problems and makes things more explicit and clear:
class MyClass():
def __init__(self, db):
self.db = db
def create(self, name):
value = self.db.create_db(name)
Then in your test:
class MyClassTest(unittest.TestCase):
def test_myclass_init(self):
db = mock.Mock()
myclass = MyClass(db)
self.assertEqual(myclass.db, db)
def test_myclass_create(self):
db = mock.Mock()
myclass = MyClass(db)
name = mock.Mock()
myclass.create(name)
myclass.db.create_db.assert_called_once_with(name)

I am not able to try with your example code. But you should try to mock create_db function not DBCreator class.

Mock entire python class

I'm trying to make a simple test in python, but I'm not able to figure it out how to accomplish the mocking process.
This is the class and def code:
class FileRemoveOp(...)
#apply_defaults
def __init__(
self,
source_conn_keys,
source_conn_id='conn_default',
*args, **kwargs):
super(v4FileRemoveOperator, self).__init__(*args, **kwargs)
self.source_conn_keys = source_conn_keys
self.source_conn_id = source_conn_id
def execute (self, context)
source_conn = Connection(conn_id)
try:
for source_conn_key in self.source_keys:
if not source_conn.check_for_key(source_conn_key):
logging.info("The source key does not exist")
source_conn.remove_file(source_conn_key,'')
finally:
logging.info("Remove operation successful.")
And this is my test for the execute function:
#mock.patch('main.Connection')
def test_remove_execute(self,MockConn):
mock_coon = MockConn.return_value
mock_coon.value = #I'm not sure what to put here#
remove_operator = FileRemoveOp(...)
remove_operator.execute(self)
Since the execute method try to make a connection, I need to mock that, I don't want to make a real connection, just return something mock. How can I make that? I'm used to do testing in Java but I never did on python..

First it is very important to understand that you always need to Mock where it the thing you are trying to mock out is used as stated in the unittest.mock documentation.
The basic principle is that you patch where an object is looked up,
which is not necessarily the same place as where it is defined.
Next what you would need to do is to return a MagicMock instance as return_value of the patched object. So to summarize this you would need to use the following sequence.
Patch Object
prepare MagicMock to be used
return the MagicMock we've just created as return_value
Here a quick example of a project.
connection.py (Class we would like to Mock)
class Connection(object):
def execute(self):
return "Connection to server made"
file.py (Where the Class is used)
from project.connection import Connection
class FileRemoveOp(object):
def __init__(self, foo):
self.foo = foo
def execute(self):
conn = Connection()
result = conn.execute()
return result
tests/test_file.py
import unittest
from unittest.mock import patch, MagicMock
from project.file import FileRemoveOp
class TestFileRemoveOp(unittest.TestCase):
def setUp(self):
self.fileremoveop = FileRemoveOp('foobar')
#patch('project.file.Connection')
def test_execute(self, connection_mock):
# Create a new MagickMock instance which will be the
# `return_value` of our patched object
connection_instance = MagicMock()
connection_instance.execute.return_value = "testing"
# Return the above created `connection_instance`
connection_mock.return_value = connection_instance
result = self.fileremoveop.execute()
expected = "testing"
self.assertEqual(result, expected)
def test_not_mocked(self):
# No mocking involved will execute the `Connection.execute` method
result = self.fileremoveop.execute()
expected = "Connection to server made"
self.assertEqual(result, expected)

I found that this simple solution works in python3: you can substitute a whole class before it is being imported for the first time. Say I have to mock class 'Manager' from real.manager
class MockManager:
...
import real.manager
real.manager.Manager = MockManager
It is possible to do this substitution in init.py if there is no better place.
It may work in python2 too but I did not check.