Currently we able to test positive cases following way:
class AuthServer(auth_grpc.AuthServicer):
def __init__(self, *args, **kwargs):
print("Initializing auth server..")
super(AuthServer, self).__init__(*args, **kwargs)
def register(self, request, context):
return auth_messages.registerResponse(uuid="Test")
With pytest fixtures looking following way:
#pytest.fixture(scope="session")
def server():
return AuthServer()
#pytest.fixture(scope="session")
def context():
return DummyGRPCTestContext()
In test case environment accessed following way:
def test_user_registration(server, context, user):
request = auth_messages.registerRequest(**user)
result = server.register(request, context)
print("RESULT %s " % result)
However, if we want test negative case and changing grpc servicer method to following:
def register(self, request, context):
context.set_code(grpc.StatusCode.ALREADY_EXISTS)
context.set_details("User already exists")
return auth_messages.registerResponse()
We're failing in errors, related to dummy context.
Where we can get grpc context, that can be easily plugged into test environment?
Contexts like this one look complex and not ready for plug and test.
grpc.ServicerContext is just an interface; within your test code you should be able to write your own implementation of it and pass that to your servicer under test.
It's true that at this time we don't provide in grpc_testing a grpc.ServicerContext implementation ready to be used by tests and passed to systems under test, but it's also not entirely clear that we could provide one that would be simply implemented and also valuable to a large number of tests. There's a large behavioral space of how servicers under test use grpc.ServicerContext objects and there's another large behavioral space of how authors write tests of servicers.
Related
I have a database handler that utilizes SQLAlchemy ORM to communicate with a database. As part of SQLAlchemy's recommended practices, I interact with the session by using it as a context manager. How can I test what a function called inside the context manager using that context manager has done?
EDIT: I realized the file structure mattered due to the complexity in introduced. I re-structured the code below to more closely mirror what the end file structure will be like, and what a common production repo in my environment would look like, with code being defined in one file and tests in a completely separate file.
For example:
Code File (delete_things_from_table.py):
from db_handler import delete, SomeTable
def delete_stuff(handler):
stmt = delete(SomeTable)
with handler.Session.begin() as session:
session.execute(stmt)
session.commit()
Test File:
import pytest
import delete_things_from_table as dlt
from db_handler import Handler
def test_delete_stuff():
handler = db_handler()
dlt.delete_stuff(handler):
# Test that session.execute was called
# Test the value of 'stmt'
# Test that session.commit was called
I am not looking for a solution specific to SQLAlchemy; I am only utilizing this to highlight what I want to test within a context manager, and any strategies for testing context managers are welcome.
After sleeping on it, I came up with a solution. I'd love additional/less complex solutions if there are any available, but this works:
import pytest
import delete_things_from_table as dlt
from db_handler import Handler
class MockSession:
def __init__(self):
self.execute_params = []
self.commit_called = False
def execute(self, *args, **kwargs):
self.execute_params.append(["call", args, kwargs])
return self
def commit(self):
self.commit_called = True
return self
def begin(self):
return self
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
pass
def test_delete_stuff(monkeypatch):
handler = db_handler()
# Parens in 'MockSession' below are Important, pass an instance not the class
monkeypatch.setattr(handler, Session, MockSession())
dlt.delete_stuff(handler):
# Test that session.execute was called
assert len(handler.Session.execute_params)
# Test the value of 'stmt'
assert str(handler.Session.execute_params[0][1][0]) == "DELETE FROM some_table"
# Test that session.commit was called
assert handler.Session.commit_called
Some key things to note:
I created a static mock instead of a MagicMock as it's easier to control the methods/data flow with a custom mock class
Since the SQLAlchemy session context manager requires a begin() to start the context, my mock class needed a begin. Returning self in begin allows us to test the values later.
context managers rely on on the magic methods __enter__ and __exit__ with the argument signatures you see above.
The mocked class contains mocked methods which alter instance variables allowing us to test later
This relies on monkeypatch (there are other ways I'm sure), but what's important to note is that when you pass your mock class you want to patch in an instance of the class and not the class itself. The parentheses make a world of difference.
I don't think it's an elegant solution, but it's working. I'll happily take any suggestions for improvement.
Given the code below, I am trying to identify the arguments sent to decorators that are used by a class.
Specifically, I am trying to identify the value 100 that is passed into the make_hyper decorator that is used in the Dog class.
Optimally, I need to be able to get this value without running the method directly since the actual method code I am working with takes a long time to run.
import inspect
def make_hyper(new_volume):
def decorator(decorated_method):
def wrapped_method(self, *args, **kwargs):
self.volume = new_volume
return decorated_method(self, *args, **kwargs)
return wrapped_method
return decorator
class Dog(object):
def __init__(self, name):
self.name = name
self.volume = 1
#make_hyper(new_volume=100)
def bark(self):
if self.volume >= 10:
print('[{}]: BARK!!'.format(self.name))
elif self.volume >= 5:
print('[{}]: Bark!'.format(self.name))
else:
print('[{}]: Bark'.format(self.name))
I have tried to use inspect.getargspec and a handful of other things and have (done my best to) scour Stackoverflow and the Internet at large but I cannot find a solution.
Any help would be greatly appreciated! Thanks for your time!
UPDATE:
Someone asked me to clarify what I was trying to do. Apologies for not being more clear.
The actual code that the above toy represents is a test automation framework that includes decorators specifying test tags:
class TestTags(object):
WIFI = 'wifi'
BLE = 'ble'
NIGHTLY = 'nightly'
REGRESSION = 'regression'
class TestBase(unittest.TestCase):
<define common test stuff and test decorators>
class ThingTester(TestBase):
#TestBase.tags(TestTags.WIFI, TestTags.BLE, TestTags.REGRESSION)
def test_all_the_things(self):
<test all the things>
# what I'm trying to get
test_tags = ???
print(test_tags) # prints out ('wifi', 'ble', 'regression')
I wrote a utility that goes through all the test modules, classes and individual tests and creates an HTML+JavaScript page that defines a Test Plan. One piece of data that I want to add to the page is what tags are associated with each test.
I found that I can make the tags decorator save an attribute to the class (i.e. self.tags = tags) but this requires the test to be run before the value is saved to the TestBase object and I need to be able to generate the test plan independently of running the tests.
I'm not sure if I'm understanding entirely what you want, but if you intend to inspect the existing test to create a plan or a report, then you probably want to store the decorator arguments in the test function, instead of the class instance, right? For example:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(myTest.tags)
>>> (TAG1, TAG2)
EDIT: Just realized that doesn't work out of the box for instance methods. You can still use it like this though:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
class TestSuite(...):
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(TestSuite.myTest.tags) # From class function
>>> (TAG1, TAG2)
myTestSuite = TestSuite()
print(myTestSuite.myTest.__func__.tags) # From instance method
>>> (TAG1, TAG2)
I am using nose test generators feature to run the same test with different contexts. Since it requires the following boiler plate for each test:
class TestSample(TestBase):
def test_sample(self):
for context in contexts:
yield self.check_sample, context
def check_sample(self, context):
"""The real test logic is implemented here"""
pass
I decided to write the following decorator:
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
The decorator is used in the following manner:
class TestSample(TestBase):
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
When the tests executed an error is thrown specifying that the attribute which is being accessed is not available. However If I change the line which calls the method to the following it works fine:
yield getattr(self, f.__name__), service
I understand that the above snippet creates a bound method where as in the first one self is passed manually to the function. However as far as my understanding goes the first snippet should work fine too. I would appreciate if anyone could clarify the issue.
The title of the question is related to calling instance methods in decorators in general but I have kept the description specific to my context.
You can use functools.partial to tie the wrapped function to self, just like a method would be:
from functools import partial
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield partial(f, self), context
return wrapper
Now you are yielding partials instead, which, when called as yieldedvalue(context), will call f(self, context).
As far as I can tell, some things don't fit together. First, your decorator goes like
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
but you use it like
#with_contexts
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
This is wrong: this calls with_context(test_sample), but you need with_context()(test_sample). So do
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
even if you don't provide the contexts argument.
Second, you decorate the wrong function: your usage shows that the test function yields the check function for each context. The function you want to wrap does the job of the check function, but you have to name it after the test function.
Applying self to a method can be done with partial as Martijn writes, but it can as well be done the way Python does it under the hood: with
method.__get__(self, None)
or maybe better
method.__get__(self, type(self))
you can achieve the same. (Maybe your original version works as well, with yielding the function to be called and the arguments to use. It was not clear to me that this is the way it works.)
I have been googling and asking on IRC to no avail. Cyclone is supposed to be a Tornado-like protocol for Twisted. But, there are no tests in the Cyclone repository and no-one has written up how to convert tornado.testing.AsyncHTTPTestCase tests to exercise code written against Cyclone.
How do I start a server to test the web interface?
Where is the self.fetch()?
Where is the documentation in Cyclone to describe how to convert an existing Tornado app?
Unfortunately, there's nothing like tornado.testing.AsyncHTTPTestCase in cyclone at the moment. Your best bet would be to use Twisted Trial to write unit tests. One (slightly kludgy) approach would be explicitly call self.listener = reactor.listenTCP(<someport>, YourCycloneApplication()) in the setUp method inside your test case and call self.listener.stopListening() in the tearDown method.
Then, inside your test methods, you could use cyclone.httpclient.fetch to fetch the pages.
This is far from ideal. But as of now, this is the only way to go.
Here is what we are currently using to test our cylcone handler like we did with tornado:
from twisted.trial.unittest import TestCase
from twisted.internet import defer, reactor
from cyclone import httpclient
# copied from tornado
_next_port = 10000
def get_unused_port():
"""Returns a (hopefully) unused port number."""
global _next_port
port = _next_port
_next_port = _next_port + 1
return port
class TxTestCase(TestCase):
def get_http_port(self):
"""Returns the port used by the HTTPServer.
A new port is chosen for each test.
"""
if self.__port is None:
self.__port = get_unused_port()
return self.__port
def setUp(self, *args, **kwargs):
self.__port = None
self._app = self.get_app()
self._listener = None
if self._app:
self._listener = reactor.listenTCP(self.get_http_port(), self._app)
return TestCase.setUp(self, *args, **kwargs)
def get_app(self):
return None
def tearDown(self):
if self._listener:
self._listener.stopListening()
#defer.inlineCallbacks
def fetch(self, url, *args, **kwargs):
response = yield httpclient.fetch('http://localhost:%s%s'%(self.get_http_port(), url), *args, **kwargs)
defer.returnValue(response)
This way, you get the fetch method back ;)
And there are no more needs to use trial.
Here is an usage example:
from twisted.internet import defer
class Test(TxTestCase):
def get_app(self):
return MyApplication()
#defer.inlineCallbacks
def some_test_method(self):
res = yield self.fetch('/path/to/resource')
self.assertEquals(200, res.code)
Hope that will help you.
I've been using testbed, webtest, and nose to test my Python GAE app, and it is a great setup. I'm now implementing something similar to Nick's great example of using the deferred library, but I can't figure out a good way to test the parts of the code triggered by DeadlineExceededError.
Since this is in the context of a taskqueue, it would be painful to construct a test that took more than 10 minutes to run. Is there a way to temporarily set the taskqueue time limit to a few seconds for the purpose of testing? Or perhaps some other way to elegantly test the execution of code in the except DeadlineExceededError block?
Abstract the "GAE context" for your code. in production provide real "GAE implementation" for testing provide a mock own that will raise the DeadlineExceededError. The test should not depend on any timeout, should be fast.
Sample abstraction (just glue):
class AbstractGAETaskContext(object):
def task_spired(): pass # this will throw exception in mock impl
# here you define any method that you call into GAE, to be mocked
def defered(...): pass
If you don't like abstraction, you can do monkey patching for testing only, also you need to define the task_expired function to be your hook for testing.
task_expired should be called during your task implementation function.
*UPDATED*This the 3rd solution:
First I want to mention that the Nick's sample implementation is not so great, the Mapper class has to many responsabilities(deferring, query data, update in batch); and this make the test hard to made, a lot of mocks need to be defined. So I extract the deferring responsabilities in a separate class. You only want to test that deferring mechanism, what actually is happen(the update, query, etc) should be handled in other test.
Here is deffering class, also this no more depends on GAE:
class DeferredCall(object):
def __init__(self, deferred):
self.deferred = deferred
def run(self, long_execution_call, context, *args, **kwargs):
''' long_execution_call should return a tuple that tell us how was terminate operation, with timeout and the context where was abandoned '''
next_context, timeouted = long_execution_call(context, *args, **kwargs)
if timeouted:
self.deferred(self.run, next_context, *args, **kwargs)
Here is the test module:
class Test(unittest.TestCase):
def test_defer(self):
calls = []
def mock_deferrer(callback, *args, **kwargs):
calls.append((callback, args, kwargs))
def interrupted(self, context):
return "new_context", True
d = DeferredCall()
d.run(interrupted, "init_context")
self.assertEquals(1, len(calls), 'a deferred call should be')
def test_no_defer(self):
calls = []
def mock_deferrer(callback, *args, **kwargs):
calls.append((callback, args, kwargs))
def completed(self, context):
return None, False
d = DeferredCall()
d.run(completed, "init_context")
self.assertEquals(0, len(calls), 'no deferred call should be')
How will look the Nick's Mapper implementation:
class Mapper:
...
def _continue(self, start_key, batch_size):
... # here is same code, nothing was changed
except DeadlineExceededError:
# Write any unfinished updates to the datastore.
self._batch_write()
# Queue a new task to pick up where we left off.
##deferred.defer(self._continue, start_key, batch_size)
return start_key, True ## make compatible with DeferredCall
self.finish()
return None, False ## make it comaptible with DeferredCall
runner = _continue
Code where you register the long running task; this only depend on the GAE deferred lib.
import DeferredCall
import PersonMapper # this inherits the Mapper
from google.appengine.ext import deferred
mapper = PersonMapper()
DeferredCall(deferred).run(mapper.run)