ReactorNotRestartable when launching two equivalent unittest with twisted and trial - python

I've two test classes (TrialTest1 and TrialTest2) written in two files (test_trial1.py and test_trial2.py) mostly identical (the only difference is the class name):
from twisted.internet import reactor
from twisted.trial import unittest
class TrialTest1(unittest.TestCase):
def setUp(self):
print("setUp()")
def test_main(self):
print("test_main")
reactor.callLater(1, self._called_by_deffered1)
reactor.run()
def _called_by_deffered1(self):
print("_called_by_deffered1")
reactor.callLater(1, self._called_by_deffered2)
def _called_by_deffered2(self):
print("_called_by_deffered2")
reactor.stop()
def tearDown(self):
print("tearDown()")
When I run each test idepently, everything is fine. But when I launch both I've the following output:
setUp()
test_main
_called_by_deffered1
_called_by_deffered2
tearDown()
setUp()
test_main
tearDown()
Error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 137, in maybeDeferred
result = f(*args, **kw)
File "/usr/lib/python2.7/site-packages/twisted/internet/utils.py", line 203, in runWithWarningsSuppressed
reraise(exc_info[1], exc_info[2])
File "/usr/lib/python2.7/site-packages/twisted/internet/utils.py", line 199, in runWithWarningsSuppressed
result = f(*a, **kw)
File "/home/kartoch/works/python/netkython/tests/test_twisted_trial2.py", line 13, in test_main
reactor.run()
File "/usr/lib/python2.7/site-packages/twisted/internet/base.py", line 1191, in run
self.startRunning(installSignalHandlers=installSignalHandlers)
File "/usr/lib/python2.7/site-packages/twisted/internet/base.py", line 1171, in startRunning
ReactorBase.startRunning(self)
File "/usr/lib/python2.7/site-packages/twisted/internet/base.py", line 683, in startRunning
raise error.ReactorNotRestartable()
ReactorNotRestartable
Error
DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x8d6482c [0.98535490036s] called=0 cancelled=0 TrialTest2._called_by_deffered1()>
Process finished with exit code 0
It seems the reactor is not switched off correctly after the first test. Does anyone know where is the problem ? It seems tearDown() is called to early (before _called_by_deffered1 in the second test), maybe a fix would be to use deferTearDown (not documented method of trial unittest).
EDIT
One of the solution proposed was to remove reactor.run() and reactor.stop() because a reactor is not restartable and you have only one reactor instance for all test by default:
class TrialTest1(unittest.TestCase):
def setUp(self):
print("setUp()")
def test_main(self):
print("test_main")
reactor.callLater(1, self._called_by_deffered1)
def _called_by_deffered1(self):
print("_called_by_deffered1")
reactor.callLater(1, self._called_by_deffered2)
def _called_by_deffered2(self):
print("_called_by_deffered2")
def tearDown(self):
print("tearDown()")
But when removing calls to such methods, my tests fail without executing _called_by_deffered methods:
setUp()
test_main
tearDown()
Error
DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x94967ec [0.99936413765s] called=0 cancelled=0 TrialTest1._called_by_deffered1()>
setUp()
test_main
tearDown()
Error
DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x94968cc [0.99958896637s] called=0 cancelled=0 TrialTest2._called_by_deffered1()>
If I want to use only one instance of reactor shared between tests, how _called_by_deffered methods could be part of the test (i.e. executed before tearDown) ?

The reactor is not restartable. There are two obvious options for your to pursue for writing your tests.
One is to use the global reactor. trial starts and stops it - your tests never have to call reactor.run or reactor.stop (and they never should). It is accessible in the usual way, from twisted.internet import reactor.
The other is to use a new reactor instance per test. There are some test-oriented reactor instances in twisted.test.proto_helpers (that is the only part of twisted.test that is a public, supported interface by the way). MemoryReactor and StringTransport get you most of the way towards being able to test network interactions. twisted.internet.task.Clock helps you out with testing time-based events.

With the help of Jean-Paul, this page and this question, I've been able to fix the problem using twisted.internet.task.deferLater(). To summarize the point i was looking for: if a test method returning a deferred, the 'tearDown()' method will be called only when all deferreds are fired.
This is the code:
from twisted.trial import unittest
from twisted.internet import reactor, task
class TrialTest1(unittest.TestCase):
def setUp(self):
print("setUp()")
def test_main(self):
print("test_main()")
return task.deferLater(reactor, 1, self._called_by_deffered1)
def _called_by_deffered1(self):
print("_called_by_deffered1()")
return task.deferLater(reactor, 1, self._called_by_deffered2)
def _called_by_deffered2(self):
print("_called_by_deffered2()")
def tearDown(self):
print("tearDown()")
Output:
setUp()
test_main()
// (1s wait)
_called_by_deffered1()
// (1s wait)
_called_by_deffered2()
tearDown()

Related

Function is not called using mock.patch

I am trying to test the Class BluetoothClient which connects to a BluetoothSocket. To avoid using real sockets I just want to test that the connect() method from the socket is called with the right parameters. Using mock.patch to replace the imported bluetooth module in my bluetooth_control module doesn't work out like expected.
As I see it, the connect() method is called but the assertion tells me otherwise.
Code:
Unit Under Test (bluetooth_control.py):
import bluetooth
class BluetoothClient(object):
def __init__(self):
self.address="98:D3:31:B2:EF:32"
self.port=1
def establishConnection(self):
self.createSocket()
self.connect()
def createSocket(self):
self.sock=bluetooth.BluetoothSocket( bluetooth.RFCOMM )
def connect(self):
print "connect: sock="+str(self.sock)
self.sock.connect((self.address, self.port))
Test (bluetooth_control_test.py):
import unittest
import mock
import bluetooth_control
import bluetooth
class TestShelf(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.bc = bluetooth_control.BluetoothClient()
print "setUp"
def tearDown(self):
self.shelf = None
print "tearDown"
#mock.patch('bluetooth_control.bluetooth')
def testEstablishConnection(self,mock_bluetooth):
self.bc.establishConnection()
print "testEstablishConnection sock="+str(self.bc.sock)
mock_bluetooth.connect().assert_called_with(self.bc.sock,("98:D3:31:B2:EF:32",1))
if __name__ == "__main__":
unittest.main()
Output:
setUp
connect: sock=<MagicMock name='bluetooth.BluetoothSocket()' id='140433322111504'>
testEstablishConnection sock=<MagicMock name='bluetooth.BluetoothSocket()' id='140433322111504'>
FtearDown
======================================================================
FAIL: testEstablishConnection (__main__.TestShelf)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mock.py", line 1201, in patched
return func(*args, **keywargs)
File "bluetooth_control_test.py", line 21, in testEstablishConnection
mock_bluetooth.connect().assert_called_with(self.bc.sock,("98:D3:31:B2:EF:32",1))
File "/usr/lib/python2.7/site-packages/mock.py", line 831, in assert_called_with
raise AssertionError('Expected call: %s\nNot called' % (expected,))
AssertionError: Expected call: mock(<MagicMock name='bluetooth.BluetoothSocket()' id='140433322111504'>, ('98:D3:31:B2:EF:32', 1))
Not called
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (failures=1)
After looking at the problem again two days later I found the stupid mistakes I made. I had to patch the actual method and remove the falsely added brackets at the assertion.
I am not going to delete this question so that maybe it will help someone to avoid these mistakes.
Test
import unittest
import mock
import bluetooth_control
import bluetooth
class TestShelf(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.bc = bluetooth_control.BluetoothClient()
print "setUp"
def tearDown(self):
self.shelf = None
print "tearDown"
#mock.patch('bluetooth_control.bluetooth.BluetoothSocket.connect')
def testEstablishConnection(self,mock_connect):
self.bc.establishConnection()
print "testEstablishConnection sock="+str(self.bc.sock)
mock_connect.assert_called_with(("98:D3:31:B2:EF:32",1))
if __name__ == "__main__":
unittest.main()

python: httplib.CannotSendRequest when nesting threaded SimpleXMLRPCServers

I am intermittently receiving a httplib.CannotSendRequest exception when using a chain of SimpleXMLRPCServers that use the SocketServer.ThreadingMixin.
What I mean by 'chain' is the following:
I have a client script which uses xmlrpclib to call a function on a SimpleXMLRPCServer. That server, in turn, calls another SimpleXMLRPCServer. I realise how convoluted that sounds, but there are good reasons that this architecture has been selected, and I don't see a reason it shouldn't be possible.
(testclient)client_script ---calls-->
(middleserver)SimpleXMLRPCServer ---calls--->
(finalserver)SimpleXMLRPCServer --- does something
If I do not use SocketServer.ThreadingMixin then this issue doesn't occur (but I need the requests to be multi-threaded so this doesn't help.)
If I only have a single level of services (ie just client script calling final server directly) this doesn't happen.
I have been able to reproduce the issue in the simple test code below. There are three snippets:
finalserver:
import SocketServer
import time
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
class AsyncXMLRPCServer(SocketServer.ThreadingMixIn,SimpleXMLRPCServer): pass
# Create server
server = AsyncXMLRPCServer(('', 9999), SimpleXMLRPCRequestHandler)
server.register_introspection_functions()
def waste_time():
time.sleep(10)
return True
server.register_function(waste_time, 'waste_time')
server.serve_forever()
middleserver:
import SocketServer
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
import xmlrpclib
class AsyncXMLRPCServer(SocketServer.ThreadingMixIn,SimpleXMLRPCServer): pass
# Create server
server = AsyncXMLRPCServer(('', 8888), SimpleXMLRPCRequestHandler)
server.register_introspection_functions()
s = xmlrpclib.ServerProxy('http://localhost:9999')
def call_waste():
s.waste_time()
return True
server.register_function(call_waste, 'call_waste')
server.serve_forever()
testclient:
import xmlrpclib
s = xmlrpclib.ServerProxy('http://localhost:8888')
print s.call_waste()
To reproduce, the following steps should be used:
Run python finalserver.py
Run python middleserver.py
Run python testclient.py
While (3) is still running, run another instance of python testclient.py
Quite often (almost every time) you will get the error below the first time you try to run step 4. Interestingly, if you immediately try to run step (4) again the error will not occur.
Traceback (most recent call last):
File "testclient.py", line 6, in <module>
print s.call_waste()
File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1297, in single_request
return self.parse_response(response)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1473, in parse_response
return u.close()
File "/usr/lib64/python2.7/xmlrpclib.py", line 793, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "<class 'httplib.CannotSendRequest'>:">
The internet appears to say that this exception can be caused by multiple calls to httplib.HTTPConnection.request without intervening getresponse calls. However, the internet doesn't discuss this in the context of SimpleXMLRPCServer. Any pointers in the direction of resolving the httplib.CannotSendRequest issue would be appreciated.
===========================================================================================
ANSWER:
Okay, I'm a bit stupid. I think I was staring at the code for too protracted a period of time that I missed the obvious solution staring me in the face (quite literally, because the answer is actually in the actual question.)
Basically, the CannotSendRequest occurs when an httplib.HTTPConnection is interrupted by an intervening 'request' operation. Each httplib.HTTPConnection.request must be paired with a .getresponse() call. If that pairing is interrupted by another request operation, the second request will produce the CannotSendRequest error. so:
connection = httplib.HTTPConnection(...)
connection.request(...)
connection.request(...)
will fail because you have two requests on the same connection before any getresponse is called.
Linking that back to my question:
the only place in the three programs where such connections are being made are in the serverproxy calls.
the problem only occurs during threading, so it's likely a race condition.
the only place a serverproxy call is shared is in middleserver.py
The solution then, is obviously to have each thread create it's own serverproxy. The fixed version of middleserver is below, and it works:
import SocketServer
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
import xmlrpclib
class AsyncXMLRPCServer(SocketServer.ThreadingMixIn,SimpleXMLRPCServer): pass
# Create server
server = AsyncXMLRPCServer(('', 8888), SimpleXMLRPCRequestHandler)
server.register_introspection_functions()
def call_waste():
# Each call to this function creates its own serverproxy.
# If this function is called by concurrent threads, each thread
# will safely have its own serverproxy.
s = xmlrpclib.ServerProxy('http://localhost:9999')
s.waste_time()
return True
server.register_function(call_waste, 'call_waste')
server.serve_forever()
Since this version results in each thread having its own xmlrpclib.serverproxy, there is no risk of the same instance of serverproxy invoking HTTPConnection.request more than once in succession. The programs work as intended.
Sorry for the bother.
Okay, I'm a bit stupid. I think I was staring at the code for to protracted a period of time that I missed the obvious solution staring me in the face (quite literally, because the answer is actually in the actual question.)
Basically, the CannotSendRequest occurs when an httplib.HTTPConnection is interrupted by an intervening 'request' operation. Basically, each httplib.HTTPConnection.request must be paired with a .getresponse() call. If that pairing is interrupted by another request operation, the second request will produce the CannotSendRequest error. so:
connection = httplib.HTTPConnection(...)
connection.request(...)
connection.request(...)
will fail because you have two requests on the same connection before any getresponse is called.
Linking that back to my question:
the only place in the three programs where such connections are being made are in the serverproxy calls.
the problem only occurs during threading, so it's likely a race condition.
the only place a serverproxy call is shared is in middleserver.py
The solution then, is obviously to have each thread create it's own serverproxy. The fixed version of middleserver is below, and it works:
import SocketServer
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
import xmlrpclib
class AsyncXMLRPCServer(SocketServer.ThreadingMixIn,SimpleXMLRPCServer): pass
# Create server
server = AsyncXMLRPCServer(('', 8888), SimpleXMLRPCRequestHandler)
server.register_introspection_functions()
def call_waste():
# Each call to this function creates its own serverproxy.
# If this function is called by concurrent threads, each thread
# will safely have its own serverproxy.
s = xmlrpclib.ServerProxy('http://localhost:9999')
s.waste_time()
return True
server.register_function(call_waste, 'call_waste')
server.serve_forever()
Since this version results in each thread having its own xmlrpclib.serverproxy, there is no risk of the serverproxy invoking HTTPConnection.request more than once in succession. The programs work as intended.
Sorry for the bother.

BadYieldError in tornado.gen when using Corduroy couchdb async library

I'm trying to use Corduroy to talk to a CouchDB asynchronously using the python Tornado Web Server.
The code below comes from the Corduroy guide, with a couple of alterations.
import tornado.web
from corduroy import Database, NotFound, relax
people_db = Database('people')
class RelaxedHello(tornado.web.RequestHandler):
#relax
def get(self, user_id):
try:
doc = yield people_db.get(user_id)
self.write('hello %s'%(doc['name']))
except NotFound:
self.write('hello whoever you are')
self.finish()
application = tornado.web.Application([
(r'/hi/([^/]+)', RelaxedHello),
]).listen(1920)
tornado.ioloop.IOLoop.instance().start()
The problem I'm having is that I receive a BadYieldError despite finding the couch document perfectly well. I suspect it's something to do with the tornado.gen module not being set up properly (or something?). Using corduroy without the #relax decorator, and with a callback works fine.
Traceback (most recent call last):
File "c:\env\pymeals_tornado\lib\site-packages\tornado\web.py", line 1074, in wrapper
return method(self, *args, **kwargs)
File "c:\env\pymeals_tornado\lib\site-packages\corduroy\__init__.py", line 43, in _r_e_l_a_x_
return gen.engine(_func_)(*args, **kwargs)
File "c:\env\pymeals_tornado\lib\site-packages\tornado\gen.py", line 107, in wrapper
runner.run()
File "c:\env\pymeals_tornado\lib\site-packages\tornado\gen.py", line 319, in run
yielded = self.gen.throw(*exc_info)
File "test.py", line 10, in get
doc = yield people_db.get(user_id)
BadYieldError: yielded unknown object <Document 65d936ee54417e46479a908f7a0038ef[2] {name:Colin}>
Reading the code for relax and for get(), it looks to me like corduroy's 3-year-old code is not designed for the most modern Tornado idioms. Specifically, get takes a callback, rather than returning a Future as modern Tornado APIs do.
That means you'll have to use an older style with gen.Task like:
from tornado import gen
class RelaxedHello(tornado.web.RequestHandler):
#relax
def get(self, user_id):
try:
doc = yield gen.Task(people_db.get, user_id)
self.write('hello %s'%(doc['name']))
except NotFound:
self.write('hello whoever you are')
self.finish()
I haven't tested this, let me know how it goes. More info on gen.Task is in the Tornado documentation.

Using Nose to test txmongo dependent code

I want to use nose to test an application that I am writing using twisted and txmongo. I can't even get simple use cases like the following working:
from nose.twistedtools import reactor, deferred, threaded_reactor
import logging
from twisted.internet import defer
import txmongo
log = logging.getLogger("common.test.test_db")
conn = txmongo.lazyMongoConnectionPool('localhost', 27017, 4)
#deferred()
def test_mongo():
tdb = conn.test
#defer.inlineCallbacks
def cb(oid):
assert oid
obj = yield tdb.test.find({"_id":oid})
log.error("In callback")
assert obj
d = tdb.test.save({"s":1, "b":2})
d.addCallback(cb)
return d
However, this always return the following:
E
======================================================================
ERROR: common.test.test_db.test_mongo
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Volumes/Users/jce/.pyenv/celery/lib/python2.6/site-packages/nose/case.py", line 186, in runTest
self.test(*self.arg)
File "/Volumes/Users/jce/.pyenv/celery/lib/python2.6/site-packages/nose/twistedtools.py", line 138, in errback
failure.raiseException()
File "/Volumes/Users/jce/.pyenv/celery/lib/python2.6/site-packages/twisted/python/failure.py", line 326, in raiseException
raise self.type, self.value, self.tb
RuntimeWarning: not connected
----------------------------------------------------------------------
Ran 1 test in 0.006s
FAILED (errors=1)
I tried manually adding a threaded_reactor() call, but it didn't help.
edit
I removed the "lazy" connections, and modified the code, and now it works... I'm still curious as to why the "lazy" didn't work. The working code is as follows:
dbconn = txmongo.MongoConnectionPool('localhost', 27017, 4)
#deferred()
def test_mongo():
#defer.inlineCallbacks
def cb(conn):
tdb = conn.test
oid = yield tdb.test.save({"s":1, "b":2})
assert oid
log.error(str(oid))
obj = yield tdb.test.find({"_id":oid})
assert obj
log.error(str(obj))
dbconn.addCallback(cb)
return dbconn
MongoConnectionPool will return a deferred, which is fired when the connection is established passing the connection handler as argument to the callback. You should conn = yield MongoConnectionPool().
lazyMongoConnectionPool will return the connection handler directly, without waiting for the connection to be established.
Lazy is usually used by web servers and other services that doesn't require immediate connection when your service starts. If you want to do so, don't use the lazy method.

Referencing other modules in atexit

I have a function that is responsible for killing a child process when the program ends:
class MySingleton:
def __init__(self):
import atexit
atexit.register(self.stop)
def stop(self):
os.kill(self.sel_server_pid, signal.SIGTERM)
However I get an error message when this function is called:
Traceback (most recent call last):
File "/usr/lib/python2.5/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/home/commando/Development/Diploma/streaminatr/stream/selenium_tests.py", line 66, in stop
os.kill(self.sel_server_pid, signal.SIGTERM)
AttributeError: 'NoneType' object has no attribute 'kill'
Looks like the os and signal modules get unloaded before atexit is called. Re-importing them solves the problem, but this behaviour seems weird to me - these modules are imported before I register my handler, so why are they unloaded before my own exit handler runs?
There are no strong guarantees about the order in which things are destroyed at program termination time, so it's best to ensure atexit-registered functions are self contained. E.g., in your case:
class MySingleton:
def __init__(self):
import atexit
atexit.register(self.stop)
self._dokill = os.kill
self._thesig = signal.SIGTERM
def stop(self):
self._dokill(self.sel_server_pid, self._thesig)
This is preferable to re-importing modules (which could conceivably cause slowdown of program termination and even unending loops, though that risk is lesser for "system-supplied" modules such as os).

Categories