I've been working with some sockets lately, and while writing some unit test cases with a listening socket I repeatedly get error: [Errno 98] Address already in use.
This is some example code that shows the error.
import unittest
import socket
class TestUnit(unittest.TestCase):
def setUp(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((socket.gethostname(), 10000))
self.socket.listen(10)
self.addCleanup(self.clean)
def test_nothing(self):
self.assertEqual(False, False)
def test_something(self):
self.assertEqual(True, True)
def clean(self):
self.socket.close()
It seems to occur when one of the tests throw an exception. Without an exception it works as expected. But that kinda makes the test useless since all tests after the first that throws an exception also throw an exception.
socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
should help
Basically a closed socket is not immediately freed by the stack. Hence if you try to reuse it (even in the scenario when you have a single bind socket, but you close and restart application) immediately, you would see the same error. REUSEADDR allows binding the same socket again.
However, if your socket is in a timed wait state and you try the same destination, it would fail.
You should also read the man page for this socket option to understand it's limitations.
SO_REUSEADDR on SO
Related
I have written a TCP/UDP intercepting proxy using Twisted and I want to add some unit tests to it. I want to setup an echo protocol, then send some data through my proxy, then check the returned response.
However, it seems like even for a simple test using a socket (let aside my intercepting proxy) to connect to the echoer, the reactor desn't seem to be spawned after setUp - the test hangs forever. If I add a timeout to the socket then a timeout exception is raised. I even tried to connect with ncat to make sure is not the manually created socket to blame - the echoer is listening indeed but I receive no echoed data back to the ncat client.
The test code I use is the following
import pytest
import socket
from twisted.trial import unittest
from twisted.internet import reactor, protocol
class EchoTCP(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoTCPFactory(protocol.Factory):
protocol = EchoTCP
class TestTCP(unittest.TestCase):
"""Twisted has its own unittest class
https://twistedmatrix.com/documents/15.2.0/core/howto/trial.html
"""
def setUp(self):
self.iface = "127.0.0.1"
self.data = b"Hello, World!"
# Setup twised echoer
self.port = reactor.listenTCP(
8080,
EchoTCPFactory(),
interface=self.iface
)
def tearDown(self):
self.port.stopListening()
def test_echo(self):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((self.iface, self.port.getHost().port))
sent = sock.send(self.data)
data = sock.recv(1024)
sock.close()
assert data == self.data
To run it I use the following command
PYTHONPATH="${PWD}" trial --reactor=default mymodule
The output is the following and stays like this until I kill the process
mymodule.test.test_network
TestTCP
test_echo ...
It seems like I'm missing something regarding how the reactor works. I've looked for similar examples but couldn't get it working.
How should I write the test to get the expected behavior?
It turned out I must run the test methods as Deffered, using inlineCallbacks, so they are called when the reactor is running. To test this behavior I've used the following snippet
from twisted.internet.defer import inlineCallbacks
# [...]
def check_reactor(self):
# time.sleep(100)
return reactor.running
#inlineCallbacks
def test_reactor(self):
reactor_running = yield threads.deferToThread(self.check_reactor)
assert reactor_running == True
...which makes the test successfully complete
mymodule.test.test_network
TestTCP
test_reactor ... [OK]
-------------------------------------------------------------------------------
Ran 1 tests in 0.007s
PASSED (successes=1)
If I enable the sleep(100) in the calledback method, and connect with ncat in that timespan, the data that I send to the listening port is indeed echoed back
I'm using a relatively simple python execution, using OSC modules, in order to 'Send' code, from an application to an other.
import OSC
import threading
#------OSC Server-------------------------------------#
receive_address = '127.0.0.1', 9002
# OSC Server. there are three different types of server.
s = OSC.ThreadingOSCServer(receive_address)
# define a message-handler function for the server to call.
def printing_handler(addr, tags, stuff, source):
if addr=='/coordinates':
print "Test", stuff
s.addMsgHandler("/coordinates", printing_handler)
def main():
# Start OSCServer
print "Starting OSCServer"
st = threading.Thread(target=s.serve_forever)
st.start()
main()
Runned once, will work just fine, an listen on port 9002.
But, runned twice, will return ERROR:
socket.error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
My goal is to:
Be able to query on active thread's port
Close them
I've tried the following...
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = s.connect_ex(('127.0.0.1', 900socket2))
print 'RESULT: ', result
s.close()
But giving me unsuccessful result. (Returns 10061 both for active and unactive port's thread)
Python ยป Documentation socketserver.BaseServer:
shutdown()
Tell the serve_forever() loop to stop and wait until it does.
server_close()
Clean up the server. May be overridden.
I'm trying to pass a TCP connection to a Twisted subprocess with adoptStreamConnection, but I can't figure out how to get the Process disposed in the main process after doing that.
My desired flow looks like this:
Finish writing any data the Protocol transport has waiting
When we know the write buffer is empty send the AMP message to transfer the socket to the subprocess
Dispose the Protocol instance in the main process
I tried doing nothing, loseConnection, abortConnection, and monkey patching _socketClose out and using loseConnection. See code here:
import weakref
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.python.sendmsg import getsockfam
from twisted.internet.protocol import Factory, Protocol
import twisted.internet.abstract
class EchoProtocol(Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(Factory):
protocol = EchoProtocol
class TransferProtocol(Protocol):
def dataReceived(self, data):
self.transport.write('main process still listening!: %s' % (data))
def connectionMade(self):
self.transport.write('this message should make it to the subprocess\n')
# attempt 1: do nothing
# everything works fine in the adopt (including receiving the written message), but old protocol still exists (though isn't doing anything)
# attempt 1: try calling loseConnection
# we lose connection before the adopt opens the socket (presumably TCP disconnect message was sent)
#
# self.transport.loseConnection()
# attempt 2: try calling abortConnection
# result is same as loseConnection
#
# self.transport.abortConnection()
# attempt 3: try monkey patching the socket close out and calling loseConnection
# result: same as doing nothing-- adopt works (including receiving the written message), old protocol still exists
#
# def ignored(*args, **kwargs):
# print 'ignored :D'
#
# self.transport._closeSocket = ignored
# self.transport.loseConnection()
reactor.callLater(0, adopt, self.transport.fileno())
class ServerFactory(Factory):
def buildProtocol(self, addr):
p = TransferProtocol()
self.ref = weakref.ref(p)
return p
f = ServerFactory()
def adopt(fileno):
print "does old protocol still exist?: %r" % (f.ref())
reactor.adoptStreamConnection(fileno, getsockfam(fileno), EchoFactory())
port = 1337
endpoint = TCP4ServerEndpoint(reactor, port)
d = endpoint.listen(f)
reactor.run()
In all cases the Protocol object still exists in the main process after the socket has been transferred. How can I clean this up?
Thanks in advance.
Neither loseConnection nor abortConnection tell the reactor to "forget" about a connection; they close the connection, which is very different; they tell the peer that the connection has gone away.
You want to call self.transport.stopReading() and self.transport.stopWriting() to remove the references to it from the reactor.
Also, it's not valid to use a weakref to test for the remaining existence of an object unless you call gc.collect() first.
As far as making sure that all the data has been sent: the only reliable way to do that is to have an application-level acknowledgement of the data that you've sent. This is why protocols that need a handshake that involves changing protocols - say, for example, STARTTLS - have a specific handshake where the initiator says "I'm going to switch" (and then stops sending), then the peer says "OK, you can switch now". Another way to handle that in this case would be to hand the data you'd like to write to the subprocess via some other channel, instead of passing it to transport.write.
I write a simple program in python, with asyncore and threading. I want to implement a asynchorous client without blocking anything, like this:
How to handle asyncore within a class in python, without blocking anything?
Here is my code:
import socket, threading, time, asyncore
class Client(asyncore.dispatcher):
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.connect((host, port))
mysocket = Client("",8888)
onethread = threading.Thread(target=asyncore.loop)
onethread.start()
# time.sleep(5)
mysocket.send("asfas\n")
input("End")
Now a exception will be throwed in send("asfas\n"), because I didn't open any server.
I think the exception in send function will call the handle_error function and won't affect the main program, but most of the time it crashes the whole program, and sometimes it works! And if I uncomment the time.sleep(5), it will only crash the thread. Why does it behave like this? Could I write a program that won't crash the whole program and don't use time.sleep() ? Thanks!
Error message:
Traceback (most recent call last):
File "thread.py", line 13, in <module>
mysocket.send("asfas\n")
File "/usr/lib/python2.7/asyncore.py", line 374, in send
result = self.socket.send(data)
socket.error: [Errno 111] Connection refused
First of all, I would suggest not using the old asyncore module but to look into more
modern and more efficient solutions: gevent, or going along the asyncio module (Python 3.4),
which has been backported somehow to Python 2.
If you want to use asyncore, then you have to know:
be careful when using sockets created in one thread (the main thread, in your case), and dispatched by another thread (managed by "onethread", in your case), sockets cannot be shared like this between threads it is not threadsafe objects by themselves
for the same reason, you can't use the global map created by default in asyncore module, you have to create a map by thread
when connecting to a server, connection may not be immediate you have to wait for it to be connected (hence your "sleep 5"). When using asyncore, "handle_write" is called when
socket is ready to send data.
Here is a newer version of your code, hopefully it fixes those issues:
import socket, threading, time, asyncore
class Client(threading.Thread, asyncore.dispatcher):
def __init__(self, host, port):
threading.Thread.__init__(self)
self.daemon = True
self._thread_sockets = dict()
asyncore.dispatcher.__init__(self, map=self._thread_sockets)
self.host = host
self.port = port
self.output_buffer = []
self.start()
def run(self):
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.connect((self.host, self.port))
asyncore.loop(map=self._thread_sockets)
def send(self, data):
self.output_buffer.append(data)
def handle_write(self):
all_data = "".join(self.output_buffer)
bytes_sent = self.socket.send(all_data)
remaining_data = all_data[bytes_sent:]
self.output_buffer = [remaining_data]
mysocket = Client("",8888)
mysocket.send("asfas\n")
If you have only 1 socket by thread (i.e a dispatcher's map with size 1), there is no
point using asyncore at all. Just use a normal, blocking socket in your threads. The
benefit of async i/o comes with a lot of sockets.
EDIT: answer has been edited following comments.
I'm trying to test some code that reconnects to a server after a disconnect. This works perfectly fine outside the tests, but it fails to acknowledge that the socket has disconnected when running the tests.
I'm using a Gevent Stream Server to mock a real listening server:
import gevent.server
from gevent import queue
class TestServer(gevent.server.StreamServer):
def __init__(self, *args, **kwargs):
super(TestServer, self).__init__(*args, **kwargs)
self.sockets = {}
def handle(self, socket, address):
self.sockets[address] = (socket, queue.Queue())
socket.sendall('testing the connection\r\n')
gevent.spawn(self.recv, address)
def recv(self, address):
socket = self.sockets[address][0]
queue = self.sockets[address][1]
print 'Connection accepted %s:%d' % address
try:
for data in socket.recv(1024):
queue.put(data)
except:
pass
def murder(self):
self.stop()
for sock in self.sockets.iteritems():
print sock
sock[1][0].shutdown(socket.SHUT_RDWR)
sock[1][0].close()
self.sockets = {}
def run_server():
test_server = TestServer(('127.0.0.1', 10666))
test_server.start()
return test_server
And my test looks like this:
def test_can_reconnect(self):
test_server = run_server()
client_config = {'host': '127.0.0.1', 'port': 10666}
client = Connection('test client', client_config, get_config())
client.connect()
assert client.socket_connected
test_server.murder()
#time.sleep(4) #tried sleeping. no dice.
assert not client.socket_connected
assert client.server_disconnect
test_server = run_server()
client.reconnect()
assert client.socket_connected
It fails at assert not client.socket_connected.
I detect for "not data" during recv. If it's None, then I set some variables so that other code can decide whether or not to reconnect (don't reconnect if it was a user_disconnect and so on). This behavior works and has always worked for me in the past, I've just never tried to make a test for it until now. Is there something odd with socket connections and local function scopes or something? it's like the connection still exists even after stopping the server.
The code I'm trying to test is open: https://github.com/kyleterry/tenyks.git
If you run the tests, you will see the one I'm trying to fix fail.
Trying to run a unit test with a real socket is a tough row to hoe. It's going to be tricky as only one set of tests can run at a time, as the server port will be used, and it's going to be slow as the sockets get set up and torn down. To top it off if this is really a unit test you don't want to test the socket, just the code that's using the socket.
If you mock the socket calls you can throw exceptions willy nilly from the mocked code and ensure that the code making use of the socket does the right thing. You don't need a real socket to ensure that the class under test does the right thing, you can fake it if you can wrap the socket calls in an object. Pass in a reference to the socket object when constructing your class and you're ready to go.
My suggestion is to wrap the socket calls in a class that supports sendall, recv, and all the methods you call on the socket. Then you can swap out the actual Socket class with a TestReconnectSocket (or whatever) and run your tests.
Take a look at mox, a python mocking framework.
Vague response, but my immediate reaction would be that your recv() call is blocking and keeping the socket alive - have you tried making the socket non-blocking, and catching the error on close instead?
One thing to keep in mind when testing sockets like this, is that operating systems don't like to reopen a socket soon after it has been in use. You can set a socket option to tell it to go ahead and reuse it anyways. Right after you create the socket set the socket's option:
mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Hopefully this will fix your issue. You may have to do it on both the server and client side depending on which one is giving you the problems.
you are calling shutdown(socket.SHUT_RDWR) so this doesn't seem like a problem with recv blocking.
however, you are using gevent.socket.socket.recv, so please check your gevent version, there is an issue with recv() that causes it to block if the underlying file descriptor is closed (version < v0.13.0)
you may still need gevent.sleep() to do cooperative yield and give the client an opportunity to exit the recv() call.