I'm creating TCP client socket with Twisted. I need to check connection status in a loop interval in connectionMade method.
from twisted.internet import reactor, protocol
class ClientProtocol(protocol.Protocol):
def connectionMade(self):
while not thread_obj.stopped.wait(10):
print ('ping')
self.transport.write(b'test') # Byte value
For check connection losing, i manually disconnect my network And I checked some variables after that as bellow:
print (self.connected)
print (self.transport.connector.state)
print (self.transport.connected)
print (self.transport.reactor.running)
print (self.transport.socket._closed)
print (self.factory.protocol.connected)
print (self._writeDissconnected)
But any variables value didn't change after disconnecting my network :(
My question is: which variables will be set when the connection lost? I mean how can i check the connection status and if it's disconnect, how can i reconnect that?
override connectionLost method to catch disconnection.
to official docs
Edit about reconnection:
Reconnection mostly is a logical decision. You may want to add logic between 'connectionLost' and 'reconnect'.
Anyway,
You can use ReconnectingClientFactory for better code.
ps: using factory pattern at reconnection is best way to keep code clean and smart.
class MyEchoClient(Protocol):
def dataReceived(self, data):
someFuncProcessingData(data)
self.transport.write(b'test')
class MyEchoClientFactory(ReconnectingClientFactory):
def buildProtocol(self, addr):
print 'Connected.'
return MyEchoClient()
def clientConnectionLost(self, connector, reason):
print 'Lost connection. Reason:', reason
ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
def clientConnectionFailed(self, connector, reason):
print 'Connection failed. Reason:', reason
ReconnectingClientFactory.clientConnectionFailed(self, connector,
reason)
Related
I'm trying to pass a TCP connection to a Twisted subprocess with adoptStreamConnection, but I can't figure out how to get the Process disposed in the main process after doing that.
My desired flow looks like this:
Finish writing any data the Protocol transport has waiting
When we know the write buffer is empty send the AMP message to transfer the socket to the subprocess
Dispose the Protocol instance in the main process
I tried doing nothing, loseConnection, abortConnection, and monkey patching _socketClose out and using loseConnection. See code here:
import weakref
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.python.sendmsg import getsockfam
from twisted.internet.protocol import Factory, Protocol
import twisted.internet.abstract
class EchoProtocol(Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(Factory):
protocol = EchoProtocol
class TransferProtocol(Protocol):
def dataReceived(self, data):
self.transport.write('main process still listening!: %s' % (data))
def connectionMade(self):
self.transport.write('this message should make it to the subprocess\n')
# attempt 1: do nothing
# everything works fine in the adopt (including receiving the written message), but old protocol still exists (though isn't doing anything)
# attempt 1: try calling loseConnection
# we lose connection before the adopt opens the socket (presumably TCP disconnect message was sent)
#
# self.transport.loseConnection()
# attempt 2: try calling abortConnection
# result is same as loseConnection
#
# self.transport.abortConnection()
# attempt 3: try monkey patching the socket close out and calling loseConnection
# result: same as doing nothing-- adopt works (including receiving the written message), old protocol still exists
#
# def ignored(*args, **kwargs):
# print 'ignored :D'
#
# self.transport._closeSocket = ignored
# self.transport.loseConnection()
reactor.callLater(0, adopt, self.transport.fileno())
class ServerFactory(Factory):
def buildProtocol(self, addr):
p = TransferProtocol()
self.ref = weakref.ref(p)
return p
f = ServerFactory()
def adopt(fileno):
print "does old protocol still exist?: %r" % (f.ref())
reactor.adoptStreamConnection(fileno, getsockfam(fileno), EchoFactory())
port = 1337
endpoint = TCP4ServerEndpoint(reactor, port)
d = endpoint.listen(f)
reactor.run()
In all cases the Protocol object still exists in the main process after the socket has been transferred. How can I clean this up?
Thanks in advance.
Neither loseConnection nor abortConnection tell the reactor to "forget" about a connection; they close the connection, which is very different; they tell the peer that the connection has gone away.
You want to call self.transport.stopReading() and self.transport.stopWriting() to remove the references to it from the reactor.
Also, it's not valid to use a weakref to test for the remaining existence of an object unless you call gc.collect() first.
As far as making sure that all the data has been sent: the only reliable way to do that is to have an application-level acknowledgement of the data that you've sent. This is why protocols that need a handshake that involves changing protocols - say, for example, STARTTLS - have a specific handshake where the initiator says "I'm going to switch" (and then stops sending), then the peer says "OK, you can switch now". Another way to handle that in this case would be to hand the data you'd like to write to the subprocess via some other channel, instead of passing it to transport.write.
I am new to Twisted python (Twisted 12.x, python 2.6.x, unfortunately I have to use the older version), and I am running client/server in which the server echoes a simple message from the client (example from chap 2 of Twisted O'reilly book) I am running server in a terminal, and then client in a separate terminal. But client and server are stuck (not returning). What is failing?
server:
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
reactor.listenTCP(50000, EchoFactory())
reactor.run()
client:
from twisted.internet import reactor, protocol
class EchoClient(protocol.Protocol):
def connectionMade(self):
self.transport.write("Hello, world!")
def dataReceived(self, data):
print "Server said:", data
self.transport.loseConnection()
class EchoFactory(protocol.ClientFactory):
def buildProtocol(self, addr):
return EchoClient()
def clientConnectionFailed(self, connector, reason):
print "Connection failed."
reactor.stop()
def clientConnectionLost(self, connector, reason):
print "Connection lost."
reactor.stop()
reactor.connectTCP("localhost", 50000, EchoFactory())
reactor.run()
Your dataReceived, clientConnectionFailed, and clientConnectionLost methods are not indented, which means they are just free functions that are never being called, rather than overriding methods on Protocol or ClientFactory. So your client code will get the default dataReceived implementation, which is "do nothing".
Also, by the way, Python 2.6 is out of security support and will not receive updates. Please upgrade to version 2.7.9 at least. You should also upgrade to a recent version of Twisted, especially if this is for new code; there is really no legitimate reason to use software this old; it is dangerous and irresponsible to do so.
I have a very simple client program:
class EchoClient(Int32StringReceiver):
def connectionMade(self):
print 'connection made.'
str = "<request><commands><dbtest /></commands></request>"
self.sendString(str)
print 'message sent'
def stringReceived(self, line):
print "receive:", line
self.transport.loseConnection()
class EchoClientFactory(ClientFactory):
def buildProtocol(self, addr):
return EchoClient()
def clientConnectionFailed(self, connector, reason):
print 'connection failed:', reason.getErrorMessage()
reactor.stop()
def clientConnectionLost(self, connector, reason):
print 'connection lost:', reason.getErrorMessage()
reactor.stop()
def main():
factory = EchoClientFactory()
reactor.connectTCP('localhost', 3604, factory)
reactor.run()
I connect to a Java service implemented in Apache CXF (and some proprietary company code).
It connects fine, sends the message, the service receives it and produces a response.
Sadly, this client does not wait for the server to produce its message, but disconnects right after the message is sent. So the output I get from the client is:
connection made.
message sent
connection lost: Connection was closed cleanly.
And of course the Java service throws an exception complaining about the connection being already closed.
What am I missing here?
EDIT: adding this line shows that the message is received, as it prints it correctly:
def dataReceived(self, data):
print(data)
self.transport.loseConnection()
So the real problem is that the stringReceived() function is not called. Maybe I have the wrong signature for this function?
I'm onto something here:
def lengthLimitExceeded(self, length):
print('length limit exceeded: {}'.format(length))
prints:
length limit exceeded: 2147483793
length limit exceeded: 2147483793
which is 0x80000091, so it seems that our propietary API is implementing the NString protocol in a strange way (maybe uses the MSB for something else).
I've decided to dip my toe into the world of asynchronous python with the help of twisted. I've implemented some of the examples from the documentation, but I'm having a difficult time finding an example of the, very simple, client I'm trying to write.
In short I'd like a client which establishes a tcp connection with a server and then sends simple "\n" terminated string messages off of a queue object to the server. The server doesn't ever respond with any messages so my client is fully unidirectional. I /think/ that what I want is some combination of this example and the twisted.internet.protocols.basic.LineReceiver convenience protocol. This feels like it should be just about the simplest thing one could do in twisted, but none of the documentation or examples I've seen online seem to fit quite right.
What I have done is not used a Queue but I am illustrating the code that sends a line, once a connection is made. There are bunch of print stuff that will help you understand on what is going on.
Usual import stuff:
from twisted.web import proxy
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.protocol import ReconnectingClientFactory
from twisted.protocols import basic
from twisted.python import log
import sys
log.startLogging(sys.stdout)
You create a protocol derived from line receiver, set the delimiter.
In this case, I simply write a string "www" once the connection is made.
The key thing is to look at protocol interface at twisted.internet.interface.py and understand the various methods of protocol and what they do and when they are called.
class MyProtocol(basic.LineReceiver):
#def makeConnection(self, transport):
# print transport
def connectionLost(self, reason):
print reason
self.sendData = False
def connectionMade(self):
print "connection made"
self.delimiter = "\n"
self.sendData = True
print self.transport
self.sendFromQueue()
def sendFromQueue(self):
while self.sendData:
msg = dataQueue.get()
self.sendLine(msg)
# you need to handle empty queue
# Have another function to resume
Finally, A protocol factory that will create a protocol instance for every connection.
Look at method : buildProtcol.
class myProtocolFactory():
protocol = MyProtocol
def doStart(self):
pass
def startedConnecting(self, connectorInstance):
print connectorInstance
def buildProtocol(self, address):
print address
return self.protocol()
def clientConnectionLost(self, connection, reason):
print reason
print connection
def clientConnectionFailed(self, connection, reason):
print connection
print reason
def doStop(self):
pass
Now you use a connector to make a connection:
reactor.connectTCP('localhost', 50000, myProtocolFactory())
reactor.run()
I ran this and connected it to an server that simply prints what it receives and hence send no ack back. Here is the output:
1286906080.08 82 INFO 140735087148064 __main__ conn_made: client_address=127.0.0.1:50277
1286906080.08 83 DEBUG 140735087148064 __main__ created handler; waiting for loop
1286906080.08 83 DEBUG 140735087148064 __main__ handle_read
1286906080.08 83 DEBUG 140735087148064 __main__ after recv
'www\n'
Recieved: 4
The above example if not fault tolerant. To reconnect , when a connection is lost, you can derive your protocol factory from an existing twisted class - ReconnectingClientFactory.
Twisted has almost all the tools that you would need :)
class myProtocolFactory(ReconnectingClientFactory):
protocol = MyProtocol
def buildProtocol(self, address):
print address
return self.protocol()
For further reference
I suggest that you read : http://krondo.com/?page_id=1327
[Edited: As per comment below]
I'm trying to write a simple load-balancer. It works ok till one of servers (BalanceServer) doesn't close connection then...
Client (ReverseProxy) disconnects but the connection in with BalanceServer stays open.
I tried to add callback (#3) to ReverseProxy.connectionLost to close the connection with one of the servers as I do with closing connection when server disconnects (clientLoseConnection), but at that time the ServerWriter is Null and I cannot terminate it at #1 and #2
How can I ensure that all connections are closed when one of sides disconnects? I guess that also some kind of timeout here would be nice when both client and one of servers hang, but how can I add it so it works on both connections?
from twisted.internet.protocol import Protocol, Factory, ClientCreator
from twisted.internet import reactor, defer
from collections import namedtuple
BalanceServer = namedtuple('BalanceServer', 'host port')
SERVER_LIST = [BalanceServer('127.0.0.1', 8000), BalanceServer('127.0.0.1', 8001)]
def getServer(servers):
while True:
for server in servers:
yield server
# this writes to one of balance servers and responds to client with callback 'clientWrite'
class ServerWriter(Protocol):
def sendData(self, data):
self.transport.write(data)
def dataReceived(self, data):
self.clientWrite(data)
def connectionLost( self, reason ):
self.clientLoseConnection()
# callback for reading data from client to send it to server and get response to client again
def transferData(serverWriter, clientWrite, clientLoseConnection, data):
if serverWriter:
serverWriter.clientWrite = clientWrite
serverWriter.clientLoseConnection = clientLoseConnection
serverWriter.sendData(data)
def closeConnection(serverWriter):
if serverWriter: #1 this is null
#2 So connection is not closed and hangs there, till BalanceServer close it
serverWriter.transport.loseConnection()
# accepts clients
class ReverseProxy(Protocol):
def connectionMade(self):
server = self.factory.getServer()
self.serverWriter = ClientCreator(reactor, ServerWriter)
self.client = self.serverWriter.connectTCP( server.host, server.port )
def dataReceived(self, data):
self.client.addCallback(transferData, self.transport.write,
self.transport.loseConnection, data )
def connectionLost(self, reason):
self.client.addCallback(closeConnection) #3 adding close doesn't work
class ReverseProxyFactory(Factory):
protocol = ReverseProxy
def __init__(self, serverGenerator):
self.getServer = serverGenerator
plainFactory = ReverseProxyFactory( getServer(SERVER_LIST).next )
reactor.listenTCP( 7777, plainFactory )
reactor.run()
You may want to look at twisted.internet.protocols.portforward for an example of hooking up two connections and then disconnecting them. Or just use txloadbalancer and don't even write your own code.
However, loseConnection will never forcibly terminate the connection if there is never any traffic going over it. So if you don't have an application-level ping or any data going over your connections, they may still never shut down. This is a long-standing bug in Twisted. Actually, the longest-standing bug. Perhaps you'd like to help work on the fix :).