I am trying to implement a simple server reply in Perspective Broker.
Possible implementation (please suggest others if possible):
Client requests server to execute a server method, Server executes then replies (by executing a client method whose sole purpose is to print a message):
[Client-side]:
class ClientPrint(pb.Referenceable):
def remote_clientprint(self, message):
print "Printing the message from the server: ", message
[Server-side]:
class RootServerObject(pb.Root):
def remote_OneFunc(self, ...):
...
print "Now sending the reply..."
*get ClientPrint object?*
clientprintobj.callRemote("clientprint", "this is the reply!")
How can I implement the grabbing of client-side objects? Is there a better way to implement server replies than grabbing a client-side object and calling a print-only client method?
Here is the full code where I am trying to implement the replies:
[Client-side]:
from twisted.internet import reactor
from twisted.spread import pb
class Client():
def __init__(self, addr, port, spec):
self.addr = None
self.port = None
self.SomeData = None
def connect(self, addr, port):
factory = pb.PBClientFactory()
reactor.connectTCP(addr, port, factory)
def1 = factory.getRootObject()
def1.addCallbacks(self.got_obj, self.err_obj)
def got_obj(self, rootsrvobj):
print "Got root server obj:", rootsrvobj
self.server = rootsrvobj
def2 = self.server.callRemote("SomeFunc", SomeData)
def err_obj(self, reason):
print "Error getting root server obj:", reason
self.quit()
def cmdsub(addr, port, SomeData):
c = Client(addr, port, SomeData)
c.connect(addr, port)
[Server-side]:
class RootServerObject(pb.Root):
def __init__(self):
self.DataOut = None
def remote_SomeFunc(self, SomeData):
self.DataOut = hash(SomeData)
print "Now sending reply..."
*implement a reply?*
Perhaps there are some more advanced Twisted (or Twisted PB) features that will make this simpler.
Documentation: https://twistedmatrix.com/documents/12.3.0/core/howto/pb-usage.html#auto3
Thanks.
The simplest way to do this is to take the client-side object that the server needs to use and pass it to the server. Almost any solution I can think of has this at its core.
Change your client's got_obj method to be something more like this:
def got_obj(self, rootsrvobj):
print "Got root server obj:", rootsrvobj
self.server = rootsrvobj
def2 = self.server.callRemote("SomeFunc", self, SomeData)
And change the implementation of remote_SomeFunc to be something more like this:
def remote_SomeFunc(self, client, SomeData):
self.DataOut = hash(SomeData)
print "Now sending reply..."
client.callRemote("client_print", "Here is your reply")
You might want to investigate Twisted Cred as a more structured way to manage references to your client object - but cred is just building on this exact feature of Perspective Broker to provide its more abstract, more featureful interface.
However, notice that I said "almost" above...
Keep in mind that Twisted's implementation of Perspective Broker has well-integrated support for Deferreds. If a remote_ method returns a Deferred then no response will be sent to the method call until the Deferred fires and then the result will be sent as the result of the method call. You might consider putting the logic of client_print into a callback on the Deferred returned by self.server.callRemote("SomeFunc", SomeData) and making the server's remote_SomeFunc return the reply, either synchronously or asynchronously (as a Deferred).
Related
I will be hosting a service that will be acting somewhat like a proxy for something I am a client to.
So I want my ProxyService (a twisted.protocol server) to takes lots of actors (clients). On the server side of things, I need a global connection (only need 1 connection to it for all clients) to an ExistingService (code I didn't write, and I'm a client to it).
When the ExistingService says something interesting, I need to broadcast it to all actors.
When an actor says something to my ProxyService, I need to check if it looks good to me. If it does, I need to inform the ExistingService.
I think I know how to solve this using global variables, but just wondering if better way to push the messages.
You have the basic design well established.
It's a basic "man in the middle" approach.
There are many ways to implement it, but this should get you started:
from twisted.internet import endpoints, protocol, reactor
class ProxyClient(protocol.Protocol):
def connectionMade(self):
print('[x] proxy connection made to server')
self.factory.proxy_proto = self
def connectionLost(self, reason):
print('[ ] proxy connection to server lost: {0}'.format(reason))
self.factory.proxy_proto = None
def dataReceived(self, data):
print('==> received {0} from server'.format(data))
print('<== transmitting data to all actors')
for actor in self.factory.actors:
actor.transport.write(data)
class Actor(protocol.Protocol):
def connectionMade(self):
print('[x] actor connection established')
self.factory.actors.add(self)
def connectionLost(self, reason):
print('[ ] actor disconnected: {0}'.format(reason))
self.factory.actors.remove(self)
def dataReceived(self, data):
print('==> received {0} from actor'.format(data))
proxy_connection = self.factory.proxy_factory.proxy_proto
if proxy_connection is not None:
print('<== transmitting data to server through the proxy')
proxy_connection.transport.write(data)
else:
print('[ ] proxy connection to server has not been established')
def setup_servers():
PROXY_HOST = '127.0.0.1'
PROXY_PORT = 9000
proxy_factory = protocol.ClientFactory()
proxy_factory.protocol = ProxyClient
proxy_factory.proxy_proto = None
proxy_factory.actors = set()
proxy_client = endpoints.TCP4ClientEndpoint(reactor, port=PROXY_PORT, host=PROXY_HOST)
proxy_client.connect(proxy_factory)
ACTOR_HOST = '127.0.0.1'
ACTOR_PORT = 8000
actor_factory = protocol.Factory()
actor_factory.protocol = Actor
actor_factory.proxy_factory = proxy_factory
actor_factory.actors = proxy_factory.actors
actor_server = endpoints.TCP4ServerEndpoint(reactor, port=ACTOR_PORT, interface=ACTOR_HOST)
actor_server.listen(actor_factory)
def main():
setup_servers()
reactor.run()
main()
The core logic that allows the data received from the server to be proxied to actors is proxy_factory.actors = set() and actor_factory.actors = proxy_factory.actors.
Most "list-like" containers, for lack of better words, are "global" and this example gives context into each connection's factory objects.
When an actor connects to the server, an Actor protocol is appended to the set and when data is received, each protocol in the set will get the data.
See the respective dataReceived() methods of each protocol object on how that works.
The example above doesn't use global variables at all, but that's not to say that you couldn't use them.
See how far you can get using this method of passing around variables that give context into other objects.
Also, certain situations weren't explicitly handled, such as caching received data in the event the server or actors haven't connected yet.
Hopefully there's enough information here for you to determine the best course of action based on your needs.
There's some room for streamlining the syntax to make it shorter as well.
As a side note. An alternative to global variables is picobox. It's a dependency injector library but I've found that it satisfies most my needs when I require parameters from external sources.
I'm attempting to do the following:
connect as client to an existing websocket
process the streaming data received from this socket, and publish it on another websocket
I'm using twisted and autobahn to do so. I have managed to have the two parts working separately, by deriving a WebSocketClientProtocol for the client, and deriving an ApplicationSession in the second. The two run with the same reactor.
I am not sure however as to how to make them communicate. I would like to send a message on my server when the client receives a message, but I don't know how to get the running instance of the WebSocketClientProtocol...
Perhaps this isn't the right approach to do this either. What's the right way to do this?
I've been trying to solve similiar problem recently, here's what worked:
f = XLeagueBotFactory()
app = Application(f)
reactor.connectTCP("irc.gamesurge.net", 6667, f)
reactor.listenTCP(port, app, interface=host)
^ This is in if __name__ == "__main__":
class Application(web.Application):
def __init__(self, botfactory):
self.botfactory = botfactory
Define the instance as self, then in my instance I was sending it to another handler for http post request (using cyclone)
class requestvouch(web.RequestHandler):
def __init__(self, application, request, **kwargs):
super(requestvouch, self).__init__(application, request, **kwargs)
self.botfactory = application.botfactory
def msg(self, channel, msg):
bot = self.botfactory.getProtocolByName("XLeagueBot")
sendmsg(bot, channel, msg) # function that processed the msg through stuff like encoding and logging and then sent it to bot.msg() function that posts it to IRC (endpoint in my case)
def post(self):
msg = "What I'm sending to the protocol of the other thing"
self.msg("#xleague", msg)
Now the important part comes in factory
class XLeagueBotFactory(protocol.ClientFactory):
protocol = XLeagueBot
def __init__(self):
self.protocols = {}
def getProtocolByName(self, name):
return self.protocols.get(name)
def registerProtocol(self, protocol):
self.protocols[protocol.nickname] = protocol
def unregisterProtocol(self, protocol):
del self.protocols[protocol.nickname]
Finally in my client class:
class XLeagueBot(irc.IRCClient):
nickname = "XLeagueBot"
def connectionMade(self):
irc.IRCClient.connectionMade(self)
self.factory.registerProtocol(self)
def connectionLost(self, reason):
self.factory.unregisterProtocol(self)
irc.IRCClient.connectionLost(self, reason)
I'm not entirely sure that this code is perfect, or bugfree, but it should +- tell you how to deal with calling instance of protocol class. The problem afaik comes from name of instance protocol being generated inside of it's factory and not being sent elsewhere.
Working with Python, Twisted, Redis and txredisapi.
How can I get the SubscriberProtocol for subscribe and unsubscribe to channels after the connection has been made?
I guess I need to get the instance of the SubscriberProtocol and then I can use "subscribe" and "unsubscribe" methods but don't know how to get it.
Code example:
import txredisapi as redis
class RedisListenerProtocol(redis.SubscriberProtocol):
def connectionMade(self):
self.subscribe("channelName")
def messageReceived(self, pattern, channel, message):
print "pattern=%s, channel=%s message=%s" %(pattern, channel, message)
def connectionLost(self, reason):
print "lost connection:", reason
class RedisListenerFactory(redis.SubscriberFactory):
maxDelay = 120
continueTrying = True
protocol = RedisListenerProtocol
Then from outside of these classes:
# I need to sub/unsub from here! (not from inside de protocol)
protocolInstance = RedisListenerProtocol # Here is the problem
protocolInstance.subscribe("newChannelName")
protocolInstance.unsubscribe("channelName")
Any suggestion?
Thanks!
The next code solves the problem:
#defer.inlineCallbacks
def subUnsub():
deferred = yield ClientCreator(reactor, RedisListenerProtocol).connectTCP(HOST, PORT)
deferred.subscribe("newChannelName")
deferred.unsubscribe("channelName")
Explanation:
Use "ClientCreator" to get an instance of SubscriberProtocol inside a function with the flag "#defer.inlineCallbacks" and don't forget the "yield" keyword for wait to complete the deferred data. Then you can use this deferred to suscribe and unsubscribe.
In my case I forgot the yield keyword, so the deferred wasn't complete and the method suscribe and unsubscribe didn't work.
connecting = ClientCreator(reactor, RedisListenerProtocol).connectTCP(HOST, PORT)
def connected(listener):
listener.subscribe("newChannelName")
listener.unsubscribe("channelName")
connecting.addCallback(connected)
I have to send data only to a connection, as I can do?
server:
import asyncore, socket, threading
class EchoHandler(asyncore.dispatcher_with_send):
def __init__(self,sock):
asyncore.dispatcher.__init__(self,sock=sock);
self.out_buffer = ''
def handle_read(self):
datos = self.recv(1024);
if datos:
print(datos);
self.sock[0].send("signal");
class Server(asyncore.dispatcher):
def __init__(self,host='',port=6666):
asyncore.dispatcher.__init__(self);
self.create_socket(socket.AF_INET, socket.SOCK_STREAM);
self.set_reuse_addr();
self.bind((host,port));
self.listen(1);
def handle_accept(self):
self.sock,self.addr = self.accept();
if self.addr:
print self.addr[0];
handler = EchoHandler(self.sock);
def handle_close(self):
self.close();
cliente = Server();
asyncore.loop()
this line is an example fails, but I want to send data to zero sock:
self.sock[0].send("probando");
for example, if I have 5 sockets choose who to send the data
Explanation
You tried to get sock from list and execute its send method. This causes error, because EchoHandler neither has sock attribute nor it's a list of sockets. The right method is to get instance of EchoHandler you want (based on, eg. IP address, or slots assigned by some user-defined protocol) and then use its send method - here (with dispatcher_with_send) its also better to use special buffer for that than send.
EchoHandler instantion is created on every accept of connection - from then it is an established channel for communication with the given host. Server listens for any non-established connection, while EchoHandlers use socks (given by Server in handle_accept) for established ones, so there are as many EchoHandler instances as connections.
Solution
You need to make some list of connections (EchoHandler instantions; we'll use buffer, not socket's send() directly) and give them opportunity to delete their entries on close:
class Server(asyncore.dispatcher):
def __init__(self, host='', port=6666):
...
self.connections = []
def handle_accept(self):
...
handler = EchoHandler(self.sock, self);
self.connections.append(self.sock)
...
def remove_channel(self, sock):
if sock in self.connections:
self.connections.remove(sock)
class EchoHandler(asyncore.dispatcher_with_send):
def __init__(self, sock, server):
...
self.server = server
def handle_read(self):
datos = self.recv(1024);
if datos:
print(datos);
self.out_buffer += 'I echo you: ' + datos
def handle_close(self):
self.server.remove_channel(self)
self.close()
EchoHandler is now aware of server instance and can remove its socket from list. This echo example is now fully functional, and with working socket list we can proceed to asynchronous sending.
But, at this point you can use this list as you wanted - cliente.connections[0].out_buffer += 'I am data' will do the work, but probably you'd want some better controlling of this. If yes, go ahead.
'For whom, by me'
In order to send data asynchronously, we need to separate asyncore from our control thread, in which we'll enter what to send and to whom.
class ServerThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True # if thread is a daemon, it'll be killed when main program exits
self.cliente = Server()
self.start()
def run(self):
print 'Starting server thread...'
asyncore.loop()
thread = ServerThread()
while True:
msg = raw_input('Enter IP and message divided by semicolon: ')
if msg == 'exit':
break
ip, data = msg.split('; ')
for sock in thread.cliente.connections:
if sock.addr[0] == ip:
sock.out_buffer += data
break
This will work and wait for destination IP and data. Remember to have client connected.
As I said, you can use anything to indicate which socket is which. It can be a class with fields for eg. IP and username, so you could send data only to peers whose usernames start with 'D'.
But...
This solution is a bit rough and needs better knowledge of asyncore module if you want to send data nicely (here it has some delay due to how select() works) and make good use of this socket wrapper.
Here and here are some resources.
Syntax note
Although your code will now work, your code has some not-nice things. Semicolons on instructions ends don't cause errors, but making nearly every variable of class attribute can lead to them. For example here:
def handle_accept(self):
self.sock,self.addr = self.accept();
if self.addr:
print self.addr[0];
handler = EchoHandler(self.sock);
self.sock and self.addr might be used in that class for something other (eg. socket-related thing; addresses) and overriding them could make trouble. Methods used for requests should never save state of previous actions.
I hope Python will be good enough for you to stay with it!
Edit: sock.addr[0] can be used instead of sock.socket.getpeername()[0] but it requires self.addr not to be modified, so handle_accept() should look like this:
def handle_accept(self):
sock, addr = self.accept()
if addr:
print addr[0]
handler = EchoHandler(sock, self)
self.connections.append(handler)
I've decided to dip my toe into the world of asynchronous python with the help of twisted. I've implemented some of the examples from the documentation, but I'm having a difficult time finding an example of the, very simple, client I'm trying to write.
In short I'd like a client which establishes a tcp connection with a server and then sends simple "\n" terminated string messages off of a queue object to the server. The server doesn't ever respond with any messages so my client is fully unidirectional. I /think/ that what I want is some combination of this example and the twisted.internet.protocols.basic.LineReceiver convenience protocol. This feels like it should be just about the simplest thing one could do in twisted, but none of the documentation or examples I've seen online seem to fit quite right.
What I have done is not used a Queue but I am illustrating the code that sends a line, once a connection is made. There are bunch of print stuff that will help you understand on what is going on.
Usual import stuff:
from twisted.web import proxy
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.protocol import ReconnectingClientFactory
from twisted.protocols import basic
from twisted.python import log
import sys
log.startLogging(sys.stdout)
You create a protocol derived from line receiver, set the delimiter.
In this case, I simply write a string "www" once the connection is made.
The key thing is to look at protocol interface at twisted.internet.interface.py and understand the various methods of protocol and what they do and when they are called.
class MyProtocol(basic.LineReceiver):
#def makeConnection(self, transport):
# print transport
def connectionLost(self, reason):
print reason
self.sendData = False
def connectionMade(self):
print "connection made"
self.delimiter = "\n"
self.sendData = True
print self.transport
self.sendFromQueue()
def sendFromQueue(self):
while self.sendData:
msg = dataQueue.get()
self.sendLine(msg)
# you need to handle empty queue
# Have another function to resume
Finally, A protocol factory that will create a protocol instance for every connection.
Look at method : buildProtcol.
class myProtocolFactory():
protocol = MyProtocol
def doStart(self):
pass
def startedConnecting(self, connectorInstance):
print connectorInstance
def buildProtocol(self, address):
print address
return self.protocol()
def clientConnectionLost(self, connection, reason):
print reason
print connection
def clientConnectionFailed(self, connection, reason):
print connection
print reason
def doStop(self):
pass
Now you use a connector to make a connection:
reactor.connectTCP('localhost', 50000, myProtocolFactory())
reactor.run()
I ran this and connected it to an server that simply prints what it receives and hence send no ack back. Here is the output:
1286906080.08 82 INFO 140735087148064 __main__ conn_made: client_address=127.0.0.1:50277
1286906080.08 83 DEBUG 140735087148064 __main__ created handler; waiting for loop
1286906080.08 83 DEBUG 140735087148064 __main__ handle_read
1286906080.08 83 DEBUG 140735087148064 __main__ after recv
'www\n'
Recieved: 4
The above example if not fault tolerant. To reconnect , when a connection is lost, you can derive your protocol factory from an existing twisted class - ReconnectingClientFactory.
Twisted has almost all the tools that you would need :)
class myProtocolFactory(ReconnectingClientFactory):
protocol = MyProtocol
def buildProtocol(self, address):
print address
return self.protocol()
For further reference
I suggest that you read : http://krondo.com/?page_id=1327
[Edited: As per comment below]