I've recently been learning twisted, so I can integrate the framework into a pygames script. I've found there are alot of examples and tutorials that override the existing methods in twisted(please correct me if I'm mistaken).
In this simple client I have the twisted.protocols.basic.LineReceiver.lineReceived method being overriden when ever a line is sent
class ChatClientProtocol(LineReceiver):
def lineReceived(self,line):
print (line)
class ChatClient(ClientFactory):
def __init__(self):
self.protocol = ChatClientProtocol
reactor.connectTCP('192.168.1.2', 6000, ChatClient())
reactor.run()
Is the LineReceiver.lineReceived a listening socket at the address sent to the reactor.connctTCP? Would there be a way to do this without overriding the method? Or is this the paradigm of twisted(overriding is the way use twisted)?
LineReceiver.lineReceived is a method that gets called when a line is received. I don't know what you mean by asking if it's a "listening socket".
Overriding is the way that you receive lines using LineReceiver. Generally speaking, overriding or implementing callbacks for specific notifications is how you get called in Twisted, yes. How else would you want to do it?
Related
I come from a Twisted background, so I have a solid understanding of protocols and factories, as implemented by Twisted. However, I am in the midst of switching over to asyncio, and I'm having a bit of trouble understanding how factories integrate into this particular framework.
In the official documentation, we have an example of a server's asyncio.Protocol class definition. It does not have a user-defined __init__ function, so we can simply call loop.create_server(EchoServerClientProtocol, addr, port).
What happens if our Protocol needs to implement some initialization logic? For instance, consider this example which sets a maximum buffer size:
import asyncio
from collections import deque
class BufferedProtocolExample(asyncio.Protocol):
def __init__(self, buffsize=None):
self.queue = deque((), buffsize)
# ...
In Twisted, you'd create a Factory class to hold all of the configuration values, which you would then pass to the function initializing the connection. Asyncio seems to work in the same way, but I cannot find any documentation.
I could use functools.partial, but what is the correct way of handling this case?
The documentation has an example where they use a lambda for this, so my guess is that functools.partial is fine. They also state that protocol_factory can be any callable. So to have something like Twisted's Factorys, you'll just need to implement __call__ on a class the way you'd implement buildProtocol in Twisted.
So I'm creating the back end for a web-based game in python. Currently it works like this...
WebSocket Handler receives message...
WebSocket Handler calls message handler...
Message Handler calls Game class functions...
Game class calls other classes to update information.
This is highly coupled, and probably should be in model-view-controller format. So I'm finally going to change that.
Basically I want it to work like this.
Controller has open a WebSocket Handler.
WebSocket Handler returns to Controller a (1?).
Controller uses (1?) calls Message Handler.
Message Handler returns to Controller a (2?).
Contoller uses (2?) to call Model.
Model sends updates to appropriate places.
So there's two problems here.
First of all, when I'm getting a message in the WebSocket Handler, it is an instance of Tornado's WebSocketHandler, and I'm not sure how I can return anything to the Controller. Is the case simply that this is not possible? Do I have to keep a small amount of coupling between the WebSocket Handler and the Message Handler? I know I could always call a function in Controller, but that doesn't seem like an actual fix, just more function calls.
Is there a way to pass a function call around in python, keeping track of it's parameters while doing so? That would be the optimal way to go about doing this, but I don't think it's implemented in the python language. Otherwise, I feel like the best way to do it would be to return a dictionary with a field for the function name to be called, and fields for the parameters. This of course is a lot more code. If it can be avoided, I'd like to, but I'm not sure the direction in which to take this.
Thanks for the tips guys, this is a big refactoring and I'm really nervous about where to start.
For the second part of your question, I believe you want to use partial functions.
Check out: http://docs.python.org/2/library/functools.html
Basically you would go:
from functools import partial
function_call(partial(future_function_call, future_argument))
I want to split a Tornado handler in two handlers. In the first handler, I want send a command to a sensor. In the second handler, I want to wait for a response from the sensor.
Is this possible? Or I can't call an handler from another? If it's possible, how I can call this second handler?
Thank you very much.
I know that original post was asked years ago, but since accepted answer doesn't solve the question asked, and I do believe I have the proper solution, as I had a need for it. Also, if some poor soul is Googling for the same issue, hopefully they will find this.
class OneHandler(RequestHandler):
def get(self, id):
AnotherHandler(self.application, self.request).get(id)
Here you can call "AnotherHandler" from within "OneHandler"
Sounds like you've got a variant of the "chat" application. Your best bet is to take a look at the basic chat demo app.
The basic operating idea is to have a mixin (or globals if you prefer) that has a list of connections waiting on a response, when that response is set it triggers the callbacks on the original handlers.
How can I handle Asyncore.dispatcher(s) and SimpleXMLRPCServer events from the same event-loop?
P.S. I already know that some of you might recommend Twisted for this, but the problem with Twisted is that it is a little bit too high-level library for my needs. In particular I am doing UDP flow-control by overriding Asyncore.dispatcher.writable() method that depends on timers. Not sure if/how this could be doable in Twisted.
You should use Twisted for this :-). You can't put SimpleXMLRPCServer into an asynchronous loop; it's synchronous code, which expects to block.
Flow-control with Twisted, even with UDP, is easy. Rather than overriding a method like writable(), your DatagramProtocol can call methods like stopReading / stopWriting / startReading / startWriting on their transport attribute. You can see these methods here.
I'm building a program that has a class used locally, but I want the same class to be used the same way over the network. This means I need to be able to make synchronous calls to any of its public methods. The class reads and writes files, so I think XML-RPC is too much overhead. I created a basic rpc client/server using the examples from twisted, but I'm having trouble with the client.
c = ClientCreator(reactor, Greeter)
c.connectTCP(self.host, self.port).addCallback(request)
reactor.run()
This works for a single call, when the data is received I'm calling reactor.stop(), but if I make any more calls the reactor won't restart. Is there something else I should be using for this? maybe a different twisted module or another framework?
(I'm not including the details of how the protocol works, because the main point is that I only get one call out of this.)
Addendum & Clarification:
I shared a google doc with notes on what I'm doing. http://docs.google.com/Doc?id=ddv9rsfd_37ftshgpgz
I have a version written that uses fuse and can combine multiple local folders into the fuse mount point. The file access is already handled within a class, so I want to have servers that give me network access to the same class. After continuing to search, I suspect pyro (http://pyro.sourceforge.net/) might be what I'm really looking for (simply based on reading their home page right now) but I'm open to any suggestions.
I could achieve similar results by using an nfs mount and combining it with my local folder, but I want all of the peers to have access to the same combined filesystem, so that would require every computer to bee an nfs server with a number of nfs mounts equal to the number of computers in the network.
Conclusion:
I have decided to use rpyc as it gave me exactly what I was looking for. A server that keeps an instance of a class that I can manipulate as if it was local. If anyone is interested I put my project up on Launchpad (http://launchpad.net/dstorage).
If you're even considering Pyro, check out RPyC first, and re-consider XML-RPC.
Regarding Twisted: try leaving the reactor up instead of stopping it, and just ClientCreator(...).connectTCP(...) each time.
If you self.transport.loseConnection() in your Protocol you won't be leaving open connections.
For a synchronous client, Twisted probably isn't the right option. Instead, you might want to use the socket module directly.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((self.host, self.port))
s.send(output)
data = s.recv(size)
s.close()
The recv() call might need to be repeated until you get an empty string, but this shows the basics.
Alternatively, you can rearrange your entire program to support asynchronous calls...
Why do you feel that it needs to be synchronous?
If you want to ensure that only one of these is happening at a time, invoke all of the calls through a DeferredSemaphore so you can rate limit the actual invocations (to any arbitrary value).
If you want to be able to run multiple streams of these at different times, but don't care about concurrency limits, then you should at least separate reactor startup and teardown from the invocations (the reactor should run throughout the entire lifetime of the process).
If you just can't figure out how to express your application's logic in a reactor pattern, you can use deferToThread and write a chunk of purely synchronous code -- although I would guess this would not be necessary.
If you are using Twisted you should probably know that:
You will not be making synchronous calls to any network service
The reactor can only ever be run once, so do not stop it (by calling reactor.stop()) until your application is ready to exit.
I hope this answers your question. I personally believe that Twisted is exactly the correct solution for your use case, but that you need to work around your synchronicity issue.
Addendum & Clarification:
Part of what I don't understand is
that when I call reactor.run() it
seems to go into a loop that just
watches for network activity. How do I
continue running the rest of my
program while it uses the network? if
I can get past that, then I can
probably work through the
synchronicity issue.
That is exactly what reactor.run() does. It runs a main loop which is an event reactor. It will not only wait for entwork events, but anything else you have scheduled to happen. With Twisted you will need to structure the rest of your application in a way to deal with its asynchronous nature. Perhaps if we knew what kind of application it is, we could advise.