I create little SimpleXMLRPCServer for check ip of client.
I try this:
Server
import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(("localhost", 8000))
def MyIp():
return "Your ip is: %s" % server.socket.getpeername()
server.register_function(MyIp)
server.serve_forever()
Client
import xmlrpclib
se = xmlrpclib.Server("http://localhost:8000")
print se.MyIp()
Error
xmlrpclib.Fault: :(107, 'Transport endpoint is not connected')">
How make client_address visible to all functions?
If you want for example to pass client_address as the first argument to every function, you could subclass SimpleXMLRPCRequestHandler (pass your subclass as the handler when you instantiate SimpleXMLRPCServer) and override _dispatch (to prepend self.client_address to the params tuple and then delegate the rest to SimpleXMLRPCRequestHandler._dispatch). If this approach is OK and you want to see code, just ask!
I'm not sure how you'd safely use anything but the function arguments to "make client_address visible" -- there's no client_address as a bare name, global or otherwise, there's just the self.client_address of each instance of the request handler class (and hacks such as copying it to a global variables feel really yucky indeed -- and unsafe under threading, etc etc).
Related
I am looking at the example of the UDP echo server:
import asyncio
class EchoServerProtocol:
def connection_made(self, transport):
self.transport = transport
def datagram_received(self, data, addr):
message = data.decode()
print('Received %r from %s' % (message, addr))
print('Send %r to %s' % (message, addr))
self.transport.sendto(data, addr)
loop = asyncio.get_event_loop()
print("Starting UDP server")
# One protocol instance will be created to serve all client requests
listen = loop.create_datagram_endpoint(
EchoServerProtocol, local_addr=('127.0.0.1', 9999))
transport, protocol = loop.run_until_complete(listen)
try:
loop.run_forever()
except KeyboardInterrupt:
pass
transport.close()
loop.close()
It seems that the call...
loop.create_datagram_endpoint(EchoServerProtocol, local_addr=('127.0.0.1', 9999))
...is doing all the work here. The method documentation states the following for the first argument (protocol_factory):
protocol_factory must be a callable returning a protocol instance.
My questions:
What defines a protocol instance?
Is returning a protocol instance a different wording for initiating a protocol object?
How does the EchoServerProtocol in the example fulfill this requirement?
What defines a protocol instance?
A protocol is a class you define that implements one of the interfaces defined in the Protocols section, i.e. provides implementations for a set of callbacks, e.g. Connection Callbacks.
So for the UDP echo server example you have posted, the EchoServerProtocol user defined class actually defines a protocol by implementing the connection_made and datagram_received.
In summary, if you implement one of those callbacks in a class, you are said to be defining a Protocol. So an instance/object of that class would be a protocol instance.
Is returning a protocol instance a different wording for initiating a protocol object?
Formally YES. Before you would return a protocol instance, you would have initialized it. So basically one is a prerequisite of the other.
How does the EchoServerProtocol in the example fulfill this requirement?
So first of all, as answered the first question, the EchoServerProtocol defines a protocol. Thus the next thing is to provide a protocol_factory, which is defined as:
protocol_factory must be a callable returning a protocol instance.
So to satisfy this requirement, you could just have this simple method:
def my_protocol_factory():
return EchoServerProtocol()
Note, that this factory first initializes the protocol instance and then returns it.
So the thing that might confuse you in the example, is that the class EchoServerProtocol itself is passed as the protocol_factory, but if you summarize what I've said, you will see that the EchoServerProtocol is actually a callable, and when it gets called, i.e. EchoServerProtocol() it actually initializes a EchoServerProtocol instance, and returns it.
So yep, the example fulfills the requirement.
I'm trying to pass a TCP connection to a Twisted subprocess with adoptStreamConnection, but I can't figure out how to get the Process disposed in the main process after doing that.
My desired flow looks like this:
Finish writing any data the Protocol transport has waiting
When we know the write buffer is empty send the AMP message to transfer the socket to the subprocess
Dispose the Protocol instance in the main process
I tried doing nothing, loseConnection, abortConnection, and monkey patching _socketClose out and using loseConnection. See code here:
import weakref
from twisted.internet import reactor
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.python.sendmsg import getsockfam
from twisted.internet.protocol import Factory, Protocol
import twisted.internet.abstract
class EchoProtocol(Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(Factory):
protocol = EchoProtocol
class TransferProtocol(Protocol):
def dataReceived(self, data):
self.transport.write('main process still listening!: %s' % (data))
def connectionMade(self):
self.transport.write('this message should make it to the subprocess\n')
# attempt 1: do nothing
# everything works fine in the adopt (including receiving the written message), but old protocol still exists (though isn't doing anything)
# attempt 1: try calling loseConnection
# we lose connection before the adopt opens the socket (presumably TCP disconnect message was sent)
#
# self.transport.loseConnection()
# attempt 2: try calling abortConnection
# result is same as loseConnection
#
# self.transport.abortConnection()
# attempt 3: try monkey patching the socket close out and calling loseConnection
# result: same as doing nothing-- adopt works (including receiving the written message), old protocol still exists
#
# def ignored(*args, **kwargs):
# print 'ignored :D'
#
# self.transport._closeSocket = ignored
# self.transport.loseConnection()
reactor.callLater(0, adopt, self.transport.fileno())
class ServerFactory(Factory):
def buildProtocol(self, addr):
p = TransferProtocol()
self.ref = weakref.ref(p)
return p
f = ServerFactory()
def adopt(fileno):
print "does old protocol still exist?: %r" % (f.ref())
reactor.adoptStreamConnection(fileno, getsockfam(fileno), EchoFactory())
port = 1337
endpoint = TCP4ServerEndpoint(reactor, port)
d = endpoint.listen(f)
reactor.run()
In all cases the Protocol object still exists in the main process after the socket has been transferred. How can I clean this up?
Thanks in advance.
Neither loseConnection nor abortConnection tell the reactor to "forget" about a connection; they close the connection, which is very different; they tell the peer that the connection has gone away.
You want to call self.transport.stopReading() and self.transport.stopWriting() to remove the references to it from the reactor.
Also, it's not valid to use a weakref to test for the remaining existence of an object unless you call gc.collect() first.
As far as making sure that all the data has been sent: the only reliable way to do that is to have an application-level acknowledgement of the data that you've sent. This is why protocols that need a handshake that involves changing protocols - say, for example, STARTTLS - have a specific handshake where the initiator says "I'm going to switch" (and then stops sending), then the peer says "OK, you can switch now". Another way to handle that in this case would be to hand the data you'd like to write to the subprocess via some other channel, instead of passing it to transport.write.
How can I use HTTPServer (or some other class) to set up an HTTP server that listens to a filesystem socket instead of an actual network socket? By "filesystem socket" I mean sockets of the AF_UNIX type.
HTTPServer inherits from SocketServer.TCPServer, so I think it's fair to say that it isn't intended for that use-case, and even if you try to work around it, you may run into problems since you are kind of "abusing" it.
That being said, however, it would be possible per se to define a subclass of HTTPServer that creates and binds Unix sockets quite simply, as such:
class UnixHTTPServer(HTTPServer):
address_family = socket.AF_UNIX
def server_bind(self):
SocketServer.TCPServer.server_bind(self)
self.server_name = "foo"
self.server_port = 0
Then, just pass the path you want to bind to by the server_address argument to the constructor:
server = UnixHTTPServer("/tmp/http.socket", ...)
Again, though, I can't guarantee that it will actually work well. You may have to implement your own HTTP server instead.
I followed the example from #Dolda2000 above in Python 3.5 and ran into an issue with the HTTP handler falling over with an invalid client address. You don't have a client address with Unix sockets in the same way that you do with TCP, so the code below fakes it.
import socketserver
...
class UnixSocketHttpServer(socketserver.UnixStreamServer):
def get_request(self):
request, client_address = super(UnixSocketHttpServer, self).get_request()
return (request, ["local", 0])
...
server = UnixSocketHttpServer((sock_file), YourHttpHandler)
server.serve_forever()
With these changes, you can perform an HTTP request against the Unix socket with tools such as cURL.
curl --unix-socket /run/test.sock http:/test
Overview
In case it help anyone else, I have created a complete example (made for Python 3.8) based on Roger Lucas's example:
Server
import socketserver
from http.server import BaseHTTPRequestHandler
class myHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write(b"Hello world!")
return
class UnixSocketHttpServer(socketserver.UnixStreamServer):
def get_request(self):
request, client_address = super(UnixSocketHttpServer, self).get_request()
return (request, ["local", 0])
server = UnixSocketHttpServer(("/tmp/http.socket"), myHandler)
server.serve_forever()
This will listen on the unix socket and respond with "Hello World!" for all GET requests.
Client Request
You can send a request with:
curl --unix-socket /tmp/http.socket http://any_path/abc/123
Troubleshooting
If you run into this error:
OSError: [Errno 98] Address already in use
Then delete the socket file:
rm /tmp/http.socket
An example of my code is as follows. I would like to arbitrarly send data at various points in the program. Twisted seems great for listening and then reacting, but how to I simply send data.
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
import os
class listener(DatagramProtocol):
def __init__(self):
def datagramReceived(self, data, (host, port)):
print "GOT " + data
def send_stuff(data):
self.transport.write(data, (host, port))
reactor.listenUDP(10000, listener())
reactor.run()
##Some things happen in the program independent of the connection state
##Now how to I access send_stuff
Your example already includes some code that sends data:
def send_stuff(data):
self.transport.write(data, (host, port))
In other words, the answer to your question is "call send_stuff" or even "call transport.write".
In a comment you asked:
#Now how to I access send_stuff
There's nothing special about how you "access" objects or methods when you're using Twisted. It's the same as in any other Python program you might write. Use variables, attributes, containers, function arguments, or any of the other facilities to maintaining references to objects.
Here are some examples:
# Save the listener instance in a local variable
network = listener()
reactor.listenUDP(10000, network)
# Use the local variable to connect a GUI event to the network
MyGUIApplication().connect_button("send_button", network.send_stuff)
# Use the local variable to implement a signal handler that sends data
def report_signal(*ignored):
reactor.callFromThread(network.send_stuff, "got sigint")
signal.signal(signal.SIGINT, report_signal)
# Pass the object referenced by the local variable to the initializer of another
# network-related object so it can save the reference and later call methods on it
# when it gets events it wants to respond to.
reactor.listenUDP(20000, AnotherDatagramProtocol(network))
And so on.
I've decided to dip my toe into the world of asynchronous python with the help of twisted. I've implemented some of the examples from the documentation, but I'm having a difficult time finding an example of the, very simple, client I'm trying to write.
In short I'd like a client which establishes a tcp connection with a server and then sends simple "\n" terminated string messages off of a queue object to the server. The server doesn't ever respond with any messages so my client is fully unidirectional. I /think/ that what I want is some combination of this example and the twisted.internet.protocols.basic.LineReceiver convenience protocol. This feels like it should be just about the simplest thing one could do in twisted, but none of the documentation or examples I've seen online seem to fit quite right.
What I have done is not used a Queue but I am illustrating the code that sends a line, once a connection is made. There are bunch of print stuff that will help you understand on what is going on.
Usual import stuff:
from twisted.web import proxy
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.protocol import ReconnectingClientFactory
from twisted.protocols import basic
from twisted.python import log
import sys
log.startLogging(sys.stdout)
You create a protocol derived from line receiver, set the delimiter.
In this case, I simply write a string "www" once the connection is made.
The key thing is to look at protocol interface at twisted.internet.interface.py and understand the various methods of protocol and what they do and when they are called.
class MyProtocol(basic.LineReceiver):
#def makeConnection(self, transport):
# print transport
def connectionLost(self, reason):
print reason
self.sendData = False
def connectionMade(self):
print "connection made"
self.delimiter = "\n"
self.sendData = True
print self.transport
self.sendFromQueue()
def sendFromQueue(self):
while self.sendData:
msg = dataQueue.get()
self.sendLine(msg)
# you need to handle empty queue
# Have another function to resume
Finally, A protocol factory that will create a protocol instance for every connection.
Look at method : buildProtcol.
class myProtocolFactory():
protocol = MyProtocol
def doStart(self):
pass
def startedConnecting(self, connectorInstance):
print connectorInstance
def buildProtocol(self, address):
print address
return self.protocol()
def clientConnectionLost(self, connection, reason):
print reason
print connection
def clientConnectionFailed(self, connection, reason):
print connection
print reason
def doStop(self):
pass
Now you use a connector to make a connection:
reactor.connectTCP('localhost', 50000, myProtocolFactory())
reactor.run()
I ran this and connected it to an server that simply prints what it receives and hence send no ack back. Here is the output:
1286906080.08 82 INFO 140735087148064 __main__ conn_made: client_address=127.0.0.1:50277
1286906080.08 83 DEBUG 140735087148064 __main__ created handler; waiting for loop
1286906080.08 83 DEBUG 140735087148064 __main__ handle_read
1286906080.08 83 DEBUG 140735087148064 __main__ after recv
'www\n'
Recieved: 4
The above example if not fault tolerant. To reconnect , when a connection is lost, you can derive your protocol factory from an existing twisted class - ReconnectingClientFactory.
Twisted has almost all the tools that you would need :)
class myProtocolFactory(ReconnectingClientFactory):
protocol = MyProtocol
def buildProtocol(self, address):
print address
return self.protocol()
For further reference
I suggest that you read : http://krondo.com/?page_id=1327
[Edited: As per comment below]