What I would like to do is combine Twisted with the Cmd module in python's stdlib.
In short I would like to be able to get the bare-bones socket fd object from a connected Protocol to use as the stdin of the cmd.Cmd module in the stdlib.
In Long, My client that interfaces with my server uses the Cmd module to process commands and send those commands to the server.
On my server I would also like to use the same command processing method with the builting Cmd module. To do this i would need to specify the stdin and stdout of the command interpreter.
I could do this easily with the builtin sockets module, but i would like to do it with twisted if possible.
Here is some code to do what i want with plain sockets:
(Works with telnet)
# server
import socket
import cmd
class CmdProcessor(cmd.Cmd, object):
def __init__(self, sock, addr):
network = sock.makefile()
super(CmdProcessor, self).__init__(stdin=network, stdout=network)
self.sock = sock
self.addr = addr
# Run the cmd.Cmd processing loop
self.cmdloop()
def do_sayhi(self, args):
# When 'sayhi' is recieved over the socket,
self.sock.send("Hey yourself!")
def do_quit(self, args):
self.sock.close()
if __name__ == "__main__":
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.bind(("0.0.0.0", 2319))
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.listen(5)
sock, addr = server_sock.accept()
print("Connection accepted")
connection = CmdProcessor(sock, addr)
This is almost what i want to to do. I just typed this up quick so i may be missing somthing. Half of it works. Currently, if you telnet into the server like:
telnet 127.0.0.1 2319
And you send 'sayhi' nothing happens. But if you type 'sayhi' at the terminal you started the server from (There is a (Cmd) prompt) the output goes to the telnet client. So the stdout of the cmd.Cmd is working. But not the stdin. That probably has something to do with the fact that telnet sends CR-LF ('\r\n') by default. Where the cmd module may just listen for \n.
So how can get the fd or file object from a protocol in twisted to do what i am trying do achieve here with bare sockets?
And any insights on what the input from telnet connected to the server is not registering with the CmdProcessor?
Any advice, tips or pointers welcome. (Wait no, no pointers.)
Thanks.
I suggest that instead you might want to look at Manhole.
In general, the point of Twisted is not to use Python socket objects directly. That's a big part of Twisted's job. When you want to interact with the network using Twisted, you use Twisted's APIs instead - protocols and transports, if you're thinking about the lowest level.
You can add use_rawinput = False
class CmdProcessor(cmd.Cmd, object):
use_rawinput = False
def __init__(self, sock, addr):
....
This produces response for sayhi from telnet
Related
I use Python 2.7.x (2.7.8) and am trying to write a program using twisted python to function like this:
Program waits for a messaged received through TCP 7001 or UDP 7000.
Messages received through UDP 7000 are bridged over to TCP 7001 going out.
I couldn't figure out how to bridge UDP to TCP, so I looked up an example on this site, like this one, Twisted UDP to TCP Bridge but the problem is that the example is lying because it doesn't work. I added a "print datagram" on datagramReceived to see if UDP responds to receiving anything at all and it does not. This is totally frustrating.
Here's my current test code altered a little bit from that example:
from twisted.internet.protocol import Protocol, Factory, DatagramProtocol
from twisted.internet import reactor
class TCPServer(Protocol):
def connectionMade(self):
self.port = reactor.listenUDP(7000, UDPServer(self))
def connectionLost(self, reason):
self.port.stopListening()
def dataReceived(self, data):
print "Server said:", data
class UDPServer(DatagramProtocol):
def __init__(self, stream):
self.stream = stream
def datagramReceived(self, datagram, address):
print datagram
self.stream.transport.write(datagram)
def main():
f = Factory()
f.protocol = TCPServer
reactor.listenTCP(7001, f)
reactor.run()
if __name__ == '__main__':
main()
As you can see I changed the ports to comply with my test environment and added a print datagram to see if anything calls datagramReceived. I have no problems sending TCP things to this program, TCPServer works just fine because dataReceived can be called.
I ran the code from your question (with one slight modification: I enabled logging). I used telnet to connect to the TCP server on port 7001. I used a Python REPL to create a UDP socket and send some datagrams to port 7000.
Here is my REPL transcript:
>>> import socket
>>> s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
>>> s.sendto('hello', ('127.0.0.1', 7000))
5
>>> s.sendto('world', ('127.0.0.1', 7000))
5
>>>
Here is my server log (formatted to fit on your screen):
... Log opened.
... Factory starting on 7001
... Starting factory <twisted.internet.protocol.Factory instance at 0x2b9b128>
... UDPServer starting on 7000
... Starting protocol <__main__.UDPServer instance at 0x2e8f8c0>
... hello
... world
And here's my telnet session transcript:
$ telnet localhost 7001
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
helloworld
My interpretation of these results is that the program actually works as specified. I wonder what you did differently when you tried it that produced different, non-working results.
I have been trying to test SCTP for a network deployment.
I do not have an SCTP server or client and was hoping to be able use pysctp.
I am fairly certain that I have the client side code working.
def sctp_client ():
print("SCTP client")
sock = sctp.sctpsocket_tcp(socket.AF_INET)
#sock.connect(('10.10.10.70',int(20003)))
sock.connect(('10.10.10.41',int(21000)))
print("Sending message")
sock.sctp_send(msg='allowed')
sock.shutdown(0)
sock.close()
Has anybody had luck with using the python sctp module for the server side?
Thank you in Advance!
I know that this topic's a bit dated, but I figured I would respond to it anyway to help out the community.
In a nutshell:
you are using pysctp with the sockets package to create either a client or a server;
you can therefore create your server connection as you normally would with a regular TCP connection.
Here's some code to get you started, it's a bit verbose, but it illustrates a full connection, sending, receiving, and closing the connection.
You can run it on your dev computer and then use a tool like ncat (nmap's implementation of netcat) to connect, i.e.: ncat --sctp localhost 80.
Without further ado, here's the code... HTH:
# Here are the packages that we need for our SCTP server
import socket
import sctp
from sctp import *
import threading
# Let's create a socket:
my_tcp_socket = sctpsocket_tcp(socket.AF_INET)
my_tcp_port = 80
# Here are a couple of parameters for the server
server_ip = "0.0.0.0"
backlog_conns = 3
# Let's set up a connection:
my_tcp_socket.events.clear()
my_tcp_socket.bind((server_ip, my_tcp_port))
my_tcp_socket.listen(backlog_conns)
# Here's a method for handling a connection:
def handle_client(client_socket):
client_socket.send("Howdy! What's your name?\n")
name = client_socket.recv(1024) # This might be a problem for someone with a reaaallly long name.
name = name.strip()
print "His name was Robert Paulson. Er, scratch that. It was {0}.".format(name)
client_socket.send("Thanks for calling, {0}. Bye, now.".format(name))
client_socket.close()
# Now, let's handle an actual connection:
while True:
client, addr = my_tcp_socket.accept()
print "Call from {0}:{1}".format(addr[0], addr[1])
client_handler = threading.Thread(target = handle_client,
args = (client,))
client_handler.start()
Unless you need the special sctp_ functions you don't need an sctp module at all.
Just use protocol 132 as IPPROTO_SCTP (is defined on my python3 socket module but not on my python2 socket module) and you can use the socket,bind,listen,connect,send,recv,sendto,recvfrom,close from the standard socket module.
I'm doing some SCTP C development and I used python to better understand SCTP behavior without the SCTP module.
I am writing server application in Python that listens for requests, processes them, and sends a response.
All req/resp are send from the same address and port to the server application. I need to recv/send messages simultaneously, and the server need to recieve/send messages from/to the same port and address. I found some tutorials for asynchore sockets, but there are only examples for TCP connections.
Unfortunately, I need UDP. When I change SOCK_STREAM to SOCK_DGRAM in the create method, I get an error.
return getattr(self._sock,name)(*args)
socket.error: [Errno 95] Operation not supported
I tried to use twisted, but I dont know how to write the sender part, which can bind to the same port as its listening. The last result was blocked port.
Is there any way how to use asyncore sockets with UDP or how to use twisted to send from the same port? Some examples will be higly appreciated.
You can pretty much just write the sending and receiving part of your code and they'll work together. Note that you can send and receive on a single listening UDP socket - you don't need one for each (particularly if you want to send from and receive on the same address).
from __future__ import print_function
from sys import stdout
from twisted.python.log import startLogging
from twisted.internet import reactor
from twisted.internet.protocol import DatagramProtocol
class SomeUDP(DatagramProtocol):
def datagramReceived(self, datagram, address):
print(u"Got a datagram of {} bytes.".format(len(datagram)))
def sendFoo(self, foo, ip, port):
self.transport.write(
(u"Foo datagram: {}".format(foo)).encode("utf-8"),
(ip, port))
class SomeSender(object):
def __init__(self, proto):
self.proto = proto
def start(self):
reactor.callLater(3, self._send)
def _send(self):
self.proto.sendFoo(u"Hello or whatever", b"127.0.0.1", 12345)
self.start()
startLogging(stdout)
proto = SomeUDP()
reactor.listenUDP(12345, proto)
SomeSender(proto).start()
reactor.run()
How can I use HTTPServer (or some other class) to set up an HTTP server that listens to a filesystem socket instead of an actual network socket? By "filesystem socket" I mean sockets of the AF_UNIX type.
HTTPServer inherits from SocketServer.TCPServer, so I think it's fair to say that it isn't intended for that use-case, and even if you try to work around it, you may run into problems since you are kind of "abusing" it.
That being said, however, it would be possible per se to define a subclass of HTTPServer that creates and binds Unix sockets quite simply, as such:
class UnixHTTPServer(HTTPServer):
address_family = socket.AF_UNIX
def server_bind(self):
SocketServer.TCPServer.server_bind(self)
self.server_name = "foo"
self.server_port = 0
Then, just pass the path you want to bind to by the server_address argument to the constructor:
server = UnixHTTPServer("/tmp/http.socket", ...)
Again, though, I can't guarantee that it will actually work well. You may have to implement your own HTTP server instead.
I followed the example from #Dolda2000 above in Python 3.5 and ran into an issue with the HTTP handler falling over with an invalid client address. You don't have a client address with Unix sockets in the same way that you do with TCP, so the code below fakes it.
import socketserver
...
class UnixSocketHttpServer(socketserver.UnixStreamServer):
def get_request(self):
request, client_address = super(UnixSocketHttpServer, self).get_request()
return (request, ["local", 0])
...
server = UnixSocketHttpServer((sock_file), YourHttpHandler)
server.serve_forever()
With these changes, you can perform an HTTP request against the Unix socket with tools such as cURL.
curl --unix-socket /run/test.sock http:/test
Overview
In case it help anyone else, I have created a complete example (made for Python 3.8) based on Roger Lucas's example:
Server
import socketserver
from http.server import BaseHTTPRequestHandler
class myHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write(b"Hello world!")
return
class UnixSocketHttpServer(socketserver.UnixStreamServer):
def get_request(self):
request, client_address = super(UnixSocketHttpServer, self).get_request()
return (request, ["local", 0])
server = UnixSocketHttpServer(("/tmp/http.socket"), myHandler)
server.serve_forever()
This will listen on the unix socket and respond with "Hello World!" for all GET requests.
Client Request
You can send a request with:
curl --unix-socket /tmp/http.socket http://any_path/abc/123
Troubleshooting
If you run into this error:
OSError: [Errno 98] Address already in use
Then delete the socket file:
rm /tmp/http.socket
I'm currently writing a telnet server in Python. It's a content server. People would connect to the server via telnet, and be presented with text-only content.
My problem is that the server would obviously need to support more than one simultaneous connection. The current implementation I have now supports only one.
This is the basic, proof-of-concept server I began with (while the program has changed greatly over time, the basic telnet framework hasn't):
import socket, os
class Server:
def __init__(self):
self.host, self.port = 'localhost', 50000
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.host, self.port))
def send(self, msg):
if type(msg) == str: self.conn.send(msg + end)
elif type(msg) == list or tuple: self.conn.send('\n'.join(msg) + end)
def recv(self):
self.conn.recv(4096).strip()
def exit(self):
self.send('Disconnecting you...'); self.conn.close(); self.run()
# closing a connection, opening a new one
# main runtime
def run(self):
self.socket.listen(1)
self.conn, self.addr = self.socket.accept()
# there would be more activity here
# i.e.: sending things to the connection we just made
S = Server()
S.run()
Thanks for your help.
Implemented in twisted:
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
class SendContent(Protocol):
def connectionMade(self):
self.transport.write(self.factory.text)
self.transport.loseConnection()
class SendContentFactory(Factory):
protocol = SendContent
def __init__(self, text=None):
if text is None:
text = """Hello, how are you my friend? Feeling fine? Good!"""
self.text = text
reactor.listenTCP(50000, SendContentFactory())
reactor.run()
Testing:
$ telnet localhost 50000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello, how are you my friend? Feeling fine? Good!
Connection closed by foreign host.
Seriously, when it comes to asynchronous network, twisted is the way to go. It handles multiple connections in a single-thread single-process approach.
You need some form of asynchronous socket IO. Have a look at this explanation, which discusses the concept in low-level socket terms, and the related examples which are implemented in Python. That should point you in the right direction.
Late for the reply, but with the only answers being Twisted or threads (ouch), I wanted to add an answer for MiniBoa.
http://code.google.com/p/miniboa/
Twisted is great, but it's a rather large beast that may not be the best introduction to single-threaded asynchronous Telnet programming. MiniBoa is a lightweight, asynchronous single-threaded Python Telnet implementation originally designed for muds, which suits the OP's question perfectly.
For a really easy win implement you solution using SocketServer & the SocketServer.ThreadingMixIn
have a look a this echo server example it looks quite similar to what you're doing anyway: http://www.oreillynet.com/onlamp/blog/2007/12/pymotw_socketserver.html
If you're up for a bit of a conceptual challenge, I'd look into using twisted.
Your case should be trivial to implement as a part of twisted.
http://twistedmatrix.com/projects/core/documentation/howto/servers.html
If you want to do it in pure python (sans-twisted), you need to do some threading. If you havnt seen it before, check out:
http://heather.cs.ucdavis.edu/~matloff/Python/PyThreads.pdf
around page 5/6 is an example that is very relevant ;)
First, buy Comer's books on TCP/IP programming.
In those books, Comer will provide several alternative algorithms for servers. There are two standard approaches.
Thread-per-request.
Process-per-request.
You must pick one of these two and implement that.
In thread-per, each telnet session is a separate thread in your overall application.
In process-per, you fork each telnet session into a separate subprocess.
You'll find that process-per-request is much, much easier to handle in Python, and it generally makes more efficient use of your system.
Thread-per-request is fine for things that come and go quickly (like HTTP requests). Telnet has long-running sessions where the startup cost for a subprocess does not dominate performance.
Try MiniBoa server? It has exactly 0 dependencies, no twisted or other stuff needed. MiniBoa is a non-blocking async telnet server, single threaded, exactly what you need.
http://code.google.com/p/miniboa/
Use threading and then add the handler into a function. The thread will call every time a request i made:
Look at this
import socket # Import socket module
import pygame
import thread
import threading,sys
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345 # Reserve a port for your service.
s.bind((host, port))
print ((host, port))
name = ""
users = []
def connection_handler (c, addr):
print "conn handle"
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
if a == "o":
c.send (str(users))
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
print a,b
s.listen(6) # Now wait for client connection.
while True:
c, addr = s.accept()
print 'Connect atempt from:', addr[0]
username = c.recv(1024)
print "2"
if username == "END_SERVER_RUBBISH101":
if addr[0] == "192.168.1.68":
break
users.append(username)
thread.start_new_thread (connection_handler, (c, addr)) #New thread for connection
print
s.close()