I'm currently writing a telnet server in Python. It's a content server. People would connect to the server via telnet, and be presented with text-only content.
My problem is that the server would obviously need to support more than one simultaneous connection. The current implementation I have now supports only one.
This is the basic, proof-of-concept server I began with (while the program has changed greatly over time, the basic telnet framework hasn't):
import socket, os
class Server:
def __init__(self):
self.host, self.port = 'localhost', 50000
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.host, self.port))
def send(self, msg):
if type(msg) == str: self.conn.send(msg + end)
elif type(msg) == list or tuple: self.conn.send('\n'.join(msg) + end)
def recv(self):
self.conn.recv(4096).strip()
def exit(self):
self.send('Disconnecting you...'); self.conn.close(); self.run()
# closing a connection, opening a new one
# main runtime
def run(self):
self.socket.listen(1)
self.conn, self.addr = self.socket.accept()
# there would be more activity here
# i.e.: sending things to the connection we just made
S = Server()
S.run()
Thanks for your help.
Implemented in twisted:
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
class SendContent(Protocol):
def connectionMade(self):
self.transport.write(self.factory.text)
self.transport.loseConnection()
class SendContentFactory(Factory):
protocol = SendContent
def __init__(self, text=None):
if text is None:
text = """Hello, how are you my friend? Feeling fine? Good!"""
self.text = text
reactor.listenTCP(50000, SendContentFactory())
reactor.run()
Testing:
$ telnet localhost 50000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello, how are you my friend? Feeling fine? Good!
Connection closed by foreign host.
Seriously, when it comes to asynchronous network, twisted is the way to go. It handles multiple connections in a single-thread single-process approach.
You need some form of asynchronous socket IO. Have a look at this explanation, which discusses the concept in low-level socket terms, and the related examples which are implemented in Python. That should point you in the right direction.
Late for the reply, but with the only answers being Twisted or threads (ouch), I wanted to add an answer for MiniBoa.
http://code.google.com/p/miniboa/
Twisted is great, but it's a rather large beast that may not be the best introduction to single-threaded asynchronous Telnet programming. MiniBoa is a lightweight, asynchronous single-threaded Python Telnet implementation originally designed for muds, which suits the OP's question perfectly.
For a really easy win implement you solution using SocketServer & the SocketServer.ThreadingMixIn
have a look a this echo server example it looks quite similar to what you're doing anyway: http://www.oreillynet.com/onlamp/blog/2007/12/pymotw_socketserver.html
If you're up for a bit of a conceptual challenge, I'd look into using twisted.
Your case should be trivial to implement as a part of twisted.
http://twistedmatrix.com/projects/core/documentation/howto/servers.html
If you want to do it in pure python (sans-twisted), you need to do some threading. If you havnt seen it before, check out:
http://heather.cs.ucdavis.edu/~matloff/Python/PyThreads.pdf
around page 5/6 is an example that is very relevant ;)
First, buy Comer's books on TCP/IP programming.
In those books, Comer will provide several alternative algorithms for servers. There are two standard approaches.
Thread-per-request.
Process-per-request.
You must pick one of these two and implement that.
In thread-per, each telnet session is a separate thread in your overall application.
In process-per, you fork each telnet session into a separate subprocess.
You'll find that process-per-request is much, much easier to handle in Python, and it generally makes more efficient use of your system.
Thread-per-request is fine for things that come and go quickly (like HTTP requests). Telnet has long-running sessions where the startup cost for a subprocess does not dominate performance.
Try MiniBoa server? It has exactly 0 dependencies, no twisted or other stuff needed. MiniBoa is a non-blocking async telnet server, single threaded, exactly what you need.
http://code.google.com/p/miniboa/
Use threading and then add the handler into a function. The thread will call every time a request i made:
Look at this
import socket # Import socket module
import pygame
import thread
import threading,sys
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345 # Reserve a port for your service.
s.bind((host, port))
print ((host, port))
name = ""
users = []
def connection_handler (c, addr):
print "conn handle"
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
if a == "o":
c.send (str(users))
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
print a,b
s.listen(6) # Now wait for client connection.
while True:
c, addr = s.accept()
print 'Connect atempt from:', addr[0]
username = c.recv(1024)
print "2"
if username == "END_SERVER_RUBBISH101":
if addr[0] == "192.168.1.68":
break
users.append(username)
thread.start_new_thread (connection_handler, (c, addr)) #New thread for connection
print
s.close()
Related
This is my server program, how can it send the data received from each client to every other client?
import socket
import os
from threading import Thread
import thread
def listener(client, address):
print "Accepted connection from: ", address
while True:
data = client.recv(1024)
if not data:
break
else:
print repr(data)
client.send(data)
client.close()
host = socket.gethostname()
port = 10016
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host,port))
s.listen(3)
th = []
while True:
print "Server is listening for connections..."
client, address = s.accept()
th.append(Thread(target=listener, args = (client,address)).start())
s.close()
If you need to send a message to all clients, you need to keep a collection of all clients in some way. For example:
clients = set()
clients_lock = threading.Lock()
def listener(client, address):
print "Accepted connection from: ", address
with clients_lock:
clients.add(client)
try:
while True:
data = client.recv(1024)
if not data:
break
else:
print repr(data)
with clients_lock:
for c in clients:
c.sendall(data)
finally:
with clients_lock:
clients.remove(client)
client.close()
It would probably be clearer to factor parts of this out into separate functions, like a broadcast function that did all the sends.
Anyway, this is the simplest way to do it, but it has problems:
If one client has a slow connection, everyone else could bog down writing to it. And while they're blocking on their turn to write, they're not reading anything, so you could overflow the buffers and start disconnecting everyone.
If one client has an error, the client whose thread is writing to that client could get the exception, meaning you'll end up disconnecting the wrong user.
So, a better solution is to give each client a queue, and a writer thread servicing that queue, alongside the reader thread. (You can then extend this in all kinds of ways—put limits on the queue so that people stop trying to talk to someone who's too far behind, etc.)
As Anzel points out, there's a different way to design servers besides using a thread (or two) per client: using a reactor that multiplexes all of the clients' events.
Python 3.x has some great libraries for this built in, but 2.7 only has the clunky and out-of-date asyncore/asynchat and the low-level select.
As Anzel says, Python SocketServer: sending to multiple clients has an answer using asyncore, which is worth reading. But I wouldn't actually use that. If you want to write a reactor-based server in Python 2.x, I'd either use a better third-party framework like Twisted, or find or write a very simple one that sits directly on select.
I have been trying to test SCTP for a network deployment.
I do not have an SCTP server or client and was hoping to be able use pysctp.
I am fairly certain that I have the client side code working.
def sctp_client ():
print("SCTP client")
sock = sctp.sctpsocket_tcp(socket.AF_INET)
#sock.connect(('10.10.10.70',int(20003)))
sock.connect(('10.10.10.41',int(21000)))
print("Sending message")
sock.sctp_send(msg='allowed')
sock.shutdown(0)
sock.close()
Has anybody had luck with using the python sctp module for the server side?
Thank you in Advance!
I know that this topic's a bit dated, but I figured I would respond to it anyway to help out the community.
In a nutshell:
you are using pysctp with the sockets package to create either a client or a server;
you can therefore create your server connection as you normally would with a regular TCP connection.
Here's some code to get you started, it's a bit verbose, but it illustrates a full connection, sending, receiving, and closing the connection.
You can run it on your dev computer and then use a tool like ncat (nmap's implementation of netcat) to connect, i.e.: ncat --sctp localhost 80.
Without further ado, here's the code... HTH:
# Here are the packages that we need for our SCTP server
import socket
import sctp
from sctp import *
import threading
# Let's create a socket:
my_tcp_socket = sctpsocket_tcp(socket.AF_INET)
my_tcp_port = 80
# Here are a couple of parameters for the server
server_ip = "0.0.0.0"
backlog_conns = 3
# Let's set up a connection:
my_tcp_socket.events.clear()
my_tcp_socket.bind((server_ip, my_tcp_port))
my_tcp_socket.listen(backlog_conns)
# Here's a method for handling a connection:
def handle_client(client_socket):
client_socket.send("Howdy! What's your name?\n")
name = client_socket.recv(1024) # This might be a problem for someone with a reaaallly long name.
name = name.strip()
print "His name was Robert Paulson. Er, scratch that. It was {0}.".format(name)
client_socket.send("Thanks for calling, {0}. Bye, now.".format(name))
client_socket.close()
# Now, let's handle an actual connection:
while True:
client, addr = my_tcp_socket.accept()
print "Call from {0}:{1}".format(addr[0], addr[1])
client_handler = threading.Thread(target = handle_client,
args = (client,))
client_handler.start()
Unless you need the special sctp_ functions you don't need an sctp module at all.
Just use protocol 132 as IPPROTO_SCTP (is defined on my python3 socket module but not on my python2 socket module) and you can use the socket,bind,listen,connect,send,recv,sendto,recvfrom,close from the standard socket module.
I'm doing some SCTP C development and I used python to better understand SCTP behavior without the SCTP module.
What I would like to do is combine Twisted with the Cmd module in python's stdlib.
In short I would like to be able to get the bare-bones socket fd object from a connected Protocol to use as the stdin of the cmd.Cmd module in the stdlib.
In Long, My client that interfaces with my server uses the Cmd module to process commands and send those commands to the server.
On my server I would also like to use the same command processing method with the builting Cmd module. To do this i would need to specify the stdin and stdout of the command interpreter.
I could do this easily with the builtin sockets module, but i would like to do it with twisted if possible.
Here is some code to do what i want with plain sockets:
(Works with telnet)
# server
import socket
import cmd
class CmdProcessor(cmd.Cmd, object):
def __init__(self, sock, addr):
network = sock.makefile()
super(CmdProcessor, self).__init__(stdin=network, stdout=network)
self.sock = sock
self.addr = addr
# Run the cmd.Cmd processing loop
self.cmdloop()
def do_sayhi(self, args):
# When 'sayhi' is recieved over the socket,
self.sock.send("Hey yourself!")
def do_quit(self, args):
self.sock.close()
if __name__ == "__main__":
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_sock.bind(("0.0.0.0", 2319))
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_sock.listen(5)
sock, addr = server_sock.accept()
print("Connection accepted")
connection = CmdProcessor(sock, addr)
This is almost what i want to to do. I just typed this up quick so i may be missing somthing. Half of it works. Currently, if you telnet into the server like:
telnet 127.0.0.1 2319
And you send 'sayhi' nothing happens. But if you type 'sayhi' at the terminal you started the server from (There is a (Cmd) prompt) the output goes to the telnet client. So the stdout of the cmd.Cmd is working. But not the stdin. That probably has something to do with the fact that telnet sends CR-LF ('\r\n') by default. Where the cmd module may just listen for \n.
So how can get the fd or file object from a protocol in twisted to do what i am trying do achieve here with bare sockets?
And any insights on what the input from telnet connected to the server is not registering with the CmdProcessor?
Any advice, tips or pointers welcome. (Wait no, no pointers.)
Thanks.
I suggest that instead you might want to look at Manhole.
In general, the point of Twisted is not to use Python socket objects directly. That's a big part of Twisted's job. When you want to interact with the network using Twisted, you use Twisted's APIs instead - protocols and transports, if you're thinking about the lowest level.
You can add use_rawinput = False
class CmdProcessor(cmd.Cmd, object):
use_rawinput = False
def __init__(self, sock, addr):
....
This produces response for sayhi from telnet
I'm trying to test some code that reconnects to a server after a disconnect. This works perfectly fine outside the tests, but it fails to acknowledge that the socket has disconnected when running the tests.
I'm using a Gevent Stream Server to mock a real listening server:
import gevent.server
from gevent import queue
class TestServer(gevent.server.StreamServer):
def __init__(self, *args, **kwargs):
super(TestServer, self).__init__(*args, **kwargs)
self.sockets = {}
def handle(self, socket, address):
self.sockets[address] = (socket, queue.Queue())
socket.sendall('testing the connection\r\n')
gevent.spawn(self.recv, address)
def recv(self, address):
socket = self.sockets[address][0]
queue = self.sockets[address][1]
print 'Connection accepted %s:%d' % address
try:
for data in socket.recv(1024):
queue.put(data)
except:
pass
def murder(self):
self.stop()
for sock in self.sockets.iteritems():
print sock
sock[1][0].shutdown(socket.SHUT_RDWR)
sock[1][0].close()
self.sockets = {}
def run_server():
test_server = TestServer(('127.0.0.1', 10666))
test_server.start()
return test_server
And my test looks like this:
def test_can_reconnect(self):
test_server = run_server()
client_config = {'host': '127.0.0.1', 'port': 10666}
client = Connection('test client', client_config, get_config())
client.connect()
assert client.socket_connected
test_server.murder()
#time.sleep(4) #tried sleeping. no dice.
assert not client.socket_connected
assert client.server_disconnect
test_server = run_server()
client.reconnect()
assert client.socket_connected
It fails at assert not client.socket_connected.
I detect for "not data" during recv. If it's None, then I set some variables so that other code can decide whether or not to reconnect (don't reconnect if it was a user_disconnect and so on). This behavior works and has always worked for me in the past, I've just never tried to make a test for it until now. Is there something odd with socket connections and local function scopes or something? it's like the connection still exists even after stopping the server.
The code I'm trying to test is open: https://github.com/kyleterry/tenyks.git
If you run the tests, you will see the one I'm trying to fix fail.
Trying to run a unit test with a real socket is a tough row to hoe. It's going to be tricky as only one set of tests can run at a time, as the server port will be used, and it's going to be slow as the sockets get set up and torn down. To top it off if this is really a unit test you don't want to test the socket, just the code that's using the socket.
If you mock the socket calls you can throw exceptions willy nilly from the mocked code and ensure that the code making use of the socket does the right thing. You don't need a real socket to ensure that the class under test does the right thing, you can fake it if you can wrap the socket calls in an object. Pass in a reference to the socket object when constructing your class and you're ready to go.
My suggestion is to wrap the socket calls in a class that supports sendall, recv, and all the methods you call on the socket. Then you can swap out the actual Socket class with a TestReconnectSocket (or whatever) and run your tests.
Take a look at mox, a python mocking framework.
Vague response, but my immediate reaction would be that your recv() call is blocking and keeping the socket alive - have you tried making the socket non-blocking, and catching the error on close instead?
One thing to keep in mind when testing sockets like this, is that operating systems don't like to reopen a socket soon after it has been in use. You can set a socket option to tell it to go ahead and reuse it anyways. Right after you create the socket set the socket's option:
mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Hopefully this will fix your issue. You may have to do it on both the server and client side depending on which one is giving you the problems.
you are calling shutdown(socket.SHUT_RDWR) so this doesn't seem like a problem with recv blocking.
however, you are using gevent.socket.socket.recv, so please check your gevent version, there is an issue with recv() that causes it to block if the underlying file descriptor is closed (version < v0.13.0)
you may still need gevent.sleep() to do cooperative yield and give the client an opportunity to exit the recv() call.
Dear all, I need to implement a TCP server in Python which receives some data from a client and then sends this data to another client. I've tried many different implementations but no way to make it run. Any help would be really appreciated.
Below is my code:
import SocketServer
import sys
import threading
buffer_size = 8182
ports = {'toserver': int(sys.argv[1]), 'fromserver': int(sys.argv[2])}
class ConnectionHandler(SocketServer.BaseRequestHandler):
def handle(self):
# I need to send the data received from the client connected to port 'toserver'
# to the client connected to port 'fromserver' - see variable 'ports' above
class TwoWayConnectionServer(threading.Thread):
def __init__(self):
self.to_server = SocketServer.ThreadingTCPServer(("", ports['toserver']), ConnectionHandler)
self.from_server = SocketServer.ThreadingTCPServer(("", ports['fromserver']), ConnectionHandler)
threading.Thread.__init__(self)
def run(self):
while (1):
self.to_server.handle_request()
self.from_server.handle_request()
def serve_non_blocking():
server = TwoWayConnectionServer()
server.run()
if __name__ == '__main__':
serve_non_blocking()
See the Twisted tutorial, and also twisted.protocols.portforward. I think that portforward module does something slightly different from what you want, it opens an outgoing connection to the destination port rather than waiting for the second client to connect, but you should be able to work from there.
Can you be more specific about what you tried and what didn't work? There are lots of ways to do this. Probably the easiest would be to use the socket library - maybe looking at some examples will help:
http://docs.python.org/library/socket.html#example