We Keep Coding

Python OSC, Query / Close active thread

I'm using a relatively simple python execution, using OSC modules, in order to 'Send' code, from an application to an other.
import OSC
import threading
#------OSC Server-------------------------------------#
receive_address = '127.0.0.1', 9002
# OSC Server. there are three different types of server.
s = OSC.ThreadingOSCServer(receive_address)
# define a message-handler function for the server to call.
def printing_handler(addr, tags, stuff, source):
if addr=='/coordinates':
print "Test", stuff
s.addMsgHandler("/coordinates", printing_handler)
def main():
# Start OSCServer
print "Starting OSCServer"
st = threading.Thread(target=s.serve_forever)
st.start()
main()
Runned once, will work just fine, an listen on port 9002.
But, runned twice, will return ERROR:
socket.error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
My goal is to:
Be able to query on active thread's port
Close them
I've tried the following...
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = s.connect_ex(('127.0.0.1', 900socket2))
print 'RESULT: ', result
s.close()
But giving me unsuccessful result. (Returns 10061 both for active and unactive port's thread)

Python » Documentation socketserver.BaseServer:
shutdown()
Tell the serve_forever() loop to stop and wait until it does.
server_close()
Clean up the server. May be overridden.

Python sctp module - server side

I have been trying to test SCTP for a network deployment.
I do not have an SCTP server or client and was hoping to be able use pysctp.
I am fairly certain that I have the client side code working.
def sctp_client ():
print("SCTP client")
sock = sctp.sctpsocket_tcp(socket.AF_INET)
#sock.connect(('10.10.10.70',int(20003)))
sock.connect(('10.10.10.41',int(21000)))
print("Sending message")
sock.sctp_send(msg='allowed')
sock.shutdown(0)
sock.close()
Has anybody had luck with using the python sctp module for the server side?
Thank you in Advance!

I know that this topic's a bit dated, but I figured I would respond to it anyway to help out the community.
In a nutshell:
you are using pysctp with the sockets package to create either a client or a server;
you can therefore create your server connection as you normally would with a regular TCP connection.
Here's some code to get you started, it's a bit verbose, but it illustrates a full connection, sending, receiving, and closing the connection.
You can run it on your dev computer and then use a tool like ncat (nmap's implementation of netcat) to connect, i.e.: ncat --sctp localhost 80.
Without further ado, here's the code... HTH:
# Here are the packages that we need for our SCTP server
import socket
import sctp
from sctp import *
import threading
# Let's create a socket:
my_tcp_socket = sctpsocket_tcp(socket.AF_INET)
my_tcp_port = 80
# Here are a couple of parameters for the server
server_ip = "0.0.0.0"
backlog_conns = 3
# Let's set up a connection:
my_tcp_socket.events.clear()
my_tcp_socket.bind((server_ip, my_tcp_port))
my_tcp_socket.listen(backlog_conns)
# Here's a method for handling a connection:
def handle_client(client_socket):
client_socket.send("Howdy! What's your name?\n")
name = client_socket.recv(1024) # This might be a problem for someone with a reaaallly long name.
name = name.strip()
print "His name was Robert Paulson. Er, scratch that. It was {0}.".format(name)
client_socket.send("Thanks for calling, {0}. Bye, now.".format(name))
client_socket.close()
# Now, let's handle an actual connection:
while True:
client, addr = my_tcp_socket.accept()
print "Call from {0}:{1}".format(addr[0], addr[1])
client_handler = threading.Thread(target = handle_client,
args = (client,))
client_handler.start()

Unless you need the special sctp_ functions you don't need an sctp module at all.
Just use protocol 132 as IPPROTO_SCTP (is defined on my python3 socket module but not on my python2 socket module) and you can use the socket,bind,listen,connect,send,recv,sendto,recvfrom,close from the standard socket module.
I'm doing some SCTP C development and I used python to better understand SCTP behavior without the SCTP module.

How to kill a socket in unit tests for reconnect test

I'm trying to test some code that reconnects to a server after a disconnect. This works perfectly fine outside the tests, but it fails to acknowledge that the socket has disconnected when running the tests.
I'm using a Gevent Stream Server to mock a real listening server:
import gevent.server
from gevent import queue
class TestServer(gevent.server.StreamServer):
def __init__(self, *args, **kwargs):
super(TestServer, self).__init__(*args, **kwargs)
self.sockets = {}
def handle(self, socket, address):
self.sockets[address] = (socket, queue.Queue())
socket.sendall('testing the connection\r\n')
gevent.spawn(self.recv, address)
def recv(self, address):
socket = self.sockets[address][0]
queue = self.sockets[address][1]
print 'Connection accepted %s:%d' % address
try:
for data in socket.recv(1024):
queue.put(data)
except:
pass
def murder(self):
self.stop()
for sock in self.sockets.iteritems():
print sock
sock[1][0].shutdown(socket.SHUT_RDWR)
sock[1][0].close()
self.sockets = {}
def run_server():
test_server = TestServer(('127.0.0.1', 10666))
test_server.start()
return test_server
And my test looks like this:
def test_can_reconnect(self):
test_server = run_server()
client_config = {'host': '127.0.0.1', 'port': 10666}
client = Connection('test client', client_config, get_config())
client.connect()
assert client.socket_connected
test_server.murder()
#time.sleep(4) #tried sleeping. no dice.
assert not client.socket_connected
assert client.server_disconnect
test_server = run_server()
client.reconnect()
assert client.socket_connected
It fails at assert not client.socket_connected.
I detect for "not data" during recv. If it's None, then I set some variables so that other code can decide whether or not to reconnect (don't reconnect if it was a user_disconnect and so on). This behavior works and has always worked for me in the past, I've just never tried to make a test for it until now. Is there something odd with socket connections and local function scopes or something? it's like the connection still exists even after stopping the server.
The code I'm trying to test is open: https://github.com/kyleterry/tenyks.git
If you run the tests, you will see the one I'm trying to fix fail.

Trying to run a unit test with a real socket is a tough row to hoe. It's going to be tricky as only one set of tests can run at a time, as the server port will be used, and it's going to be slow as the sockets get set up and torn down. To top it off if this is really a unit test you don't want to test the socket, just the code that's using the socket.
If you mock the socket calls you can throw exceptions willy nilly from the mocked code and ensure that the code making use of the socket does the right thing. You don't need a real socket to ensure that the class under test does the right thing, you can fake it if you can wrap the socket calls in an object. Pass in a reference to the socket object when constructing your class and you're ready to go.
My suggestion is to wrap the socket calls in a class that supports sendall, recv, and all the methods you call on the socket. Then you can swap out the actual Socket class with a TestReconnectSocket (or whatever) and run your tests.
Take a look at mox, a python mocking framework.

Vague response, but my immediate reaction would be that your recv() call is blocking and keeping the socket alive - have you tried making the socket non-blocking, and catching the error on close instead?

One thing to keep in mind when testing sockets like this, is that operating systems don't like to reopen a socket soon after it has been in use. You can set a socket option to tell it to go ahead and reuse it anyways. Right after you create the socket set the socket's option:
mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Hopefully this will fix your issue. You may have to do it on both the server and client side depending on which one is giving you the problems.

you are calling shutdown(socket.SHUT_RDWR) so this doesn't seem like a problem with recv blocking.
however, you are using gevent.socket.socket.recv, so please check your gevent version, there is an issue with recv() that causes it to block if the underlying file descriptor is closed (version < v0.13.0)
you may still need gevent.sleep() to do cooperative yield and give the client an opportunity to exit the recv() call.

How to determine from a python application if X server/X forwarding is running?

I'm writing a linux application which uses PyQt4 for GUI and which will only be used during remote sessions (ssh -XY / vnc).
So sometimes it may occur that a user will forget to run ssh with X forwarding parameters or X forwarding will be unavailable for some reason. In this case the application crashes badly (unfortunately I am force to use an old C++ library wrapped into python and it completely messes user's current session if the application crashes).
I cannot use something else so my idea is to check if X forwarding is available before loading that library. However I have no idea how to do that.
I usually use xclock to check if my session has X forwarding enabled, but using xclock sounds like a big workaround.
ADDED
If possible I would like to use another way than creating an empty PyQt window and catching an exception.

Check to see that the $DISPLAY environment variable is set - if they didn't use ssh -X, it will be empty (instead of containing something like localhost:10).

As mentioned before, you can check the DISPLAY environment variable:
>>> os.environ['DISPLAY']
'localhost:10.0'
If you're so inclined, you could actually connect to the display port to see that sshd is listening on it:
import os
import socket
def tcp_connect_to_display():
# get the display from the environment
display_env = os.environ['DISPLAY']
# parse the display string
display_host, display_num = display_env.split(':')
display_num_major, display_num_minor = display_num.split('.')
# calculate the port number
display_port = 6000 + int(display_num_major)
# attempt a TCP connection
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((display_host, display_port))
except socket.error:
return False
finally:
sock.close()
return True
This relies on standard X configuration using ports 6000 + display number.

Similar to your xclock solution, I like to run xdpyinfo and see if it returns an error.

X-Server can be checked with TkInter as in the following example (but there should be a similar way with PyQt4):
import time
import sys
try:
import Tkinter as tk
except ImportError:
import tkinter as tk
while True:
try:
root = tk.Tk()
break
except tk.TclError as e:
if "$DISPLAY" in str(e):
print("$DISPLAY not set. Exiting.")
sys.exit(1)
print("Waiting for X server to start...")
time.sleep(1)
print("X server running")
root.destroy()
sys.exit(0)
This will check $DISPLAY setting, X-Server process and related xhost authorization (but uses TkInter instead of PyQt4).

Multiple simultaneous network connections - Telnet server, Python

I'm currently writing a telnet server in Python. It's a content server. People would connect to the server via telnet, and be presented with text-only content.
My problem is that the server would obviously need to support more than one simultaneous connection. The current implementation I have now supports only one.
This is the basic, proof-of-concept server I began with (while the program has changed greatly over time, the basic telnet framework hasn't):
import socket, os
class Server:
def __init__(self):
self.host, self.port = 'localhost', 50000
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.host, self.port))
def send(self, msg):
if type(msg) == str: self.conn.send(msg + end)
elif type(msg) == list or tuple: self.conn.send('\n'.join(msg) + end)
def recv(self):
self.conn.recv(4096).strip()
def exit(self):
self.send('Disconnecting you...'); self.conn.close(); self.run()
# closing a connection, opening a new one
# main runtime
def run(self):
self.socket.listen(1)
self.conn, self.addr = self.socket.accept()
# there would be more activity here
# i.e.: sending things to the connection we just made
S = Server()
S.run()
Thanks for your help.

Implemented in twisted:
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
class SendContent(Protocol):
def connectionMade(self):
self.transport.write(self.factory.text)
self.transport.loseConnection()
class SendContentFactory(Factory):
protocol = SendContent
def __init__(self, text=None):
if text is None:
text = """Hello, how are you my friend? Feeling fine? Good!"""
self.text = text
reactor.listenTCP(50000, SendContentFactory())
reactor.run()
Testing:
$ telnet localhost 50000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello, how are you my friend? Feeling fine? Good!
Connection closed by foreign host.
Seriously, when it comes to asynchronous network, twisted is the way to go. It handles multiple connections in a single-thread single-process approach.

You need some form of asynchronous socket IO. Have a look at this explanation, which discusses the concept in low-level socket terms, and the related examples which are implemented in Python. That should point you in the right direction.

Late for the reply, but with the only answers being Twisted or threads (ouch), I wanted to add an answer for MiniBoa.
http://code.google.com/p/miniboa/
Twisted is great, but it's a rather large beast that may not be the best introduction to single-threaded asynchronous Telnet programming. MiniBoa is a lightweight, asynchronous single-threaded Python Telnet implementation originally designed for muds, which suits the OP's question perfectly.

For a really easy win implement you solution using SocketServer & the SocketServer.ThreadingMixIn
have a look a this echo server example it looks quite similar to what you're doing anyway: http://www.oreillynet.com/onlamp/blog/2007/12/pymotw_socketserver.html

If you're up for a bit of a conceptual challenge, I'd look into using twisted.
Your case should be trivial to implement as a part of twisted.
http://twistedmatrix.com/projects/core/documentation/howto/servers.html

If you want to do it in pure python (sans-twisted), you need to do some threading. If you havnt seen it before, check out:
http://heather.cs.ucdavis.edu/~matloff/Python/PyThreads.pdf
around page 5/6 is an example that is very relevant ;)

First, buy Comer's books on TCP/IP programming.
In those books, Comer will provide several alternative algorithms for servers. There are two standard approaches.
Thread-per-request.
Process-per-request.
You must pick one of these two and implement that.
In thread-per, each telnet session is a separate thread in your overall application.
In process-per, you fork each telnet session into a separate subprocess.
You'll find that process-per-request is much, much easier to handle in Python, and it generally makes more efficient use of your system.
Thread-per-request is fine for things that come and go quickly (like HTTP requests). Telnet has long-running sessions where the startup cost for a subprocess does not dominate performance.

Try MiniBoa server? It has exactly 0 dependencies, no twisted or other stuff needed. MiniBoa is a non-blocking async telnet server, single threaded, exactly what you need.
http://code.google.com/p/miniboa/

Use threading and then add the handler into a function. The thread will call every time a request i made:
Look at this
import socket # Import socket module
import pygame
import thread
import threading,sys
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345 # Reserve a port for your service.
s.bind((host, port))
print ((host, port))
name = ""
users = []
def connection_handler (c, addr):
print "conn handle"
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
if a == "o":
c.send (str(users))
a = c.recv (1024)
if a == "c":
b = c.recv (1024)
print a,b
s.listen(6) # Now wait for client connection.
while True:
c, addr = s.accept()
print 'Connect atempt from:', addr[0]
username = c.recv(1024)
print "2"
if username == "END_SERVER_RUBBISH101":
if addr[0] == "192.168.1.68":
break
users.append(username)
thread.start_new_thread (connection_handler, (c, addr)) #New thread for connection
print
s.close()