We Keep Coding

Socket Server refuses multiple connections with threading

I'm trying to do a multi-threaded python socket server, I created one thread that listens for connections and accepts them, within it I also creates sub-threads for each new connection.
PROBLEM: after a connection is established with one client , when another client tries to connect i get the following error in the client-side:
connectionrefusederror: [winerror 10061] no connection could be made because the target machine actively refused it >>> in multiclientgui
I didn't wanna try multiprocessing to create a process for listening only,
Is it something in my code? Or in the OS (windows 10)? or what would it be?
def serv():
""" the main thread for listening"""
global future ,soc ,add
while True:
s.listen(5)
soc,add =s.accept()
future =pool.submit(clientHandle,soc,add) #handling thread
print(" a new thread created to serve the client")
def clientHandle(socObj, address):
print("thread been called")
global client_data
client_data =pd.DataFrame(columns=['Location','lat','long','speed','ID','Time'])
client_sockets.append(socObj)
client_addr.append(address)
while True :
content=socObj.recv(1024)
if len(content)>0:
data=content.decode()
str(data)
print(data)

How can I detect when my socket has lost its TCP connection and reconnect?

My application has an issue where if it loses its connection, it blocks forever and the whole application needs to be restarted.
Code in question:
def run(encoder, port, channel):
decoder_socket = socket.socket()
decoder_socket.connect((decoder, port))
decoder_fp = decoder_socket.makefile("r", 0)
parser = xml.sax.make_parser(['xml.sax.IncrementalParser'])
parser.setContentHandler(RftDecoder())
while True:
data = decoder_fp.readline().strip("\x00")
if not(data):
raise socket.error("no data")
LOG.info("%s" % data.strip())
parser.feed(data)
If the connection is actively closed by the other side, the exception is raised as expected, no problems. However when the connection is dropped (e.g., if Decoder crashes, or if the cable is unplugged), then the decoder_fp.readline() call blocks indefinitely.
Strace doesn't show anything when this happens, it just continues sitting on a recv() call.
So my question is, how can I detect when the connection is lost as described, and reconnect?

Python OSC, Query / Close active thread

I'm using a relatively simple python execution, using OSC modules, in order to 'Send' code, from an application to an other.
import OSC
import threading
#------OSC Server-------------------------------------#
receive_address = '127.0.0.1', 9002
# OSC Server. there are three different types of server.
s = OSC.ThreadingOSCServer(receive_address)
# define a message-handler function for the server to call.
def printing_handler(addr, tags, stuff, source):
if addr=='/coordinates':
print "Test", stuff
s.addMsgHandler("/coordinates", printing_handler)
def main():
# Start OSCServer
print "Starting OSCServer"
st = threading.Thread(target=s.serve_forever)
st.start()
main()
Runned once, will work just fine, an listen on port 9002.
But, runned twice, will return ERROR:
socket.error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
My goal is to:
Be able to query on active thread's port
Close them
I've tried the following...
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = s.connect_ex(('127.0.0.1', 900socket2))
print 'RESULT: ', result
s.close()
But giving me unsuccessful result. (Returns 10061 both for active and unactive port's thread)

Python » Documentation socketserver.BaseServer:
shutdown()
Tell the serve_forever() loop to stop and wait until it does.
server_close()
Clean up the server. May be overridden.

Python: How to interrupt raw_input() in other thread

I am writing a simple client-server program in python. In the client program, I am creating two threads (using Python's threading module), one for receiving, one for sending. The receiving thread continuously receives strings from the server side; while the sending thread continuously listens to the user input (using raw_input()) and send it to the server side. The two threads communicate using a Queue (which is natively synchronized, LIKE!).
The basic logic is like following:
Receiving thread:
global queue = Queue.Queue(0)
def run(self):
while 1:
receive a string from the server side
if the string is QUIT signal:
sys.exit()
else:
put it into the global queue
Sending thread:
def run(self):
while 1:
str = raw_input()
send str to the server side
fetch an element from the global queue
deal with the element
As you can see, in the receiving thread, I have a if condition to test whether the server has sent a "QUIT signal" to the client. If it has, then I want the whole program to stop.
The problem here is that for most of its time, the sending thread is blocked by "raw_input()" and waiting for the user input. When it is blocked, calling "sys.exit()" from the other thread (receiving thread) will not terminate the sending thread immediately. The sending thread has to wait for the user to type something and hit the enter button.
Could anybody inspire me how to get around with this? I do not mind using alternatives of "raw_input()". Actually I do not even mind changing the whole structure.
-------------EDIT-------------
I am running this on a linux machine, and my Python version is 2.7.5

You could just make the sending thread daemonic:
send_thread = SendThread() # Assuming this inherits from threading.Thread
send_thread.daemon = True # This must be called before you call start()
The Python interpreter won't be blocked from exiting if the only threads left running are daemons. So, if the only thread left is send_thread, your program will exit, even if you're blocked on raw_input.
Note that this will terminate the sending thread abruptly, no matter what its doing. This could be dangerous if it accesses external resources that need to be cleaned up properly or shouldn't be interrupted (like writing to a file, for example). If you're doing anything like that, protect it with a threading.Lock, and only call sys.exit() from the receiving thread if you can acquire that same Lock.

The short answer is you can't. input() like a lot of such input commands is blocking and it's blocking whether everything about the thread has been killed. You can sometimes call sys.exit() and get it to work depending on the OS, but it's not going to be consistent. Sometimes you can kill the program by deferring out to the local OS. But, then you're not going to be widely cross platform.
What you might want to consider if you have this is to funnel the functionality through the sockets. Because unlike input() we can do timeouts, and threads and kill things rather easily. It also gives you the ability to do multiple connections and maybe accept connections more broadly.
import socket
import time
from threading import Thread
def process(command, connection):
print("Command Entered: %s" % command)
# Any responses are written to connection.
connection.send(bytes('>', 'utf-8'))
class ConsoleSocket:
def __init__(self):
self.keep_running_the_listening_thread = True
self.data_buffer = ''
Thread(target=self.tcp_listen_handle).start()
def stop(self):
self.keep_running_the_listening_thread = False
def handle_tcp_connection_in_another_thread(self, connection, addr):
def handle():
while self.keep_running_the_listening_thread:
try:
data_from_socket = connection.recv(1024)
if len(data_from_socket) != 0:
self.data_buffer += data_from_socket.decode('utf-8')
else:
break
while '\n' in self.data_buffer:
pos = self.data_buffer.find('\n')
command = self.data_buffer[0:pos].strip('\r')
self.data_buffer = self.data_buffer[pos + 1:]
process(command, connection)
except socket.timeout:
continue
except socket.error:
if connection is not None:
connection.close()
break
Thread(target=handle).start()
connection.send(bytes('>', 'utf-8'))
def tcp_listen_handle(self, port=23, connects=5, timeout=2):
"""This is running in its own thread."""
sock = socket.socket()
sock.settimeout(timeout)
sock.bind(('', port))
sock.listen(connects) # We accept more than one connection.
while self.keep_running_the_listening_thread:
connection = None
try:
connection, addr = sock.accept()
address, port = addr
if address != '127.0.0.1': # Only permit localhost.
connection.close()
continue
# makes a thread deals with that stuff. We only do listening.
connection.settimeout(timeout)
self.handle_tcp_connection_in_another_thread(connection, addr)
except socket.timeout:
pass
except OSError:
# Some other error.
if connection is not None:
connection.close()
sock.close()
c = ConsoleSocket()
def killsocket():
time.sleep(20)
c.stop()
Thread(target=killsocket).start()
This launches a listener thread for the connections set on port 23 (telnet), and you connect and it passes that connection off to another thread. And it starts a killsocket thread that disables the various threads and lets them die peacefully (for demonstration purposes). You cannot however connect localhost within this code, because you'd need input() to know what to send to the server, which recreates the problem.

Close listening socket in python thread

I have a problem trying to learn about sockets for network communication. I have made a simple thread that listens for connections and creates processes for connecting clients, my problem though is that I can't get the thread to join properly as I haven't found a way to cancel the socket.accept()-call when I want to quit the program.
My code looks like this;
class ServerThread( threading.Thread ):
def __init__(self, queue, host, port):
threading.Thread.__init__(self)
self.queue = queue
self.running = True
self.hostname = host
self.port = port
def run(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.hostname, self.port))
self.socket.listen(1)
while self.running:
try:
conn, address = self.socket.accept()
process = Process(target=server_slave, args=(conn, address, self.queue))
process.daemon = True
process.start()
except socket.timeout:
pass
def stop(self):
self.running = False
self.socket.close()
I have managed to get the program to close by setting self.setDaemon(True) and just exiting the main program, handing everything to the great garbage collector - but that seems like a bad solution. I've also tried setting a timeout for the socket but that results in getting [Errno 35] Resource temporarily unavailable (regardless of the actual timeout, even when I set it to years...).
What am I doing wrong? Have I designed the thread in a dumb way or have I missed something about accepting connections?

One way to get the thread to close seems to be to make a connection to the socket, thus continuing the thread to completion.
def stop(self):
self.running = False
socket.socket(socket.AF_INET,
socket.SOCK_STREAM).connect( (self.hostname, self.port))
self.socket.close()
This works, but it still feels like it might not be optimal...

In most cases you will open a new thread or process once a connection is accepted. To close the connection, break the while loop. Garbage collection will remove the thread or process but join will ensure none get left behind.
Persistent sockets close when the user closes them or they timeout. Non-persistent, like static webpages will close after they've sent the information.
Here's a good example of a persistent socket server in Python. It uses multiprocessing which means it can run across multiple cores for CPU-bound tasks. More commonly known as multithreading.
import socket
import multiprocessing
def run():
host = '000.000.000.000'
port = 1212
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', port))
sock.listen(3)
while True:
p = multiprocessing.Process(target=worker, args=sock.accept()).start()
def worker(conn, addr):
while True:
if data == '':
#remote connection closed
break
if len(dataList) > 2:
# do stuff
print 'This code is untested'
run()

A dirty solution which allows to exit your program is to use os._exit(0).
def stop(self):
self.socket.close()
os._exit(0)
note that sys.exit doesn't work/blocks as it tries to exit cleanly/release resources. But os._exit is the most low level way and it works, when nothing else does.
The operating system itself will release the resources (on any modern system) like when doing exit in a C program.

The best way to do this is to have a single listening thread that has nothing to do with your connection threads and give it a reasonable length timeout. On timeout, check if this thread should shutdown and if not, loop again and go back to listening.
def tcp_listen_handle(self, port=23, connects=5, timeout=2):
"""This is running in its own thread."""
sock = socket.socket()
sock.settimeout(timeout)
sock.bind(('', port))
sock.listen(connects) # We accept more than one connection.
while self.keep_running_the_listening_thread():
connection = None
addr = None
try:
connection, addr = sock.accept()
print("Socket Connected: %s" % str(addr))
# makes a thread deals with that stuff. We only do listening.
self.handle_tcp_connection_in_another_thread(connection, addr)
except socket.timeout:
pass
except OSError:
# Some other error.
print("Socket was killed: %s" % str(addr))
if connection is not None:
connection.close()
sock.close()
The only thing this does is listen, timeout, checks if it should die during the timeout, and goes back to listening. The general rule of thumb is that threads should check whether they should die and try to do that themselves as fast as they can. And if you don't want to take the 2 second hit for timeout wait before the thread unblocks and checks. You can connect to it yourself.

Partially tested solution
Put self.socket.settimeout(0.1) right before while
Put conn.settimeout(None) right after accept