I am preparing new driver (that should work over TCP/IP) and I am having kind of issue.
The main idea is, that there will be two separated loops.
Loop binds on port and keeps listening for incoming connection requests. Once a request is accepted, it authorizes client, if authorized, passes his connection to second loop. After client is passed to second loop, it should continue in listening for incoming connections. If new client connects, it is passed to second loop, and so on..
Second loop takes connected client, and manages sending and receiving data with him. However, it also checks if there is not some new connection, and if so, it will close actual connection to client and use new one.
This should ensure that if client connection is lost, we do not have to wait for timeout to get new connection (which is the case why I am creating this new driver). If new connection is activated, we just close the old one and continue communication with modem with much shorter break time.
Here is the code (simplified):
class SocketDriver(Process):
def __init__(self):
Process.__init__(self)
self.stop_event = Event()
self.client = None
self.addr = None
self.client_queue = Queue()
def connection_manager(self):
while not self.stop_event.is_set():
try:
log.info('Binding to %s on port: %s' % (self.atmel_name, self.port))
self.socket_object.bind((self.ip, self.port))
break
except socket.error, e:
if e.errno == errno.EADDRINUSE:
log.error('Socket error: %s, re-trying' % e)
time.sleep(5)
else:
log.exception('Unknown exception')
break
while not self.stop_event.is_set():
self.socket_object.listen(5)
log.info('Waiting for incoming connections')
# This cannot be touched, because accept command wont pass until something connect
client, addr = self.socket_object.accept()
log.info('Accepted connection from %s:%s' % (addr[0], addr[1]))
self.client_queue.put([client, addr])
def open(self, port, bootloader=False):
"""
Open port and load drivers
"""
self.socket_object = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(self.connection_manager ,())
def run(self):
"""
Loop throught FIFO structures, check for incomming packet,
send to peripheral device, receive response and deliver it
"""
log.info('Starting SocketDriver loop')
while not self.stop_event.is_set():
# Watchdog touch
open(touchfile, 'w').close()
#Check if there is new client available
if not self.client_queue.empty():
log.info('New client found!')
if self.client != None:
self.client.close()
conn_data = self.client_queue.get()
self.client = conn_data[0]
self.addr = conn_data[1]
log.info('Connected to new client with address: %s:%s' % (self.addr[0], self.addr[1]))
# Check if there is any new connection
if self.client != None:
log.info('Client found, checking queues')
for out_queue, in_queue, driver_id in self.fifocom_list:
while not self.stop_event.is_set():
if not out_queue.empty():
self.write_packet(atmel_packet, driver_id)
# Vycitame a spracujeme odpoved
ready = select.select([self.client], [], [], 1)
if ready[0]:
log.info('[%s] Reading', self.source_name)
atmel_packet = self.read_packet()
time.sleep(0.1)
The basic idea is to keep listening for new connections while communicating with actually connected client.
However, when I put anything else into "self.client_queue", the "if not self.client_queue.empty():" works. However, when I put there client and address from "self.socket_object.accept()", it crushes:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/queues.py", line 266, in _feed
send(obj)
TypeError: expected string or Unicode object, NoneType found
Could anyone please explain what is going on here? I have read that there is problem with moving opened socets between processes, but the thread is being run within one process.
Related
hi i make model server client which works fine and i also create separate GUI which need to two input server IP and port it only check whether server is up or not. But when i run server and then run my GUI and enter server IP and port it display connected on GUI but on server side it throw this error. The Server Client working fine but integration of GUI with server throw below error on server side.
conn.send('Hi'.encode()) # send only takes string BrokenPipeError: [Errno 32] Broken pip
This is server Code:
from socket import *
# Importing all from thread
import threading
# Defining server address and port
host = 'localhost'
port = 52000
data = " "
# Creating socket object
sock = socket()
# Binding socket to a address. bind() takes tuple of host and port.
sock.bind((host, port))
# Listening at the address
sock.listen(5) # 5 denotes the number of clients can queue
def clientthread(conn):
# infinite loop so that function do not terminate and thread do not end.
while True:
# Sending message to connected client
conn.send('Hi'.encode('utf-8')) # send only takes string
data =conn.recv(1024)
print (data.decode())
while True:
# Accepting incoming connections
conn, addr = sock.accept()
# Creating new thread. Calling clientthread function for this function and passing conn as argument.
thread = threading.Thread(target=clientthread, args=(conn,))
thread.start()
conn.close()
sock.close()
This is part of Gui Code which cause problem:
def isOpen(self, ip, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect((ip, int(port)))
data=s.recv(1024)
if data== b'Hi':
print("connected")
return True
except:
print("not connected")
return False
def check_password(self):
self.isOpen('localhost', 52000)
Your problem is simple.
Your client connects to the server
The server is creating a new thread with an infinite loop
The server sends a simple message
The client receives the message
The client closes the connection by default (!!!), since you returned from its method (no more references)
The server tries to receive a message, then proceeds (Error lies here)
Since the connection has been closed by the client, the server cannot send nor receive the next message inside the loop, since it is infinite. That is the cause of the error! Also there is no error handling in case of closing the connection, nor a protocol for closing on each side.
If you need a function that checks whether the server is online or not, you should create a function, (but I'm sure a simple connect is enough), that works like a ping. Example:
Client function:
def isOpen(self, ip, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect((str(ip), int(port)))
s.send("ping".encode('utf-8'))
return s.recv(1024).decode('utf-8') == "pong" # return whether the response match or not
except:
return False # cant connect
Server function:
def clientthread(conn):
while True:
msg = conn.recv(1024).decode('utf-8') #receiving a message
if msg == "ping":
conn.send("pong".encode('utf-8')) # sending the response
conn.close() # closing the connection on both sides
break # since we only need to check whether the server is online, we break
From your previous questions I can tell you have some problems understanding how TCP socket communication works. Please take a moment and read a few articles about how to communicate through sockets. If you don't need live communications (continous data stream, like a video, game server, etc), only login forms for example, please stick with well-known protocols, like HTTP. Creating your own reliable protocol might be a little complicated if you just got into socket programming.
You could use flask for an HTTP back-end.
I'm trying to run new process for each new instance of class Server. Each Server instance should listen on specific port. I have this (simplified) code so far: source
class Server(object):
def handle(connection, address):
print("OK...connected...")
try:
while True:
data = connection.recv(1024)
if data == "":
break
connection.sendall(data)
except Exception as e:
print(e)
finally:
connection.close()
def __init__(self, port, ip):
self.port = port
self.ip = ip
self.socket = socket(AF_INET, SOCK_STREAM)
self.socket.bind((self.ip, self.port))
self.socket.listen(1)
while True:
print("Listening...")
conn, address = self.socket.accept()
process = multiprocessing.Process(target=Pmu.handle, args=(conn, address))
process.daemon = True
process.start()
s1 = Server(9001,"127.0.0.1")
s2 = Server(9002,"127.0.0.1")
But when I run this script only first server s1 is running and waiting for connection. How to make both servers listening at the same time?
Your current server is effectively a SocketServer.ForkingTCPServer that enters a tight loop in its __init__, foerever accepting new connections, and creating a new child process for each incoming connection.
The problem is that __init__ never returns, so only one server gets instantiated, one socket gets bound, and only one port will accept new requests.
A common way of solving this type of problem is to move the accept loop into a worker thread. This code would look something like this:
import multiprocessing
import threading
import socket
class Server(object):
def handle(self, connection, address):
print("OK...connected...")
try:
while True:
data = connection.recv(1024)
if data == "":
break
connection.sendall(data)
except Exception as e:
print(e)
finally:
connection.close()
print("Connection closed")
def accept_forever(self):
while True:
# Accept a connection on the bound socket and fork a child process
# to handle it.
print("Waiting for connection...")
conn, address = self.socket.accept()
process = multiprocessing.Process(
target=self.handle, args=(conn, address))
process.daemon = True
process.start()
# Close the connection fd in the parent, since the child process
# has its own reference.
conn.close()
def __init__(self, port, ip):
self.port = port
self.ip = ip
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.ip, self.port))
self.socket.listen(1)
# Spin up an acceptor thread
self.worker = threading.Thread(target=self.accept_forever)
self.worker.daemon = True
self.worker.start()
def join(self):
# threading.Thread.join() is not interruptible, so tight loop
# in a sleep-based join
while self.worker.is_alive():
self.worker.join(0.5)
# Create two servers that run in the background
s1 = Server(9001,"127.0.0.1")
s2 = Server(9002,"127.0.0.1")
# Wait for servers to shutdown
s1.join()
s2.join()
Note one other change I snuck in here:
# Wait for servers to shutdown
s1.join()
s2.join()
Using the saved reference to the Server's accept worker, we call .join() from the main thread to force things to block while the servers are running. Without this, your main program will exit nearly immediately, due to the workers' .daemon attribute being set.
It's also worth noting that this approach will have some quirks:
Since the handler functions are running in separate processes, you will need to share data structures between them carefully using Queue, Value, Pipe, and other multiprocessing constructs if they depend on each other.
There is no rate limiting of active concurrent connections; creating a new process for every single request can be expensive, and can create a vector for your service being easily DoSed.
I am doing a client-server project for my college project,
we have to allocate the login to the client.
Client system will request its status for every 2 seconds(to check whether the client is locked or unlocked). and server will accept the client request and reply the client status to the system.
But the problem is server thread is not responding to the client request.
CLIENT THREAD:
def checkPort():
while True:
try:
s = socket.socket()
s.connect((host, port))
s.send('pc1') # send PC name to the server
status = s.recv(1024) # receive the status from the server
if status == "unlock":
disableIntrrupts() # enable all the functions of system
else:
enableInterrupts() # enable all the functions of system
time.sleep(5)
s.close()
except Exception:
pass
SERVER THREAD:
def check_port():
while True:
try:
print "hello loop is repeating"
conn, addr = s.accept()
data = conn.recv(1024)
if exit_on_click == 1:
break
if (any(sublist[0] == data for sublist in available_sys)):
print "locked"
conn.send("lock")
elif (any(sublist[0] == data for sublist in occupied_sys)):
conn.send("unlock")
print "unlocked"
else:
print "added to gui for first time"
available_sys.append([data,addr[0],nameText,usnText,branchText])
availSysList.insert('end',data)
except Exception:
pass
But my problem is server thread is not executing more than 2 time,
So its unable to accept client request more than one time.
can't we handle multiple client sockets using single server socket?
How to handle multiple client request from server ?
Thanks for any help !!
Its because your server, will block waiting for a new connection on this line
conn, addr = s.accept()
This is because calls like .accept and .read are blocking calls that hold the process
You need to consider an alternative design, where in you either.
Have one process per connection (this idea is stupid)
One thread per connection (this idea is less stupid than the first but still mostly foolish)
Have a non blocking design that allows multiple clients and read/write without blocking execution.
To achieve the first, look at multiprocessing, the second is threading the third is slightly more complicated to get your head around but will yield the best results, the go to library for event driven code in Python is twisted but there are others like
gevent
tulip
tornado
And so so many more that I haven't listed here.
here's an full example of implementing a threaded server. it's fully functional and comes with the benefit of using SSL as well. further, i use threaded event objects to signal another class object after storing my received data in a database.
please note, _sni and _cams_db are additional modules purely of my own. if you want to see the _sni module (provides SNI support for pyOpenSSL), let me know.
what follows this, is a snippet from camsbot.py, there's a whole lot more that far exceeds the scope of this question. what i've built is a centralized message relay system. it listens to tcp/2345 and accepts SSL connections. each connection passes messages into the system. short lived connections will connect, pass message, and disconnect. long lived connections will pass numerous messages after connecting. messages are stored in a database and a threading.Event() object (attached to the DB class) is set to tell the bot to poll the database for new messages and relay them.
the below example shows
how to set up a threaded tcp server
how to pass information from the listener to the accept handler such as config data and etc
in addition, this example also shows
how to employ an SSL socket
how to do some basic certificate validations
how to cleanly wrap and unwrap SSL from a tcp socket
how to use poll() on the socket instead of select()
db.pending is a threading.Event() object in _cams_db.py
in the main process we start another thread that waits on the pending object with db.pending.wait(). this makes that thread wait until another thread does db.pending.set(). once it is set, our waiting thread immediately wakes up and continues to work. when our waiting thread is done, it calls db.pending.clear() and goes back to the beginning of the loop and starts waiting again with db.pending.wait()
while True:
db.pending.wait()
# after waking up, do code. for example, we wait for incoming messages to
# be stored in the database. the threaded server will call db.pending.set()
# which will wake us up. we'll poll the DB for new messages, relay them, clear
# our event flag and go back to waiting.
# ...
db.pending.clear()
snippet from camsbot.py:
import sys, os, sys, time, datetime, threading, select, logging, logging.handlers
import configparser, traceback, re, socket, hashlib
# local .py
sys.path.append('/var/vse/python')
import _util, _webby, _sni, _cams_db, _cams_threaded_server, _cams_bot
# ...
def start_courier(config):
# default values
host = '::'
port = 2345
configp = config['configp']
host = configp.get('main', 'relay msp hostport')
# require ipv6 addresses be specified in [xx:xx:xx] notation, therefore
# it is safe to look for :nnnn at the end
if ':' in host and not host.endswith(']'):
port = host.split(':')[-1]
try:
port = int(port, 10)
except:
port = 2345
host = host.split(':')[:-1][0]
server = _cams_threaded_server.ThreadedTCPServer((host, port), _cams_threaded_server.ThreadedTCPRequestHandler, config)
t = threading.Thread(target=server.serve_forever, name='courier')
t.start()
_cams_threaded_server.py:
import socket, socketserver, select, datetime, time, threading
import sys, struct
from OpenSSL.SSL import SSLv23_METHOD, SSLv3_METHOD, TLSv1_METHOD, OP_NO_SSLv2
from OpenSSL.SSL import VERIFY_NONE, VERIFY_PEER, VERIFY_FAIL_IF_NO_PEER_CERT, Context, Connection
from OpenSSL.SSL import FILETYPE_PEM
from OpenSSL.SSL import WantWriteError, WantReadError, WantX509LookupError, ZeroReturnError, SysCallError
from OpenSSL.crypto import load_certificate
from OpenSSL import SSL
# see note at beginning of answer
import _sni, _cams_db
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_address, HandlerClass, config):
socketserver.BaseServer.__init__(self, server_address, HandlerClass)
self.address_family = socket.AF_INET6
self.connected = []
self.logger = config['logger']
self.config = config
self.socket = socket.socket(self.address_family, self.socket_type)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sc = Context(TLSv1_METHOD)
sc.set_verify(VERIFY_PEER|VERIFY_FAIL_IF_NO_PEER_CERT, _sni.verify_cb)
sc.set_tlsext_servername_callback(_sni.pick_certificate)
self.sc = sc
self.server_bind()
self.server_activate()
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
config = self.server.config
logger = self.server.logger
connected = self.server.connected
sc = self.server.sc
try:
self.peer_hostname = socket.gethostbyaddr(socket.gethostbyname(self.request.getpeername()[0]))[0]
except:
self.peer_hostname = '!'+self.request.getpeername()[0]
logger.info('peer: {}'.format(self.peer_hostname))
ssl_s = Connection(sc, self.request)
ssl_s.set_accept_state()
try:
ssl_s.do_handshake()
except:
t,v,tb = sys.exc_info()
logger.warn('handshake failed {}'.format(v))
ssl_s.setblocking(True)
self.ssl_s = ssl_s
try:
peercert = ssl_s.get_peer_certificate()
except:
peercert = False
t,v,tb = sys.exc_info()
logger.warn('SSL get peer cert failed: {}'.format(v))
if not peercert:
logger.warn('No peer certificate')
else:
acl = config['configp']['main'].get('client cn acl', '').split(' ')
cert_subject = peercert.get_subject().CN
logger.info('Looking for {} in acl: {}'.format(cert_subject,acl))
if cert_subject in acl:
logger.info('{} is permitted'.format(cert_subject))
else:
logger.warn('''client CN not approved''')
# it's ok to block here, every socket has its own thread
ssl_s.setblocking(True)
self.db = config['db']
msgcount = 0
p = select.poll()
# don't want writable, just readable
p.register(self.request, select.POLLIN|select.POLLPRI|select.POLLERR|select.POLLHUP|select.POLLNVAL)
peername = ssl_s.getpeername()
x = peername[0]
if x.startswith('::ffff:'):
x = x[7:]
peer_ip = x
try:
host = socket.gethostbyaddr(x)[0]
except:
host = peer_ip
logger.info('{}/{}:{} connected'.format(host, peer_ip, peername[1]))
connected.append( [host, peername[1]] )
if peercert:
threading.current_thread().setName('{}/port={}/CN={}'.format(host, peername[1], peercert.get_subject().CN))
else:
threading.current_thread().setName('{}/port={}'.format(host, peername[1]))
sockclosed = False
while not sockclosed:
keepreading = True
#logger.debug('starting 30 second timeout for poll')
pe = p.poll(30.0)
if not pe:
# empty list means poll timeout
# for SSL sockets it means WTF. we get an EAGAIN like return even if the socket is blocking
continue
logger.debug('poll indicates: {}'.format(pe))
#define SSL_NOTHING 1
#define SSL_WRITING 2
#define SSL_READING 3
#define SSL_X509_LOOKUP 4
while keepreading and not sockclosed:
data,sockclosed,keepreading = self._read_ssl_data(2, head=True)
if sockclosed or not keepreading:
time.sleep(5)
continue
plen = struct.unpack('H', data)[0]
data,sockclosed,keepreading = self._read_ssl_data(plen)
if sockclosed or not keepreading:
time.sleep(5)
continue
# send thank you, ignore any errors since we appear to have gotten
# the message
try:
self.ssl_s.sendall(b'ty')
except:
pass
# extract the timestamp
message_ts = data[0:8]
msgtype = chr(data[8])
message = data[9:].decode()
message_ts = struct.unpack('d', message_ts)[0]
message_ts = datetime.datetime.utcfromtimestamp(message_ts).replace(tzinfo=datetime.timezone.utc)
self.db.enqueue(config['group'], peer_ip, msgtype, message, message_ts)
self.db.pending.set()
# we're recommended to use the return socket object for any future operations rather than the original
try:
s = ssl_s.unwrap()
s.close()
except:
pass
connected.remove( [host, peername[1]] )
t_name = threading.current_thread().getName()
logger.debug('disconnect: {}'.format(t_name))
def _read_ssl_data(self, wantsize=16384, head=False):
_w = ['WANT_NOTHING','WANT_READ','WANT_WRITE','WANT_X509_LOOKUP']
logger = self.server.logger
data = b''
sockclosed = False
keepreading = True
while len(data) < wantsize and keepreading and not sockclosed:
rlen = wantsize - len(data)
try:
w,wr = self.ssl_s.want(),self.ssl_s.want_read()
#logger.debug(' want({}) want_read({})'.format(_w[w],wr))
x = self.ssl_s.recv(rlen)
#logger.debug(' recv(): {}'.format(x))
if not ( x or len(x) ):
raise ZeroReturnError
data += x
if not (len(x) == len(data) == wantsize):
logger.info(' read={}, len(data)={}, plen={}'.format(len(x),len(data),wantsize))
except WantReadError:
# poll(), when ready, read more
keepreading = False
logger.info(' got WantReadError')
continue
except WantWriteError:
# poll(), when ready, write more
keepreading = False
logger.info(' got WantWriteError')
continue
except ZeroReturnError:
# socket got closed, a '0' bytes read also means the same thing
keepreading = False
sockclosed = True
logger.info(' ZRE, socket closed normally')
continue
except SysCallError:
keepreading = False
sockclosed = True
t,v,tb = sys.exc_info()
if v.args[0] == -1: # normal EOF
logger.info(' EOF found, keepreading=False')
else:
logger.info('{} terminated session abruptly while reading plen'.format(self.peer_hostname))
logger.info('t: {}'.format(t))
logger.info('v: {}'.format(v))
continue
except:
t,v,tb = sys.exc_info()
logger.warning(' fucked? {}'.format(v))
raise
if not head and not len(data) == wantsize:
logger.warn(' short read {} of {}'.format(len(data), wantsize))
return data,sockclosed,keepreading
let's start with a bare bones threaded tcp server.
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, server_address, HandlerClass):
socketserver.BaseServer.__init__(self, server_address, HandlerClass)
self.address_family = socket.AF_INET
self.socket = socket.socket(self.address_family, self.socket_type)
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
# self.request is your accepted socket, do all your .read() and .wirte() on it
s = self.request
request = s.read(1024)
# decide locked or unlocked. this example arbitrarily writes back 'locked'
s.write('locked')
# we're done, close the socket and exit with a default return of None
s.close()
ok, start your threaded server with this in your main() function:
server = threading.ThreadedTCPServer(('127.0.0.1', 1234), ThreadedTCPRequestHandler)
t = threading.Thread(target=server.serve_forever, name='optional_name')
t.start()
now you can let the threading module handle the semantics of concurrency and not worry about it.
You might want to take a look at 0MQ and concurrent.futures. 0MQ has a Tornado event loop in the library and it reduces the complexity of socket programming. concurrent.futures is a high level interface over threading or multiprocessing.
You can see different concurrent server approaches at
https://bitbucket.org/arco_group/upper/src
These will help you to choose the better way for you.
Cheers
I'm trying to create a small program that will log information output from a device via TCP
Basically this just streams data out, that i want to capture, and dump into a database for dealing with later
but the device reboots so i need to be able to reconnect when the socket closes with out any human interference
so this is what i have so far
import socket, time, logging, sys, smtplib # Import socket module
logging.basicConfig(filename='Tcplogger.log',level=logging.DEBUG,format='%(asctime)s : %(levelname)s : %(message)s')
logging.info('|--------------------------------------|')
logging.info('|--------------- TCP Logger Starting---|')
logging.info('|--------------------------------------|')
host = '127.0.0.01' # host or Ip address
port = 12345 # output port
retrytime = 1 # reconnect time
reconnectattemps = 10 # Number of time to try and reconnect
class TPCLogger:
def __init__(self):
logging.debug('****Trying connection****')
print('****Trying connection****')
self.initConnection()
def initConnection(self):
s = socket.socket()
try:
s.connect((host, port))
logging.debug('****Connected****')
except IOError as e:
while 1:
reconnectcount = 0;
logging.error(format(e.errno)+' : '+format(e.strerror))
while 1:
reconnectcount = reconnectcount + 1
logging.error('Retrying connection to Mitel attempt : '+str(reconnectcount))
try:
s.connect((host, port))
connected = True
logging.debug('****Connected****')
except IOError as e:
connected = False
logging.error(format(e.errno)+' : '+format(e.strerror))
if reconnectcount == reconnectattemps:
logging.error('******####### Max Reconnect attempts reached logger will Terminate ######******')
sys.exit("could Not connect")
time.sleep(retrytime)
if connected == True:
break
break
while 1:
s.recv(1034)
LOGGER= TCPLogger()
Which all works fine on start up if a try to connect and its not there it will retry the amount of times set by reconnectattemps
but he is my issue
while 1:
s.recv(1034)
when this fails i want to try to reconnect
i could of course type out or just copy my connection part again but what i want to be able todo is call a function that will handle the connection and retry and hand me back the connection object
for example like this
class tcpclient
#set some var
host, port etc....
def initconnection:
connect to socket and retry if needed
RETURN SOCKET
def dealwithdata:
initconnection()
while 1:
try:
s.recv
do stuff here copy to db
except:
log error
initconnection()
I think this is possible but im really not geting how the class/method system works in python so i think im missing something here
FYI just in case you didn't notice iv very new to python. any other comments on what i already have are welcome too :)
Thanks
Aj
Recommendation
For this use-case I would recommend something higher-level than sockets. Why? Controlling all these exceptions and errors for yourself can be irritating when you just want to retrieve or send data and maintain connection.
Of course you can achieve what you want with your plain solution, but you mess with code a bit more, methinks. Anyway it'll look similarly to class amustafa wrote, with handling socket errors to close/reconnect method, etc.
Example
I made some example for easier solution using asyncore module:
import asyncore
import socket
from time import sleep
class Client(asyncore.dispatcher_with_send):
def __init__(self, host, port, tries_max=5, tries_delay=2):
asyncore.dispatcher.__init__(self)
self.host, self.port = host, port
self.tries_max = tries_max
self.tries_done = 0
self.tries_delay = tries_delay
self.end = False # Flag that indicates whether socket should reconnect or quit.
self.out_buffer = '' # Buffer for sending.
self.reconnect() # Initial connection.
def reconnect(self):
if self.tries_done == self.tries_max:
self.end = True
return
print 'Trying connecting in {} sec...'.format(self.tries_delay)
sleep(self.tries_delay)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
try:
self.connect((self.host, self.port))
except socket.error:
pass
if not self.connected:
self.tries_done += 1
print 'Could not connect for {} time(s).'.format(self.tries_done)
def handle_connect(self):
self.tries_done = 0
print 'We connected and can get the stuff done!'
def handle_read(self):
data = self.recv(1024)
if not data:
return
# Check for terminator. Can be any action instead of this clause.
if 'END' in data:
self.end = True # Everything went good. Shutdown.
else:
print data # Store to DB or other thing.
def handle_close(self):
print 'Connection closed.'
self.close()
if not self.end:
self.reconnect()
Client('localhost', 6666)
asyncore.loop(timeout=1)
reconnnect() method is somehow core of your case - it's called when connection is needed to be made: when class initializes or connection brokes.
handle_read() operates any recieved data, here you log it or something.
You can even send data using buffer (self.out_buffer += 'message'), which will remain untouched after reconnection, so class will resume sending when connected again.
Setting self.end to True will inform class to quit when possible.
asyncore takes care of exceptions and calls handle_close() when such events occur, which is convenient way of dealing with connection failures.
You should look at the python documentation to understand how classes and methods work. The biggest difference between python methods and methods in most other languages is the addition of the "self" tag. The self represents the instance that a method is called against and is automatically fed in by the python system. So:
class TCPClient():
def __init__(self, host, port, retryAttempts=10 ):
#this is the constructor that takes in host and port. retryAttempts is given
# a default value but can also be fed in.
self.host = host
self.port = port
self.retryAttempts = retryAttempts
self.socket = None
def connect(self, attempt=0):
if attempts<self.retryAttempts:
#put connecting code here
if connectionFailed:
self.connect(attempt+1)
def diconnectSocket(self):
#perform all breakdown operations
...
self.socket = None
def sendDataToDB(self, data):
#send data to db
def readData(self):
#read data here
while True:
if self.socket is None:
self.connect()
...
Just make sure you properly disconnect the socket and set it to None.
I have a problem trying to learn about sockets for network communication. I have made a simple thread that listens for connections and creates processes for connecting clients, my problem though is that I can't get the thread to join properly as I haven't found a way to cancel the socket.accept()-call when I want to quit the program.
My code looks like this;
class ServerThread( threading.Thread ):
def __init__(self, queue, host, port):
threading.Thread.__init__(self)
self.queue = queue
self.running = True
self.hostname = host
self.port = port
def run(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.hostname, self.port))
self.socket.listen(1)
while self.running:
try:
conn, address = self.socket.accept()
process = Process(target=server_slave, args=(conn, address, self.queue))
process.daemon = True
process.start()
except socket.timeout:
pass
def stop(self):
self.running = False
self.socket.close()
I have managed to get the program to close by setting self.setDaemon(True) and just exiting the main program, handing everything to the great garbage collector - but that seems like a bad solution. I've also tried setting a timeout for the socket but that results in getting [Errno 35] Resource temporarily unavailable (regardless of the actual timeout, even when I set it to years...).
What am I doing wrong? Have I designed the thread in a dumb way or have I missed something about accepting connections?
One way to get the thread to close seems to be to make a connection to the socket, thus continuing the thread to completion.
def stop(self):
self.running = False
socket.socket(socket.AF_INET,
socket.SOCK_STREAM).connect( (self.hostname, self.port))
self.socket.close()
This works, but it still feels like it might not be optimal...
In most cases you will open a new thread or process once a connection is accepted. To close the connection, break the while loop. Garbage collection will remove the thread or process but join will ensure none get left behind.
Persistent sockets close when the user closes them or they timeout. Non-persistent, like static webpages will close after they've sent the information.
Here's a good example of a persistent socket server in Python. It uses multiprocessing which means it can run across multiple cores for CPU-bound tasks. More commonly known as multithreading.
import socket
import multiprocessing
def run():
host = '000.000.000.000'
port = 1212
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', port))
sock.listen(3)
while True:
p = multiprocessing.Process(target=worker, args=sock.accept()).start()
def worker(conn, addr):
while True:
if data == '':
#remote connection closed
break
if len(dataList) > 2:
# do stuff
print 'This code is untested'
run()
A dirty solution which allows to exit your program is to use os._exit(0).
def stop(self):
self.socket.close()
os._exit(0)
note that sys.exit doesn't work/blocks as it tries to exit cleanly/release resources. But os._exit is the most low level way and it works, when nothing else does.
The operating system itself will release the resources (on any modern system) like when doing exit in a C program.
The best way to do this is to have a single listening thread that has nothing to do with your connection threads and give it a reasonable length timeout. On timeout, check if this thread should shutdown and if not, loop again and go back to listening.
def tcp_listen_handle(self, port=23, connects=5, timeout=2):
"""This is running in its own thread."""
sock = socket.socket()
sock.settimeout(timeout)
sock.bind(('', port))
sock.listen(connects) # We accept more than one connection.
while self.keep_running_the_listening_thread():
connection = None
addr = None
try:
connection, addr = sock.accept()
print("Socket Connected: %s" % str(addr))
# makes a thread deals with that stuff. We only do listening.
self.handle_tcp_connection_in_another_thread(connection, addr)
except socket.timeout:
pass
except OSError:
# Some other error.
print("Socket was killed: %s" % str(addr))
if connection is not None:
connection.close()
sock.close()
The only thing this does is listen, timeout, checks if it should die during the timeout, and goes back to listening. The general rule of thumb is that threads should check whether they should die and try to do that themselves as fast as they can. And if you don't want to take the 2 second hit for timeout wait before the thread unblocks and checks. You can connect to it yourself.
Partially tested solution
Put self.socket.settimeout(0.1) right before while
Put conn.settimeout(None) right after accept