We Keep Coding

After sending data to the server by threading.Timer(700ms), I received in other side 685ms?

I have been developing an application for measuring the gap between two frames to measure the time on the stopwatch by the camera, so I need to trigger a function after a specific time to send data to the server (Server-TCP). but I have one issue sometimes the server receive request less than the specified time, for example, if I run a thread with Timer 700ms, I find some requests inside the server such as 0.682,0.690,0.692,0.699 even if I calculate the gap in the real-image it also the same.
the normal behavior I expected to be if the function Trigger after 700ms, and I do some process, It must be bigger than 700 ms, this is the normal behavior.
I don't know where the mistake I made, and how that was possible to happen, all requests have a unique name, so it's easy to trace them.
this the main core in which I trigger the method to send requests to a server.
#2nd image
def generate2ndTicketBasedonTime(trackedObject, attributes, object_id,event_obj):
try:
while not event_obj.wait(0.700):
ImageProcessing.getInstance().serviceSnapshot(camera, DataToQeue)
event_obj.set()
pass
#1st image
def snapshot
ImageProcessing.getInstance().serviceSnapshot(camera, DataToQeue)
event_obj = threading.Event()
threading.Thread(target=generate2ndTicketBasedonTime, args=[self, self.attributes, self.car.object_id,event_obj]).start()
# client-side -< send data to the server
def sendRequestToCameraService(self, data):
Request = {'cameraRequest': json.dumps(data)}
self.SendToServer(Request)
def SendToServer(self, data):
try:
if(self.isConnected):
data = json.dumps(data).encode()
dataSize = struct.pack("<i", len(data))
self.currentSocket.sendall(b'\xAA\xBB'+dataSize+data+b'\xCC\xDD')
self.triggerTime = time.time()
except BrokenPipeError:
self.isConnected = False
#server setting
def run(self):
srv_address = "0.0.0.0"
srv_port = 4504
srv_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv_sock.bind((srv_address, int(srv_port)))
srv_sock.listen(5)
while True:
cl_sock, cl_address = srv_sock.accept()
ClientHandler(cl_sock, cl_address, f"Client -{len(self.clients)}")
example for the case which I'm facing
image for gap time between client and server-side
image gap time before trigger the 2nd method in the clock
image for 300ms gap time

I figure out the issue, it was the camera that return the frame before the specific time for a mysterious reason.
1- I tried the code without socket and threads, to avoid complexity so the main problem was from the camera
2- there is a slight difference between the applications when they start up(take time.time()) so now I'm sending the timestamp from the main application to other servers in the request, so the sequences for the process seem good now.

non-blocking i/o using select

I have a sample client-server program that does non-blocking I/O for several sockets not using processes or threads. It uses select. Unfortunately, the server just shows lots of blank lines and that's all. Where is the mistake?
Running on MacOS.
Thanks in advance.
Server:
import socket
import select
sock = socket.socket()
sock.bind(('', 10001))
sock.listen()
conn1, _ = sock.accept()
conn2, _ = sock.accept()
conn1.setblocking(0)
conn2.setblocking(0)
epoll = select.poll()
epoll.register(conn1.fileno(), select.POLLIN | select.POLLOUT)
epoll.register(conn2.fileno(), select.POLLIN | select.POLLOUT)
conn_map = {
conn1.fileno(): conn1,
conn2.fileno(): conn2,
}
while True:
events = epoll.poll(1)
for fileno, event in events:
if event & select.POLLIN:
data = conn_map[fileno].recv(1024)
print(data.decode('utf8'))
elif event & select.POLLOUT:
conn_map[fileno].send('ping'.encode('utf8'))
Client:
import socket
from multiprocessing import Pool
def create_socket_and_send_data(number):
with socket.create_connection(('127.0.0.1', 10001)) as sock:
try:
sock.sendall(f'client {number}\n'.encode('utf8'))
except socket.error as ex:
print('data sending error', ex)
print(f'data for {number} has been sent')
if __name__ == '__main__':
with Pool(processes=2) as pool:
pool.map(create_socket_and_send_data, range(2))

Unfortunately, the server just shows lots of blank lines and that's all.
Actually this is not true.
The server prints at the beginning the lines it got from the clients. After they've send these lines the client close the connection which means that select.POLLIN gets triggered again on the socket and recv returns empty data.
This empty data is the sign that the peer has closed the connection. Once it got this sign the server should close the connection to the client and remove the fileno from the select. Instead your server prints the empty string with a newline and continues to expect new POLLIN events. These will come again and again and will always an empty buffer, thus leading to all the empty lines you see.

select is paradoxically easier to use for input than for output. For input, you receive an event each time new data arrives on a socket, so you always ask for all the sockets and have something to process for every new event.
For output, select will just say that a socket if ready to accept new data. Which is almost always true except if you have just filled a buffer. So you should only poll for an output socket when you have something to write there.
So you should register your sockets with select.POLLIN only. For the write part, you should either directly write to a socket without polling if you can hope that the peer should always be able to receive, or set up a queue with pending output per socket, modify the polling state of a socket with select.POLLIN | select.POLLOUT when there is something in its queue and modify it back with select.POLLIN back when the queue is empty again.

Can't make a Python socket blocking

I'm trying to write a fairly simple client-server Python application using socket and SocketServer. To allow for two-way communication between client and server, the client maintains one connected socket with the server so it can listen for messages in a separate thread, while the main thread creates one-time-use sockets to send messages to the server. I want my "listening" socket to be blocking, as it is running in a separate thread whose only purpose is to wait for data without blocking the main program. Here is the function where I create this socket:
def connect(self, alias, serverIP):
if not alias or not isinstance(alias, str):
print "ERROR: Must specify an alias"
return
self.serverIP = serverIP
self.downConnection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.downConnection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.downConnection.setblocking(1)
self.downConnection.connect((self.serverIP, 11100))
self.downConnection.send("SENDSERVER CONNECT %s" % alias)
Here is the loop where the persistent socket listens for messages from the server (with some debugging code thrown in):
i = 0
while True:
print "LOOP", i,
if self.closed:
break
try:
data = self.downConnection.recv(1024)
except socket.timeout, e:
print "Timeout"
pass
else:
print "Received %d" % len(data)
if data:
self.received(data)
i += 1
I would expect to see "Received ##" messages only when the server sends data, and maybe periodic "Timeout" messages otherwise. Instead, the output grows very rapidly and looks entirely like this:
LOOP 33858 Received 0
LOOP 33859 Received 0
LOOP 33860 Received 0
LOOP 33861 Received 0
LOOP 33862 Received 0
LOOP 33863 Received 0
LOOP 33864 Received 0
LOOP 33865 Received 0
So it seems that self.downConnection.recv() is immediately returning an empty string each time it is called, rather than blocking until it receives substantive data like it's supposed to. This is puzzling, as I'm explicitly setting the socket to be blocking (which I think is also the default setting). Constantly executing this loop instead of the thread spending most of its time waiting for data is wasting a good deal of CPU time. What am I doing wrong in setting up the blocking socket?
Here is the full server code. The Comms class is also the superclass of the client class, to allow for some basic common functionality.
Something does seem to be wrong with the connection from the server's end. The server can receive data from the client, but trying to send data to the client gives a socket.error: [Errno 9] Bad file descriptor exception.

Cant receive data from socket

I'm making a client-server program, and there is problem with client part.
Problem is in infinite receiving data. I've tested this particular class, listed below, in a python interpreter. I've succesfuly(maybe not) connected to google, but then program stoped in function recvData() in data = self.socket.recv(1024)
class client():
def __init__(self, host, port):
self.host = host
self.port = port
self.socket = self.connect()
self.command = commands()
def connect(self):
'''
Connect to a remote host.
'''
try:
import socket
return socket.create_connection((self.host, self.port))
except socket.error:
print(":: Failed to connect to a remote port : ")
def sendCommand(self, comm):
'''
Send command to remote host
Returns server output
'''
comman = comm.encode()
# for case in switch(comman):
# if case(self.command.RETRV_FILES_LIST.encode()):
# self.socket.send(b'1')
# return self.recvData()
# if case():
# print(":: Got wrong command")
if (comman == b'1'):
self.socket.send(b'1')
return self.recvData()
def recvData(self):
'''
Receives all the data
'''
i = 0
total_data = []
while(True):
data = self.socket.recv(1024)
if not data: break
total_data.append(data)
i += 1
if i > 9:
break
return total_data
about commented part :
I thought problem in Case realization, so used just if-then statement. But it's not.

Your problem is that self.socket.recv(1024) only returns an empty string when the socket has been shut down on the server side and all data has been received. The way you coded your client, it has no idea that the full message has been received and waits for more. How you deal with the problem depends very much on the protocol used by the server.
Consider a web server. It sends a line-delimited header including a content-length parameter telling the client exactly how many bytes it should read. The client scans for newlines until the header is complete and then uses that value to do recv(exact_size) (if large, it can read chunks instead) so that the recv won't block when the last byte comes in.
Even then, there a decisions to make. The client knows how large the web page is but may want to send a partial data to the caller so it can start painting the page before all the data is received. Of course, the caller needs to know that is what happens - there is a protocol or set of rules for the API itself.
You need to define how the client knows a message is complete and what exactly it passes back to its caller. A great way to deal with the problem is to let some other protocol such as [zeromq](http://zeromq.org/ do the work for you. A simple python client / server can be implemented with xmlrpc. And there are many other ways.
You said you are implementing a client/server program then you mentioned "connected to google" and telnet... These are all very different things and a single client strategy won't work with all of them.

Retrieve all the response in socket at one time

I am writing a client-sever program based on Python socket.
The client sends a command to the server and the server responds.
But now, some client can broadcast a message to other clients, so the client can receive more than one response at the same time.
data = s.recv(1024)
the line of code above will retrieve only one response from the server.
but if I use a while loop like this
while True:
data = s.recv(1024)
if not data: break
actually, data=s.recv(1024) will block the program when there is no data left.
I don't want to block the program and want to retrieve all the responses available in the connection at one time. Can anyone find a solution? Thank you.

You can use the select module to wait until the socket is readable or until a timeout has elapsed; you can then perform other processing. For example:
while True:
# If data can be received without blocking (timeout=0), read it now
ready = select.select([s], [], [], 0)
if s in ready[0]:
data = s.recv(1024)
# Process data
else:
# No data is available, perform other tasks

You could make the socket (s) non-blocking. This way, it will retrieve all the received responses and when there is none, it will return back. Of course, with non-blocking, you will have to periodically retry.
You could make the socket (s) non-blocking using the setblocking() method:
s.setblocking(0)
The other option is to use another thread to handle the receive part. This way, your main thread can continue doing its main task and act upon the message only if it receives one.

You can use socket.setblocking or socket.settimeout:
import socket
import sys
HOST = 'www.google.com'
PORT = 80
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.setblocking(0)
s.sendall('Hello, world')
try:
data = s.recv(1024)
except:
print 'Oh noes! %s' % sys.exc_info()[0]
s.close()
socket.recv takes two parameters, the second is a set of flags. If you're on a Linux system, you can do man recv for a list of flags you can supply, and their corresponding errors.
Lastly, in general, you can't really know that the other side is done with sending you data (unless you're controlling both sides), even if you're both following a protocol. I believe the right way to go about it is to use timeouts, and quit after sending a reset (how you do this will depend upon what protocol you're using).