I'm having an issue with Python's socket module that I haven't been able to find anywhere else.
I'm building a simple TCP chat client, and while it successfully connects to the server initially, the script hangs endlessly on sock.recv() despite the fact that I explicitly set a timeout length.
I've tried using different timeout values and including setblocking(False) but no matter what I do it keeps acting like the socket is in blocking mode.
Here are the relevant parts of my code:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(1)
def listen_to_server():
global connected
while connected:
ready_to_read, ready_to_write, in_error = select.select([sock], [], [])
if ready_to_read:
try:
data = sock.recv(1024)
except socket.timeout:
print('TIMEOUT')
if not data:
append_to_log('Disconnected from server.\n')
connected = False
else:
append_to_log(str(data))
Any suggestions would be helpful, I'm at a total loss here.
You've mixed two things the socket timeout and the select.
When you set socket timeout then you are telling to that socket: if I try do some operation (e.g. recv()) and it won't be finished until my limit then raise timeout exception.
The select takes file descriptors (on Windows only sockets) and start checking if the rlist (the first parameter) contains any socket ready to read (aka some data have arrived). If any data arrived then the program continues.
Now your code do this:
Set timeout for socket operations
Select start waiting for data (if you don't send them then they never arrives)
and that's it. You stuck at the select.
You should just call the recv() without select. Than your timeout should be applied.
If you need manage multiple sockets at once than you have to use select and set the 4th parameter timeout.
Related
I'm using the line "conn, addr = httpSocket.accept()", but I don't want to wait for it every iteration of my loop because there won't always be someone trying to connect. Is there a way to check if anyone is trying to connect, and move on if there isn't?
I have looked at using asyncio (I can't use threads because this is micropython on an esp8266, and threading is not supported) but my line is not awaitable.
with open('page.html', 'r') as file:
html = file.read()
while True:
conn, addr = httpSocket.accept()
print('Got a connection from %s' % str(addr))
conn.send('HTTP/1.1 200 OK\n')
conn.send('Content-Type: text/html\n')
conn.sendall(html)
conn.close()
If threads isn't an option you can always use the select module.
With select you basically split your sockets into 3 categories:
Sockets that you want to read data from them (including new connections).
Sockets that you want to send them data.
Exceptional sockets ( usually for error checking).
And with each iteration select returns to you lists of sockets by these categories, so you know how to handle each one instead of waiting for a new connection each time.
You can see an example here:
https://steelkiwi.com/blog/working-tcp-sockets/
I'm new to socket programming in python. Here is an example of opening a TCP socket in a Mininet host and sending a photo from one host to another. In fact I changed the code that I had used to send a simple message to another host (writing the received data to a text file) in order to meet my requirements. Although when I implement this revised code, there is no error and it seems to transfer correctly, I am not sure whether this is a correct way to do this transmission or not. Since I'm running both hosts on the same machine, I thought it may have an influence on the result. I wanted to ask you to check whether this is a correct way to transfer or I should add or remove something.
mininetSocketTest.py
#!/usr/bin/python
from mininet.topo import Topo, SingleSwitchTopo
from mininet.net import Mininet
from mininet.log import lg, info
from mininet.cli import CLI
def main():
lg.setLogLevel('info')
net = Mininet(SingleSwitchTopo(k=2))
net.start()
h1 = net.get('h1')
p1 = h1.popen('python myClient2.py')
h2 = net.get('h2')
h2.cmd('python myServer2.py')
CLI( net )
#p1.terminate()
net.stop()
if __name__ == '__main__':
main()
myServer2.py
import socket
import sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('10.0.0.1', 12345))
buf = 1024
f = open("2.jpg",'wb')
s.listen(1)
conn , addr = s.accept()
while 1:
data = conn.recv(buf)
print(data[:10])
#print "PACKAGE RECEIVED..."
f.write(data)
if not data: break
#conn.send(data)
conn.close()
s.close()
myClient2.py:
import socket
import sys
f=open ("1.jpg", "rb")
print sys.getsizeof(f)
buf = 1024
data = f.read(buf)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('10.0.0.1',12345))
while (data):
if(s.sendall(data)):
#print "sending ..."
data = f.read(buf)
print(f.tell(), data[:10])
else:
s.close()
s.close()
This loop in client2 is wrong:
while (data):
if(s.send(data)):
print "sending ..."
data = f.read(buf)
As the send
docs say:
Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this topic, consult the Socket Programming HOWTO.
You're not even attempting to do this. So, while it probably works on localhost, on a lightly-loaded machine, with smallish files, it's going to break as soon as you try to use it for real.
As the help says, you need to do something to deliver the rest of the buffer. Since there's probably no good reason you can't just block until it's all sent, the simplest thing to do is to call sendall:
Unlike send(), this method continues to send data from bytes until either all data has been sent or an error occurs. None is returned on success. On error, an exception is raised…
And this brings up the next problem: You're not doing any exception handling anywhere. Maybe that's OK, but usually it isn't. For example, if one of your sockets goes down, but the other one is still up, do you want to abort the whole program and hard-drop your connection, or do you maybe want to finish sending whatever you have first?
You should at least probably use a with clause of a finally, to make sure you close your sockets cleanly, so the other side will get a nice EOF instead of an exception.
Also, your server code just serves a single client and then quits. Is that actually what you wanted? Usually, even if you don't need concurrent clients, you at least want to loop around accepting and servicing them one by one.
Finally, a server almost always wants to do this:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Without this, if you try to run the server again within a few seconds after it finished (a platform-specific number of seconds, which may even depend whether it finished with an exception instead of a clean shutdown), the bind will fail, in the same way as if you tried to bind a socket that's actually in use by another program.
First of all, you should use TCP and not UDP. TCP will ensure that your client/server has received the whole photo properly. UDP is more used for content streaming.
Absolutely not your use case.
def mp_worker(row):
ip = row[0]
ip_address = ip
tcp_port = 2112
buffer_size = 1024
# Read the reset message sent from the sign when a new connection is established
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
print('Connecting to terminal: {0}'.format(ip_address))
s.connect((ip_address, tcp_port))
#Putting a breakpoint on this call in debug makes the script work
s.send(":08a8RV;")
#data = recv_timeout(s)
data = s.recv(buffer_size)
strip = data.split("$", 1)[-1].rstrip()
strip = strip[:-1]
print(strip)
termStat = [ip_address, strip]
terminals.append(termStat)
except Exception as exc:
print("Exception connecting to: " + ip_address)
print(exc)
The above code is the section of the script that is causing the problem. It's a pretty simple function that connects to a socket based on a passed in IP from a DB query and receives a response that indicates the hardware's firmware version.
Now, the issue is that when I run it in debug with a breakpoint on the socket I get the entire expected response from the hardware, but if I don't have a breakpoint in there or I full on Run the script it only responds with part of the expected message. I tried both putting a time.sleep() in after the send to see if it would get the entire response and I tried using the commented out recv_timeout() method in there which uses a non-blocking socket and timeout to try to get an entire response, both with the exact same results.
As another note, this works in a script with everything in one main code block, but I need this part separated into a function so I can use it with the multiprocessing library. I've tried running it on both my local Windows 7 machine and on a Unix server with the same results.
I'll expands and reiterate on what I've put into a comment moment ago. I am still not entirely sure what is behind the different behavior in either scenario (apart from timing guess apparently disproved by an attempt to include sleep.
However, it's somewhat immaterial as stream sockets do not guarantee you get all the requested data at once and in chunks as requested. This is up for an application to deal with. If the server closes the socket after full response was sent, you could replace:
data = s.recv(buffer_size)
with recv() until zero bytes were received, this would be equivalent of getting 0 (EOF) from from the syscall:
data = ''
while True:
received = s.recv(buffer_size)
if len(received) == 0:
break
data += received
If that is not the case, you would have to rely on fixed or known (sent in the beginning) size you want to consider together. Or deal with this on protocol level (look for characters, sequences used to signal message boundaries.
I just recently found out a solution here, and thought I'd post it in case anyone else has issue, I just decided to try and call socket.recv() before calling socket.send() and then calling socket.recv() again afterwards and it seems to have fixed the issue; I couldn't really tell you why it works though.
data = s.recv(buffer_size)
s.send(":08a8RV;")
data = s.recv(buffer_size)
I'm new to socket programming (and somewhat to Python too) and I'm having trouble getting the select timeout to work the way I want to (on the server side). Before clients connect, timeout works just fine. I give it a value of 1 second and the timeout expires in my loop every 1 second.
Once a client connects, however, it doesn't wait 1 second to tell me the timeout expires. It just loops as fast as it can and tells me the timeout expires. Here's a snippet of my code:
while running:
try:
self.timeout_expired = False
inputready, outputready, exceptready = select.select(self.inputs, self.outputs, [], self.timeout)
except select.error, e:
break
except socket.error, e:
break
if not (inputready):
# Timeout expired
print 'Timeout expired'
self.timeout_expired = True
# Additional processing follows here
I'm not sure if this is enough code to see where my problem is, so please let me know if you need to see more. Basically, after a client connects, it at least appears that it ignores the timeout of 1 second and just runs as fast as it can, continuously telling me "Timeout expired". Any idea what I'm missing?
Thanks much!!
Edit: I should clarify..."inputready" represents input from a client connecting or sending data to the server, as well as stdin from the server. The other variables returned from select are only server-side variables, and is what I'm trying to do is detect whether the CLIENT took too long to reply, so I'm only checking if inputready is empty.
It is only a timeout if inputready, outputready, and exceptready are ALL empty. My guess is you have added the client socket to both self.inputs and self.outputs. Since the output socket is usually writable, it will always show up in outputready. Only add the client socket to self.outputs if you are ready to output something.
"When the timeout expires, select() returns three empty lists.
...To use a timeout requires adding the extra argument to the select() call and handling the empty lists after select() returns."
readable, writable, exceptional = select.select(inputs, outputs, inputs,timeout)
if not (readable or writable or exceptional):
print(' timed out, do some other work here', file=sys.stderr)
[https://pymotw.com/3/select/index.html][1]
I'm making server that make a tcp socket and work over port range, with each port it will listen on that port for some time, then continue the rest of the code.
like this::
import socket
sck = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
msg =''
ports = [x for x in xrange(4000)]
while True:
try:
for i in ports:
sck.bind(('',i))
## sck.listen(1)
## make it just for some time and then continue this
## if there a connection do this
conn, addr = sck.accept()
msg = conn.recv(2048)
## do something
##if no connection continue the for loop
conn.close()
except KeyboardInterrupt:
exit()
so how i could make sck.listen(1) work just for some time ??
You can settimeout on the socket to the maximum amount of time you want to wait on it each time (call it again before every listen to the time you want to wait this time around) -- you'll get an exception, socket.timeout, if the timer expires, so be sure to have a try/except socket.timeout: around it to catch that case. (A select.select with a timeout would also work, and has the advantage of being able to wait on multiple sockets and for various conditions, but it's a bit less direct as an answer to your very specific question).
I got many downvotes last time I gave such an answer... presumably by purists who want to make sure nobody, ever, programs in way they disapprove of (e.g. through a very peculiar construct such as yours as opposed to the many normal, usual way of writing servers). Let's see what happens this time around!-)