I learned how to add error handling. But I am little bit confused about when we need to add it to my code.
See this following example.
Do I need error handling for every line of calling the socket?
Server.py
import socket
sockfd = socket.socket()
try:
sockfd.bind(("127.0.0.1", 20001))
except socket.error as emsg:
print("Socket bind error: ", emsg)
sys.exit(1)
print("I_am", socket.gethostname(), "and_I_am_listening_...")
sockfd.listen(5)
new, who = sockfd.accept() # Return the TCP connection
print("A_connection_with", who, "has_been_established")
try:
message = new.recv(50)
except socket.error as err:
print("Recv error: ", err)
if message:
print("\'"+message.decode("ascii")+"\'", "is received from", who)
else:
print("Connection is broken")
new.close()
sockfd.close()
One commonly used approach to error handling is to add it when there is a possibility for something unexpected to happen, i.e. something that you did not expect when writing the code. This means that if you have e.g. a simple function whose different possible error states you know, that is when you do not need to add try-statements since you can handle the errors with e.g. if-else statements. This also usually reduces memory consumption, since try-block has a bigger overhead. But in case there are things that you do not know or have no control over, e.g. external program returns or API calls, that is when you should add error handling.
Related
I have a Python program that connects to a server to send it some commands, but occasionally I get this error:
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
In previous situations like this, I would use something like this:
try:
Do something
except KeyError:
do something else
Could I do the same thing in this same situation? I.e.,
try:
Do something
except TimeoutError:
Do something again
And if so, what would I do after the except TimeoutError? Would I just do the same command again?
Could I do the same thing in this same situtation
Yes! You can use try/except for any exception, and TimeoutError is nothing special.
and if so what would I do after the except TimeoutError? would I just do the same command again?
If you only want to retry once, and let a second timeout count as a real error, yes:
try:
do something
except TimeoutError:
do something
(If "do something" is more than a simple statement, you probably want to factor out the code so you don't repeat yourself.)
However, if you want to retry multiple times, you probably want a loop:
for _ in range(max_retries):
try:
do something
break
except TimeoutError:
pass
You may want to add an else clause to the for to distinguish the two cases (succeeded, and did a break, vs. never succeeded and just ran out of tries).
Since the idea here is usually to deal with possibly-transient errors, there are additional things you might want to add, such as:
Re-raising the error, or raising a different one, after max_retries.
Logging at progressively higher levels (e.g., a debug message for a single failure, but a warning for `max_retries).
Retrying with exponential backoff (wait 1 second, then 2, 4, 8, …).
Pushing the URL to the end of the work queue instead of retrying immediately. You can use (URL, retry_count) pairs if you also want max_retries logic, (URL, timestamp) pairs if you also want exponential backoff, or both if you want both. (Of course this only works if you don't care about the order of responses, or can reorder them at the end.)
Different rules for different exceptions (e.g., 500, 502, 503, and 504 errors can all be caused by overload on a server or proxy, but the best retry logic may be different—and the best heuristics for 2018 may be different from 2010 or 2025).
For complicated combinations, a retry decorator, like the one linked in jterrace's helpful answer, is a great way to wrap up the behavior.
You can catch the TimeoutError like you mentioned:
import socket
import sys
try:
dosomething()
except socket.TimeoutError:
print >> sys.stderr, 'Retrying after TimeoutError'
dosomething()
You could also use the retry decorator pattern on a function:
#retry(socket.TimeoutError)
def dosomething():
# code that causes a TimeoutError
...
def f():
pass #insert code here
To repeat once after the error:
try:
f()
except TimeoutError:
f()
Or to loop until success:
while True:
try:
f()
break
except TimeoutError:
pass
Or with a limited number:
attempts = 3
while attempts:
try:
f()
break
except TimeoutError:
attempts -= 1
Please run each example. They are ready!
Example 1
import sys
try:
incorrect.syntaxThatIJustMadeUP()
except:
print((sys.exc_info()[0])) #Now you know what to except and CATCH
else:
print("You will never see this message")
Example 2
import sys
try:
incorrect.syntaxThatIJustMadeUP()
except NameError:
print("There is a problem with your SYNTAX, Dude!")
except:
print((sys.exc_info()[0])) # In case another uncontrollable network problem occurs, User rages-Snaps IJ45
else:
print("You will never see this message unless TRY succeeds")
print("Why not put this try in a loop?")
Sometimes I do not get any data in via the serial interface and I want to catch this case with an exception in the following way:
ser = serial.Serial(3)
ser.baudrate = 115200
timeout = 1
while (some condidion)
try:
dump = ser.read(40)
except ser1.SerialTimeoutException:
print('Data could not be read')
ser.close()
However, when I run this program, the exception is never caught when no
data is in the buffer and I am stuck in an endless loop. Anyone an idea
what I am doing wrong here?
I didn't even know about that exception. After having a look at the API documentation, you'll see that this exception only applies to write().
If you read(), you'll just have a shortened or even empty output.
And this applies only if you have opened the connection with a timeout. Something like serial.Serial(3, timeout=.1).
I am new to Python and currently have to write a python socket to be run as a script that communicates with a device over TCP/IP (a weather station).
The device acts as the Server Side (listening over IP:PORT, accepting connection, receiving request, transferring data).
I only need to send one message, receive the answer and then peacefully and nicely shutdown and close the socket.
try:
comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(1)
try:
comSocket.connect((''))
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(2)
comSocket.send('\r')
comSocket.recv(128)
comSocket.send('\r')
comSocket.recv(128)
comSocket.send('\r\r')
comSocket.recv(128)
comSocket.send('1I\r\r3I\r\r4I\r\r13I\r\r5I\r\r8I\r\r7I\r\r9I\r\r')
rawData = comSocket.recv(512)
comSocket.shutdown(1)
comSocket.close()
The problem I'm having is:
The communication channel is unreliable, the device is slow. So, sometimes the device response with message of length 0 (just an ACK), the my code will freeze and wait for response forever.
This piece of code contains the portion that involves SOCKET, the whole code will be run under CRON so freezing is not a desirable behavior.
My question is:
What would be the best way in Python to handle that behavior, so that the code doesn't freeze and wait forever but will attempt to move on to the next send (or such).
You can try a timeout approach, like Russel code or you can use a non-blocking socket, as shown in the code below. It will never block at socket.recv and you can use it inside a loop to retry as many times you want. This way your program will not hang at timeout. This way, you can test if data is available and if not, you can do other things and try again later.
socket.setblocking(0)
while (retry_condition):
try:
data = socket.recv(512)
except socket.error:
'''no data yet..'''
I'd recommend eventlet and green threads for this.
Twisted is a good library but a little steep learning curve for such a simple use case.
Check out some examples here.
Try, before receiving, putting a timeout on the socket:
comSocket.settimeout(5.0)
try:
rawData = comSocket.recv(512)
except socket.timeout:
print "No response from server"
Some of you might remember a question very similar to this, as I seeked your help writin the original util in C (using libssh2 and openssl).
I'm now trying to port it to python and got stuck at an unexpected place. Ported about 80% of the core and functionality in 30 minutes, and then spend 10hours+ and still haven't finished that ONE function, so I'm here again to ask for you help one more time :)
The whole source (~130 lines, should be easily readable, not complex) is available here: http://pastebin.com/Udm6Ehu3
The connecting, switching on SSL, handshaking, authentication and even sending (encrypted) commands works fine (I can see from my routers log that I log in with proper user and password).
The problem is with ftp_read in the tunnel scenario (else from self.proxy is None). One attempt was this:
def ftp_read(self, trim=False):
if self.proxy is None:
temp = self.s.read(READBUFF)
else:
while True:
try:
temp = self.sock.bio_read(READBUFF)
except Exception, e:
print type(e)
if type(e) == SSL.WantReadError:
try:
self.chan.send(self.sock.bio_read(10240))
except Exception, e:
print type(e)
self.chan.send(self.sock.bio_read(10240))
elif type(e) == SSL.WantWriteError:
self.chan.send(self.sock.bio_read(10240))
But I end up stuck at either having a blocked waiting for bio read (or channel read in the ftp_write function), or exception OpenSSL.SSL.WantReadError which, ironicly, is what I'm trying to handle.
If I comment out the ftp_read calls, the proxy scenario works fine (logging in, sending commands no problem), as mentioned. So out of read/write unencrypted, read/write encrypted I'm just missing the read tunnel encrypted.
I've spend 12hours+ now, and feel like I'm getting nowhere, so any thoughts are highly appreciated.
EDIT: I'm not asking someone to write the function for me, so if you know a thing or two about SSL (especially BIOs), and you can see an obvious flaw in my interaction between tunnel and BIO, that'll suffice as a answer :) Like: maybe the ftp_write returns more data than those 10240 bytes requested (or just sends two texts ("blabla\n", "command done.\n")) so it isn't properly flushed. Which might be true, but apparently I can't rely on .want_write()/.want_read() from pyOpenSSL to report anything but 0 bytes available.
Okay, so I think I manged to sort it out.
sarnold, you'll like this updated version:
def ftp_read(self, trim=False):
if self.proxy is None:
temp = self.s.read(READBUFF)
else:
temp = ""
while True:
try:
temp += self.sock.recv(READBUFF)
break
except Exception, e:
if type(e) == SSL.WantReadError:
self.ssl_wants_read()
elif type(e) == SSL.WantWriteError:
self.ssl_wants_write()
where ssl_wants_* is:
def ssl_wants_read(self):
try:
self.chan.send(self.sock.bio_read(10240))
except Exception, e:
chan_output = None
chan_output = self.chan.recv(10240)
self.sock.bio_write(chan_output)
def ssl_wants_write(self):
self.chan.send(self.sock.bio_read(10240))
Thanks for the input, sarnold. It made things a bit clearer and easier to work with. However, my issue seemed to be one missed error handling (broke out of SSL.WantReadError exception too soon).
I've written a simple multi-threaded game server in python that creates a new thread for each client connection. I'm finding that every now and then, the server will crash because of a broken-pipe/SIGPIPE error. I'm pretty sure it is happening when the program tries to send a response back to a client that is no longer present.
What is a good way to deal with this? My preferred resolution would simply close the server-side connection to the client and move on, rather than exit the entire program.
PS: This question/answer deals with the problem in a generic way; how specifically should I solve it?
Assuming that you are using the standard socket module, you should be catching the socket.error: (32, 'Broken pipe') exception (not IOError as others have suggested). This will be raised in the case that you've described, i.e. sending/writing to a socket for which the remote side has disconnected.
import socket, errno, time
# setup socket to listen for incoming connections
s = socket.socket()
s.bind(('localhost', 1234))
s.listen(1)
remote, address = s.accept()
print "Got connection from: ", address
while 1:
try:
remote.send("message to peer\n")
time.sleep(1)
except socket.error, e:
if isinstance(e.args, tuple):
print "errno is %d" % e[0]
if e[0] == errno.EPIPE:
# remote peer disconnected
print "Detected remote disconnect"
else:
# determine and handle different error
pass
else:
print "socket error ", e
remote.close()
break
except IOError, e:
# Hmmm, Can IOError actually be raised by the socket module?
print "Got IOError: ", e
break
Note that this exception will not always be raised on the first write to a closed socket - more usually the second write (unless the number of bytes written in the first write is larger than the socket's buffer size). You need to keep this in mind in case your application thinks that the remote end received the data from the first write when it may have already disconnected.
You can reduce the incidence (but not entirely eliminate) of this by using select.select() (or poll). Check for data ready to read from the peer before attempting a write. If select reports that there is data available to read from the peer socket, read it using socket.recv(). If this returns an empty string, the remote peer has closed the connection. Because there is still a race condition here, you'll still need to catch and handle the exception.
Twisted is great for this sort of thing, however, it sounds like you've already written a fair bit of code.
Read up on the try: statement.
try:
# do something
except socket.error, e:
# A socket error
except IOError, e:
if e.errno == errno.EPIPE:
# EPIPE error
else:
# Other error
SIGPIPE (although I think maybe you mean EPIPE?) occurs on sockets when you shut down a socket and then send data to it. The simple solution is not to shut the socket down before trying to send it data. This can also happen on pipes, but it doesn't sound like that's what you're experiencing, since it's a network server.
You can also just apply the band-aid of catching the exception in some top-level handler in each thread.
Of course, if you used Twisted rather than spawning a new thread for each client connection, you probably wouldn't have this problem. It's really hard (maybe impossible, depending on your application) to get the ordering of close and write operations correct if multiple threads are dealing with the same I/O channel.
I face with the same question. But I submit the same code the next time, it just works.
The first time it broke:
$ packet_write_wait: Connection to 10.. port 22: Broken pipe
The second time it works:
[1] Done nohup python -u add_asc_dec.py > add2.log 2>&1
I guess the reason may be about the current server environment.
My answer is very close to S.Lott's, except I'd be even more particular:
try:
# do something
except IOError, e:
# ooops, check the attributes of e to see precisely what happened.
if e.errno != 23:
# I don't know how to handle this
raise
where "23" is the error number you get from EPIPE. This way you won't attempt to handle a permissions error or anything else you're not equipped for.