I'm trying to prevent my UDP client program from waiting forever in cases of a lost packet from the server. I want to use signals to implement this. My code is:
def handler(signum):
raise TimeoutException("Timeout occured!")
signal.signal(signal.SIGALRM, sig_alarm)
sock.sendto(data,serv)
signal.alarm(5)
try:
received_time, servaddr = sock.recvfrom(size)
except TimeoutException:
print "timeout"
signal.alarm(0)
When running this code in the instance in which the remote server is down, I get the following error:
global name 'TimeoutException' is not defined
I assume the program does not "see" my handler. What may cause it?
There is no TimeoutException exception type in either the Python 2 or Python 3 standard libraries. In Python 3.3 and newer, there is a TimeoutError type which is raised when an operating system call times out in the system (corresponding to errno being set to the ETIMEDOUT value). However, I do not recommend you raise this value yourself, due to its specific intent of being raised in response to timeouts returned by the operating system.
Correct use of signal.signal
In your comment, after defining a custom timeout exception type, you indicate a further error arises: handler() takes exactly 1 argument (2 given).
This occurs because your signal handler does not adhere to the definition of a signal handler function as expected by the signal package (those docs for Python 3, but identical to Python 2):
signal.signal(signum, handler)
Set the handler for signal signalnum to the function handler... The handler is called with two arguments: the signal number and the current stack frame
To remedy this, adjust the definition of your custom signal handler to receive the stack frame as the second argument. You need to accept this in the function call, as the standard library expects, but you don't need to do anything with the value passed in.
def handler(signum, frame):
raise TimeoutException("Timeout occurred!")
Better way? Use socket.settimeout
There is built-in support in the socket library for what you are trying to accomplish: the settimeout method of socket objects. I would recommend using this in your code, rather than attempting to reinvent the wheel; it declares intent while abstracting away the details of the implementation and has a built-in exception type (socket.timeout) you can catch to determine when timeouts occur.
Using this method, you would simply call socket.settimeout(5) on your socket prior to making blocking calls which may timeout. You can also set a default timeout on all sockets using socket.setdefaulttimeout before opening the first socket in the application.
Where did TimeoutException come from originally?
Obviously, your use of this may have originated from anywhere. However, I note there is a TimeoutException in Selenium, which is a widely-used Python library used for automating browsers. It is widely used and examples/Q&A associated with Selenium is common, so I would not be surprised if you had found documentation or examples which led you to believe this was an exception type present in the standard library whereas it is actually a type specific to that library. It does not appear that anything related to Selenium is applicable to your present usecase.
For this error : global name 'TimeoutException' is not defined that means u must declare your custom exception .. the simple way to do so is by adding this to your code :
class TimeoutException(Exception):
pass
one thing to mention is that selenium has an exception named TimeoutException which is not an exception type present in the standard library but specific to the selenium library :
from selenium.common.exceptions import TimeoutException
and finally you will run to an another error ,because your signal handler must recieve the stack frame as the second argument (The handler is called with two arguments: the signal number and the current stack frame) :
def handler(signum, frame):
raise TimeoutException("Timeout occurred!")
please read the following example from the docs minimal example program
Related
According to Python's Logging tutorial, logging.error is to be used to "report suppression of an error without raising an exception.", whereas if I wanted to report an error I'm supposed to just raise an exception and not use logging at all.
What if I want to report an error and then raise an exception?
For example:
try:
os.path.getsize('/nonexistent')
except os.error as error:
logging.error(str(error))
logging.error('something went wrong')
raise SystemExit(1)
This results in the following lines being printed to the standard error:
ERROR:root:[Errno 2] No such file or directory: '/nonexistent'
ERROR:root:something went wrong
It seems sensible. Or:
try:
os.path.getsize('/nonexistent')
except os.error as error:
logging.error(str(error))
raise SomeUserDefinedException('something went wrong')
Is this bad practice?
logging is used to report.
raise is an action which interrupts the normal flow of the program.
Typically, you raise exceptions where there is an actual flow to interrupt, and when you want to report, you can use logging, which will log errors and exceptions by default.
The logging part usually comes as a wrapper around a module, class, or middleware in your program.
logging within the code should only be used for DEBUG purposes, or for INFO purpose if you wanted to keep track of some information in your server logs for example. Not for ERROR.
I do not believe that is a bad practice. Exceptions and logging serve two different purposes and in any given situation you may want one, the other or both. Exceptions are for programmatic consumption and control flow within your code, while logging is for external consumption, by either a human or some sort of automated log watcher (Splunk for example). For instance, you may log the values of certain variables to help you figure out what happened and improve the code, though in that case, a level of debug would be more appropriate. Or you may want a systems administrator to be aware of the issue and potentially take some action. In that case, setting the log level to error and using something like a syslog logging handler makes sense.
Consider you have this kind of code for flask api:
#app.route('/doing/foo')
def foo():
try:
...
except Exception as ex:
doing some stuff
Now doing some stuff accidentally threw another exception. How do you prevent the server from showing another generic internal server error without any clue what happened?
In other words, how do I catch exceptions that may have originated from the except clause?
Aha! That is a 100 years question.
First option is to move to Python 3.
Python 3 has exception chaining and will print both of the exceptions - the one being handled and the one that was thrown due to the mishandling.
Second option is to implement exception chaining by yourself:
Manually inject the new exception into the one being handled using another try-except clause inside the current except:.
Keep in mind it WILL cause circular reference. Since it's a one-time exception, I wouldn't bother dealing with it. Further usage of this method will require cleaning the traceback using traceback.clear_frames().
In a request handler, log the error and return an HTTP 500 error and some HTML noting that an "Internal error has occurred" and containing a link back to some safe point for users to restart whatever they were doing.
You can attempt to catch more specific exceptions (from most specific and common towards Exception (which is the most general of the general exceptions). You can also attempt to introspect on the exception/traceback object (ex in your example).
Mostly you just want to ensure that you don't display internal application state to the user through their browser (possible security implications) and that you don't simply drop the user's request on the floor.
I am using logging throughout my code for easier control over logging level. One thing that's been confusing me is
when do you prefer logging.error() to raise Exception()?
To me, logging.error() doesn't really make sense, since the program should stop when encountered with an error, like what raise Exception() does.
In what scenarios, do we put up an error message with logging.error() and let the program keep running?
Logging is merely to leave a trace of events for later inspection but does absolutely nothing to influence the program execution; raising an exception allows you to signal an error condition to higher up callers and let them handle it. Typically you use both together, it's not an either-or.
That purely depends on your process flow but it basically boils down to whether you want to deal with an encountered error (and how), or do you want to propagate it to the user of your code - or both.
logging.error() is typically used to log when an error occurs - that doesn't mean that your code should halt and raise an exception as there are recoverable errors. For example, your code might have been attempting to load a remote resource - you can log an error and try again at later time without raising an exception.
btw. you can use logging to log an exception (full, with a stack trace) by utilizing logging.exception().
I'm using Cython to interface an external C function. The function (a science code with 100k code lines) internally checks for invalid conditions and "exits" if they occur.
How can I get Cython to detect and raise an exception when the external function "exits" instead of "returns"?
Edit: user2864740 is right, cython actually exits. (I was running the function inside a python mulitprocessing.Process. To simplify I got rid of multiprocessing.) The question now is:
How can I prevent the "exit()" call from the external library to exit Cython (or Python, at a later point). A try-except construct doesn't catch this. Can this be caught as exception instead of the whole process dying?
Thanks!
There are guidelines for "Joining zombie processes" in the multiprocessing documentation.
https://docs.python.org/3/library/multiprocessing.html#programming-guidelines
If in the master process you find that the worker process is defunct, you can raise an exception.
Alternatively you could try registering an atexit handler from within the Cython code
http://linux.die.net/man/3/atexit
A third idea is to use some kind of LD_PRELOAD library to override the exit function so that it raises a Python exception.
I've written an http downloader using webclient.Agent.
The connectionLost function of a body consumer object is called with a reason parameter.
def connectionLost(self, reason):
if isinstance(reason.value, ResponseDone):
self.df.callback(None)
else:
self.df.errback(reason.value)
I would really want to know what are the possible exception classes in 'reason'
because I need to catch them all and deal with them later in the calling
function which uses inlineCallbacks.
So far I identified:
ConnectError, BindError, ConnectionClosed, ResponseFailed
Is this documented somewhere?
Most of the documentation contains just vague
"...errback with a description of the error..." statements.
You don't need to catch them all specifically. Exception handling respects inheritance: if you try to catch a base class and a subclass is raised, you'll catch that too. Exception is the base class for just about all exceptions in Python, so if you catch that, you'll catch just about everything.
try:
yield agent.request(...)
except Exception as e:
print 'Oh no, some failure'
else:
print 'Success'
There is no complete list of all the exceptions that Agent.request may fail with, because it is impossible to know this list in advance. Failures may be due to connection setup - but is it a plain TCP connection or an SSL connection for an HTTPS URL? Perhaps it's a connection over a SOCKS proxy due to an unfortunate user's network configuration. These may all fail in different ways. The same applies to the actual HTTP request itself - who knows how the server will behave? Perhaps it will respond with an unexpected error code, or perhaps it will respond with something Agent isn't even capable of parsing and trigger an exception from the low-level HTTP client protocol implementation. The parser is hand crafted, so there's probably a lot of different exceptions that could come from that kind of problem. And then there's the variation on content and transfer encoding, which may invoke lots of different functionality (eg, a gzip decoder) which adds still more possible exceptions.
So, instead of trying to come up with the full list of all of these exceptions at every place you use Agent.request, only specifically handle the exception types you know about and have special code for and then handle everything else with an except Exception.