Python curses - getting error texts in nice way - python

I am developing ncurses app in python. But everytime, when my second thread (or anything else) throw exception, or print som text, it mess my window.
Is there any option, how to catch this text and handle it properly, without ruining my awesome ncurses program design? :)
Thank you

First and foremost, catch all appropriate exceptions at the entry to your threads' call stacks. In all likelyhood, the slave threads should catch it in order to relay the exception (via Queue, e.g.) to the display, or perhaps a log.
Alternatively, you can suppress the printout of the default exception handler (though your question focuses on this aspect, it's almost certainly not what you want):
import sys
sys.stdout.close()
sys.stderr.close()

Related

Exception with Python requests module, "requests.exceptions.ConnectTimeout" is mutliple levels deep in code out of my control. How can I workaround?

There is no code to show as this is a conceptual question. So here is the concise concept breakdown:
Local server (my code) -calls-> acquire_remote_data function (my code) -calls-> data module (not my code) -data module uses-> requests (obviously not my code lol) which requests then throws the above titled exception if the server loses its internet connection temporarily.
I of course, first thing, used the "try/except" method in Python in the acquire_remote_data function to call the data module to quickly realize that it can't capture an exception so far down the path in modules I have no control over.
An option is to contact the author of the data module but who knows when, if ever, they will modify their code to capture the exception if they can.
Is there a workaround technique that can be used to keep this from crashing the server?
*** I figured it out. Using the try/except method was the correct first choice. However, I needed to use the except Exception catch all. I didn't really think of that the first time around because of my darn linter squawking about too general of an exception catch in the past. What I learned was how to shut my linter up about certain issues.

Between raise Error and logging the Error, which one is a better practice in Python?

Python tutorial have a section called Errors and Exceptions which use the structure as;
try:
[statement]
except [Built-in ExceptionType]:
do something when an exception has been captured.
finally:
do something whether exception occurred or not.
This can also handle by raise an Error directly, too.
try:
raise [Built-in ExceptionType]
except [Built-in ExceptionType above] as e:
print(f'foo error: {e}')
There are several built-in ExceptionType which the good practice is developer should catch every specific exceptions type that developer should look for. (refer from Why is "except: pass" a bad programming practice?)
However, after I reading logging section. I'm thinking about I want to log the error into the log file and let user to aware the error (maybe just print the text and user can inform IT support) rather than throwing an exception on screen. Therefore, instead of try / except combo above then I can use the following error message and logged it.
if len(symbol) != 1:
error_message = f'Created box with symbol={symbol}, width={width} and height={height}, Symbol needs to be a string of length 1'
logger.error(error_message)
print(f'\nError! symbol={symbol}, width={width} and height={height} -- Symbol needs to be a string of length 1')
return
I doubt that what the better current practice is between:
a) raise Error on screen and
b) logging the Error for further investigation?
c) Other method (please suggest me)
I hope that I'm not try to comparing two different things.For me, I want to choose b) and store the error in log. this is easier for investigation and does not give unnecessary information to the users. but I do not sure I'm on the right track or not. Thank you very much.
Logging and raising exceptions are two different things for two different purposes. Logs let you inspect what your program did after the fact. Raising exceptions has important effects on the program flow right now. Sometimes you want one, sometimes you want the other, sometimes you want both.
The question is always whether an error is expected or unexpected, whether you have some plan what to do in case it occurs, and whether it is useful to notify anyone about the occurrence of the error or not.
Expected errors that you have a "backup plan" for should be caught and handled, that is regular program control flow. Unexpected errors probably should halt the program, or at least the particular function in which they occurred. If a higher up caller feels like handling the exception, let them. And whether to log an error or not (in addition to handling or not handling it) depends on whether anyone can glean any useful insight from that log entry or whether it would just be noise.
As deceze already mentions, logging and exceptions are two totally distinct things. As a general rule:
you raise an exception when you encounter an unexpected / invalid condition that you cannot handle at this point of the code. What will became of this exception (whether someone up in the call stack will handle it or not) is none of your concern at this point
you only catch an exception when you can handle it or when you want to log it (eventually with additionnal context informations) THEN reraise it
at your application's top-level, you eventually add a catchall exception handler that can log the exception and decide on the best way to deal with the situation depending on the type of application (command-line script, GUI app, webserver etc...).
Logging is mostly a developper/admin tool used for post-mortem inspection, program behaviour analysis etc, it's neither an error handling tool nor an end-user UI feature. Your example:
if len(symbol) != 1:
error_message = f'Created box with symbol={symbol}, width={width} and height={height}, Symbol needs to be a string of length 1'
logger.error(error_message)
print(f'\nError! symbol={symbol}, width={width} and height={height} -- Symbol needs to be a string of length 1')
return
is a perfect antipattern. If your function expects a one character string and the caller passed anything else then the only sane solution is to raise an exception and let the caller deal with it. How it does is, once again, none of your concerns.
You should log all errors no matter if they are thrown or handled for log analysis and refactoring amongst other purposes.
Whether to throw or to handle the error usually depends on the intended purpose of the code and the severity of the error.
Although "throwing" should be only used in debugging and handled
'gracefully' by dedicated exception code in the production version of the
application. No user likes crashes.
If the error impacts the business logic, end result of the code, or can cause damages to the user of the software, you want to terminate the execution. You do it either by throwing the error or handling it through a dedicated error handling procedure that shutdowns the program.
If the error is not severe and does not impact the normal functionality of the program, you can choose whether you want to handle it or throw it based on the best practices in your team and requirements of the project.
From the Docs:
Returns a new instance of the SMTPHandler class. The instance is initialized with the from and to addresses and subject line of the email. The toaddrs should be a list of strings.
logging.handlers.SMTPHandler can be used to send logged error message.
import logging
import logging.handlers
smtp_handler = logging.handlers.SMTPHandler(mailhost=("example.com", 25),
fromaddr="from#example.com",
toaddrs="to#example.com",
subject="Exception notification")
logger = logging.getLogger()
logger.addHandler(smtp_handler)
you can break the logic here if the code execution needs to be stopped.
Also look around this answer i have referred earlier collate-output-in-python-logging-memoryhandler-with-smtphandler

Why is my threading/multiprocessing python script not exiting properly?

I have a server script that I need to be able to shutdown cleanly. While testing the usual try..except statements I realized that Ctrl-C didn't work the usual way. Normally I'd wrap long running tasks like this
try:
...
except KeyboardInterrupt:
#close the script cleanly here
so the task could be shutdown cleanly on Ctrl-C. I have never ran into any problems with this before, but somehow when I hit Ctrl-C when this particular script is running the script just exits without catching the Ctrl-C.
The initial version was implemented using Process from multiprocessing. I rewrote the script using Thread from threading, but same issue there. I have used threading many times before, but I am new to the multiprocessing library. Either way, I have never experienced this Ctrl-C behavior before.
Normally I have always implemented sentinels etc to close down Queues and Thread instances in an orderly fashion, but this script just exits without any response.
Last, I tried overriding signal.SIGINT as well like this
def handler(signal, frame):
print 'Ctrl+C'
signal.signal(signal.SIGINT, handler)
...
Here Ctrl+C was actually caught, but the handler doesn't execute, it never prints anything.
Besides the threading / multiprocessing aspect, parts of the script contains C++ SWIG objects. I don't know if that has anything to do with it. I am running Python 2.7.2 on OS X Lion.
So, a few questions:
What's going on here?
How can I debug this?
What do I need to learn in order to understand the root cause?
PLEASE NOTE: The internals of the script is proprietary so I can't give code examples. I am however very willing to receive pointers so I could debug this myself. I am experienced enough to be able to figure it out if someone could point me in the right direction.
EDIT: I started commenting out imports etc to see what caused the weird behavior, and I narrowed it down to an import of a C++ SWIG library. Any ideas why importing a C++ SWIG library 'steals' Ctrl-C? I am not the author of the guilty library however and my SWIG experience is limited so don't really know where to start...
EDIT 2: I just tried the same script on a windows machine, and in Windows 7 the Ctrl-C is caught as expected. I'm not really going to bother with the OS X part, the script will be run in an Windows environment anyway.
This might have to do with the way Python manages threads, signals and C calls.
In short - Ctrl-C cannot interrupt C calls, since the implementation requires that a python thread will handle the signal, and not just any thread, but the main thread (often blocked, waiting for other threads).
In fact, long operations can block everything.
Consider this:
>>> nums = xrange(100000000)
>>> -1 in nums
False (after ~ 6.6 seconds)
>>>
Now, Try hitting Ctrl-C (uninterruptible!)
>>> nums = xrange(100000000)
>>> -1 in nums
^C^C^C (nothing happens, long pause)
...
KeyboardInterrupt
>>>
The reason Ctrl-C doesn't work with threaded programs is that the main thread is often blocked on an uninterruptible thread-join or lock (e.g, any 'wait', 'join' or just a plain empty 'main' thread, which in the background causes python to 'join' on any spawned threads).
Try to insert a simple
while True:
time.sleep(1)
in your main thread.
If you have a long running C function, do signal handling in C-level (May the Force be with you!).
This is largely based on David Beazley's video on the subject.
It exits because something else is likely catching the KeyboardInterupt and then raising some other exception, or simply returning None. You should still get a traceback to help debug. You need to capture the stderr output or run your script with the -i commandline option so you can see traceback. Also, add another except block to catch all other exceptions.
If you suspect the C++ function call to be catching the CTRL+C try catching it's output. If the C function is not returning anything then there isn't much you can do except ask the author to add some exception handling, return codes, etc.
try:
#Doing something proprietary ...
#catch the function call output
result = yourCFuncCall()
#raise an exception if it's not what you expected
if result is None:
raise ValueError('Unexpected Result')
except KeyboardInterupt:
print('Must be a CTRL+C')
return
except:
print('Unhandled Exception')
raise
how about atexit?
http://docs.python.org/library/atexit.html#module-atexit

Handle assertion dialog box with python subprocess

I am using python to create a sub process to check and see that no assertions occur.
I want to catch the error output along with the return code. That works fine, but the problem I run into is that when it runs into the assertion it gives me a dialog box that just hangs there. I have to then click the assertion box before I retrieve any information. Is there a way to make it not pop up and continue with the program or to send a message to close the window?
This is a problem since this is an automation service.
import subprocess
pipe = subprocess.Popen('test2.exe', shell=True, stderr=subprocess.PIPE)
for line in pipe.stderr:
print line
The executable is compiled from c++ code and has an assertion that will fail for testing purposes.
There's not really an easy solution in general, since a program could in theory create any number of windows waiting for user input. If you have the source code for the inferior process, the easiest thing to do would be to modify it to call _set_abort_behavior(0, _CALL_REPORTFAULT) to disable the message box.
If you don't have the source code, it's going to be much, much tougher. You could probably write a big hack that did something like attaching a debugger to the inferior process and setting a breakpoint on the call to abort(). If that breakpoint gets hit, kill the process and return an appropriate error status. But that's an extreme non-trivial kludge.
As I mentioned in the comments, pop-ups for assertions isn't a normal thing in Python. If you can modify the code running in a subprocess, you might be able to capture the assertion and handle it yourself, rather than letting the environment handle it with a popup.
import sys
try:
code that raises the assertion
catch AssertionError, e:
sys.stderr.write("Assertion failed: " + str(e))
sys.exit(1)
If it's looking for assertions in particular, this should work, because it will capture the assertion, print an error message and then raise a simple SystemExit exception instead.

Crash reporting in Python

Is there a crash reporting framework that can be used for pure Python Tkinter applications? Ideally, it should work cross-platform.
Practically speaking, this is more of 'exception reporting' since the Python interpreter itself hardly crashes.
Here's a sample crash reporter:
Rather than polluting your code with try..except everywhere, you should just implement your own except hook by setting sys.excepthook. Here is an example:
import sys
import traceback
def install_excepthook():
def my_excepthook(exctype, value, tb):
s = ''.join(traceback.format_exception(exctype, value, tb))
dialog = ErrorReportDialog(None, s)
dialog.exec_()
sys.excepthook = my_excepthook
Call install_exception() when your application starts.
ErrorReportDialog is a Qt dialog I've made. traceback.format_exception() will format argument passed to the except hook in the same way it does in Python's interpreter.
EDIT: I forgot to mention a little gotcha with that. It doesn't work with threads (well, at least it didn't last time I checked). For code running in another thread, you will need to wrap it in a try..except block.
Stick try excepts everywhere your application can crash (I/O, networking etc.). Whenever an except is called, call a function that will kill the old window, spawn a new tkinter notification window, or a custom one with your error message.
Do a root.after to the new window and send your error report (urllib).
Put a restart button if you wish.
There is no crash reporting framework - as tkinter is not that type of GUI. It's pretty much a wrapper for simple command line apps.
Go pyqt/gtk or wxpython if you want the features seen in the screen-shot above. But I'm pretty sure that where ever you go, you'll have to write your own reporter.

Categories