Python - do something and continue when process finished with exit code 1 - python

I have created a text file content reader - I have created several functions but I can't handle all cases - if some case is not defined, the program returns the message "Process finished with exit code 1" - Is it possible to get around this? If there is a problem - the Process finished with exit code 1 message - then I would like to save the name of the file and continue reading the rest of the files - at the moment it works so that if there is an error the program just stops working - but I want it to save the name of the file with the error and continue working. How to add such a condition to existing functions?
Please give me some advice
Example of such an error:
import pandas as pd
list=['a',2,3,4]
df = pd.DataFrame(list)
df[0] = df[0].astype(int)
print (df)
Can't convert a character into an int so an error occurs.
This is only an example of a possible error, but the name of the file where the error occurs should be saved and the loop should continue.

As you haven't included any code, I assume that the error was created by a block of code. In that case, you can enclose it within a try block and add an 'except` block to save the file's name in a file.
for example,
try :
# to read your files
except: # an exception occurred
# save the file name in a log
continue
try to share your code for better help.
EDIT 1 : Added an explanation for using plain except.
As documented, SystemExit does not inherit from Exception. You would have to use except BaseException.
However, this is for a reason:
This exception inherits from BaseException instead of StandardError or
Exception so that it is not accidentally caught by code that catches
Exception.
It is unusual to want to handle "real" exceptions in the same way you want to handle SystemExit. You might be better off catching SystemExit explicitly with except SystemExit.

Related

How to make python not close it self after error, python exception

I have made an program in python that i converted into .exe using auto-py-to-exe and im wondering how to stop .exe stop closing it self after an error (python exception) the .exe closes it self in the speed of light and you cant read the error.
Does someone knows how to make it not close it self?
using input doesnt work if the exception happens in a pip library
Thanks
You can use a try-except based system (that you have to build suitably to catch every exception of your code) and to print the exception you can use the module traceback like so:
try:
## code with error ##
except Exception:
print("Exception in user code:")
print("-"*60)
traceback.print_exc(file=sys.stdout)
print("-"*60)
or you could use a simpler form like:
try:
## code with error ##
except Exception as e:
print(e, file='crash log.txt')
that only prints the error class (like file not found).
I should also point out that the finally keyword exists with the purpose of executing code either if an exception arose or not:
try:
## code with error ##
except: #optional
## code to execute in case of exception ##
finally:
## code executed either way ##
Something you could do on top of that is logging everything your program does with
print(status_of_program, file=open('log.txt','a'))
this is useful to see exactly at what point the program has crashed and in general to see the program in action step by step.
But a thing you should do is properly test the program while in .py form and if it works you could possibly assume the error comes from the actual exportation method and consult the documentation or try exporting simpler programs to catch the difference (and so the error).
i'd suggest to consult:
https://docs.python.org/3/library/exceptions.html
to learn the error types and the try-except construct.
If input() doesn't work try using sys.stdout().
I gather that this is a console app. If it's a GUI app, you can use a similar approach but the details will be different.
The approach I'd use is to set sys.excepthook to be your own function that prints the exception, then waits for input. You can call the existing exception hook to actually do the printing.
import sys, traceback as tb
oldhook = sys.excepthook
def waitexcepthook(type, exception, traceback):
oldhook(type, exception, traceback)
input()
sys.excepthook = waitexcepthook

Python as Hive UDF - Clean Exit on Exception

I'm trying to do a clean exit from the program whenever my python as Hive UDF fails with an exception.
Here is an example:
SELECT TRANSFORM (id, name) USING
'D:\Python27\python.exe streaming.py' AS (id string,
name string, count integer) FROM hivesampletable;
#streaming.py
import sys
from datetime import datetime
try:
for line in sys.stdin.readlines():
fields = line.strip().split('\t')
fields.append(len(name))
print "\t".join(fields)
except:
#I want program to break/clean exit with out printing (writing back) to table
Ideas appreciated
The pass statement will let you ignore the error and return control flow to your program.
except Exception, e:
pass
#To see your exception
print str(e)
# Alternately, you could get details of your exception using
print sys.exc_info()
If you essentially want to "swallow" the exception, then I would recommend in the except block you explicitly call sys.exit(0), which will both exit the program and indicate (from a shell level) that the program is "OK".
e.g. You will end up with a a truly clean exit that even a shell, e.g. bash, will see as "success".
Note: If you want to exit without printing anything but allow a shell to know something went awry, pass a non-zero error code to exit.
Response to comment from OP:
Hmm, I wouldn't expect that, since you're explicitly swallowing the exception...
The next step would likely be to print out what the exception is, as was suggested in the other answer, and go from there, depending on what that exception is.
Another thing that may be contributing is I don't think your Python script matches your TRANSFORM statement, so that may be contributing to the issue.
Also, you are referencing name without that being initialized (that might be the exception here -- NameError: name 'name' is not defined).

why do we use 'pass' in error handling of python?

It is conventional to use pass statement in python like the following piece of code.
try:
os.makedirs(dir)
except OSError:
pass
So, 'pass' bascially does not do anything here. In this case, why would we still put a few codes like this in the program? I am confused. Many thanks for your time and attention.
It's for the parser. If you wrote this:
try:
# Code
except Error:
And then put nothing in the except spot, the parser would signal an error because it would incorrectly identify the next indentation level. Imagine this code:
def f(x):
try:
# Something
except Error:
def g(x):
# More code
The parser was expecting a statement with a greater indentation than the except statement but got a new top-level definition. pass is simply filler to satisfy the parser.
This is in case you want the code to continue right after the lines in the try block. If you won't catch it - it either skips execution until it is caught elsewhere - or fails the program altogether.
Suppose you're creating a program that attempts to print to a printer, but also prints to the standard output - you may not want it to file if the printer is not available:
try:
print_to_printer('hello world')
except NoPrinterError:
pass # no printer - that's fine
print("hello world")
If you would not use a try-catch an error would stop execution until the exception is caught (or would fail the program) and nothing would be printed to standard output.
The pass is used to tell the program what to do when it catches an error. In this particular case you're pretty much ignoring it. So you're running your script and if you experience an error keep going without worrying as to why and how.
That particular case is when you are definite on what is expected. There are other cases where you can break and end the program, or even assign the error to a variable so you can debug your program by using except Error as e.
try:
os.makedirs(dir)
except OSError:
break
or:
try:
os.makedirs(dir)
except OSError as e:
print(str(e))
try:
# Do something
except:
# again some code
# few more code
There are two uses of pass. First, and most important use :- if exception arises for the code under try, the execution will jump to except block. And if you have nothing inside the except block, it will throw IndentationError at the first place. So, to avoid this error, even if you have nothing to do when exception arises, you need to put pass inside except block.
The second use, if you have some more code pieces after the try-except block (e.g. again some code and few more code), and you don't put pass inside except, then that code piece will not be executed (actually the whole code will not be executed since compiler will throw IndentationError). So, in order to gracefully handle the scenario and tell the interpreter to execute the lines after except block, we need to put pass inside the except block, even though we don't want to do anything in case of exception.
So, here pass as indicated from name, handles the except block and then transfers the execution to the next lines below the except block.

Get Exit Status of a Nested Function?

I have a function called within a different function. In the nested function, various errors (e.g. improper arguments, missing parameters, etc.) should result in exit status 1. Something like:
if not os.path.isdir(filepath):
print('Error: could not find source directory...')
sys.exit(1)
Is this the correct way to use exit statuses within python? Should I have, instead,
return sys.exit(1)
??? Importantly, how would I reference the exit status of this nested function in the other function once the nested function had finished?
sys.exit() raises a SystemExit exception. Normally, you should not use it unless you really mean to exit your program.
You could catch this exception:
try:
function_that_uses_sys.exit()
except SystemExit as exc:
print exc.code
The .code attribute of the SystemExit exception is set to the proposed exit code.
However, you should really use a more specific exception, or create a custom exception for the job. A ValueError might be appropriate here, for example:
if not os.path.isdir(filepath):
raise ValueError('Error: could not find source directory {!r}'.format(filepath))
then catch that exception:
try:
function_that_may_raise_valueerror()
except ValueError as exc:
print "Oops, something went wrong: {}".format(exc.message)
By using sys.exit, you typically signal that you want the entire program to end. If you want to handle the error in a calling function, you should probably have the inner function raise a more specific exception instead. (You could catch the exception raised by SystemExit, but it would be a rather awkward way to pass error information out.)
I guess that the right thing to do is this:
if not os.path.isdir(filepath):
raise ValueError('the given filepath is not a directory')
However, the code as it stands still could be improved. One point is that a path to a file should never be a directory, so that is not an exceptional state. Maybe what you want is to just name it path without adding an unintended connotations.
Further, and that has actual functional implications, you are still not guaranteed to be able to access a directory there even if isdir() returns true! The reason is that something could have switched the thing under your feet, typically a malicious attacker, or, more simple, you could simply not have the rights to access it. If you care, you should rather just open the directory and handle the according errors instead of trying in advance to determine if something in the future will fail. This is in general a better approach, as the "normal" code doesn't get cluttered by such checks and you also don't pay any albeit small performance penalty except when an error occurs.

Handling Exceptions with else clause

The Python Tutorial states:
The try ... except statement has an optional else clause, which,
when present, must follow all except clauses. It is useful for code
that must be executed if the try clause does not raise an exception.
For example:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except IOError:
print 'cannot open', arg
else:
print arg, 'has', len(f.readlines()), 'lines'
f.close()
The use of the else clause is better than adding additional code
to the try clause because it avoids accidentally catching an exception
that wasn’t raised by the code being protected by the try ... except
statement.
Question 1> After reading the above document, I still don't get the idea why we cannot simply move the code from else clause into try clause.
Question 2> How does try clause can accidentally catch an exception since all those catches are done in the except clause, right?
You could put the else code in the try suite, but then you'd catch any exceptions that might be raised there. If you didn't intend that to happen, it would be "accidental," hence the wording of the document you linked to.
Best practice is to put as little code as possible in a try block, so that when an error occurs, you know what operation caused it and can handle it appropriately. If you have five lines of code in a try block and only expect one of them to ever raise an exception, your exception-handling code will be ill-prepared when an exception occurs in a line you didn't expect it to. Better in that case to let the exception be raised than handle it the wrong way.
If you move the code from the else into the try then that becomes part of the "critical path" which can raise an exception. If f.readlines() raises some sort of exception (perhaps an I/O error while reading the file because of a bad sector on the disk) then that error will be conflated with the one error that you currently catch. (Technically the "cannot open" error message would be wrong at that point ... because opening a file can succeed when reading it later fails; in fact opening it must succeed before you can even get an I/O error while processing it).
Normally you'd use a pattern more like:
foo = None
try:
# some code to access/load/initialize foo from an external source
except ...:
# handle various types of file open/read, database access, etc errors
else:
foo = something
... so that your subsequently run code and simply check if foo is None and use it or
work around it's unavailability in whatever way you see fit.
Answer for both questions is similar,
If you move code to the try clause, then you can not be sure from where the exception is coming from. Thus if you have another line of code that produces an unexpected IOError you could end searching for a problem where there is not.
So to better disect your code you want to simplify as much as possible the lines in the try making the catch as especific as possible.
1) Of course you could just move code from the else clause into the try clause. You could move it outside of the try block entirely, but this allows extra flexibility, and further modulation of the code. Also, perhaps specificity with the errors being caught. You could list a whole load of different exceptions that might be likely to happen, each with different statements. The stuff in the else clause would still only happen if no exception was raised, after the execution of the final line in the try block. e.g. printing a successful return message.
Also, try clauses add some extra CPU overhead with regards to garbage collection management and error tracing, so anything outside of the try clause is not protected in the same manner and may run more efficiently.
2) Error catching is quite specific. e.g. The except clause in your above example will only be run if an IOError is raised when running the f = open(arg,'r') line. If you want to catch any form of exception, use except Exception:.

Categories