We Keep Coding

python try statement or if statement

Hello everyone I have my code almost done but I'm trying to add in some sort of check to avoid errors. But I'm not really understanding what statement would be better to use to test the code. I know there are a few options of either using a loop, if-statement, or try. But here is the code in regards to doing captcha. I need it to run the first set of code which if the captcha doesn't pop up I continue on. But if the captcha does pop up solve it then continue on.
Some times captcha doesnt appear and if I run the whole set of code I get an error because we are expecting captcha to pop up.
Or if the captcha does appear to solve it which would.
I would really appreciate any help please as I'm not sure the right statement to use.

try should be used when something would return an error and would otherwise cause your program to stop/crash. An example of this would be:
try:
import pandas
except:
print("Unable to import pandas. Module not installed")
In this example your program will attempt to import the pandas module. If this fails it will then print out a line of text and continue running.
if statements are used to decided when to do something or not based on the returned logic. The key difference is that logic IS returned and not an error.
if x > 10:
print("This is a large number")
else:
print("This is a small number")
With this example, if 'x' did not exist it would produce an error, no more code will be executed, and the program will crash. The main difference between IF and TRY is whether logic is returned as true/false or is something just plains fails.
With your specific example it is important to know if the captcha appearing or not will break your code. Does the logic boil down to captcha = false or does captcha not exist at all and logic fails entirely?

Q: How do you define sometimes captcha doesn't appear (1%, 20%, 50%, ...)?
A: Maybe 5% of the time captcha doesn't appear.
In this case, I prefer to use Exception handling: do stuff and if something goes wrong, fix it
try:
# > 95% of the time the code below works when the captcha appears
except SomeException:
# < 5% of the time the code is called when the captcha doesn't appear
IMHO, you have not really 2 different codes: you have one and a fallback solution, it's really different than:
if x % 2:
...
else:
...

Python as Hive UDF - Clean Exit on Exception

I'm trying to do a clean exit from the program whenever my python as Hive UDF fails with an exception.
Here is an example:
SELECT TRANSFORM (id, name) USING
'D:\Python27\python.exe streaming.py' AS (id string,
name string, count integer) FROM hivesampletable;
#streaming.py
import sys
from datetime import datetime
try:
for line in sys.stdin.readlines():
fields = line.strip().split('\t')
fields.append(len(name))
print "\t".join(fields)
except:
#I want program to break/clean exit with out printing (writing back) to table
Ideas appreciated

The pass statement will let you ignore the error and return control flow to your program.
except Exception, e:
pass
#To see your exception
print str(e)
# Alternately, you could get details of your exception using
print sys.exc_info()

If you essentially want to "swallow" the exception, then I would recommend in the except block you explicitly call sys.exit(0), which will both exit the program and indicate (from a shell level) that the program is "OK".
e.g. You will end up with a a truly clean exit that even a shell, e.g. bash, will see as "success".
Note: If you want to exit without printing anything but allow a shell to know something went awry, pass a non-zero error code to exit.
Response to comment from OP:
Hmm, I wouldn't expect that, since you're explicitly swallowing the exception...
The next step would likely be to print out what the exception is, as was suggested in the other answer, and go from there, depending on what that exception is.
Another thing that may be contributing is I don't think your Python script matches your TRANSFORM statement, so that may be contributing to the issue.
Also, you are referencing name without that being initialized (that might be the exception here -- NameError: name 'name' is not defined).

Guidance on exceptions given by the Python docs

According to the Python documentation on idioms and anti-idioms in relation to exceptions: "You should try to use as few except clauses in your code as you can — the ones you do use will usually be inside calls which should always succeed, or a catch-all in a main function." Taking this sentence in sections...
"You should try to use as few except clauses in your code as you can"
A bit confusing for a newbie like myself, I thought it was good practise in Python to use the EAFP style -i.e. many try and except statements. Or am I missing the point?
"the ones you do use will usually be inside calls which should always succeed"
I don't understand the point that's being made here.
"or a catch-all in a main function."
So it's good style to any let code that throws an exception to simply pass it up the call stack until it reaches the top level where you have really generic exception handling?

"You should try to use as few except clauses in your code as you can"
It's easy to litter your code with exceptions:
def solve_linear(mat1, mat2):
det1 = determinant(mat1)
det2 = determinant(mat2)
try:
return det1 / det2
except ZeroDivisionError:
raise NoSolution
Here, it's probably fine to let the ZeroDivisionError propagate. You don't need to catch it.
"the ones you do use will usually be inside calls which should always succeed"
For example, look at this code which reads a file, or returns a cached value. It normally succeeds, in spite of the KeyError exception:
def read_file(path):
try:
return cache[path]
except KeyError:
fp = open(path, 'rb')
data = fp.read()
fp.close()
cache[path] = data
return data
"or a catch-all in a main function."
If your program is interactive, you'll probably want to catch almost everything at the top level. Here's the top loop of an interactive command-line program:
def main():
while True:
try:
text = raw_input('enter command: ')
cmd = parse_command(text)
except EOFError:
return
except KeyboardInterrupt:
return
except ValueError:
print 'Syntax error'
continue
try:
run_cmd(cmd)
except SystemExit:
raise
except Exception:
traceback.print_exc()

Concerning the first point, the whole point of using exceptions is that you don't have to wrap every line in one! E.g. in C, errors are usually determined by the return value of a function call. So you have to check those after every call if you want to catch all errors. With Python you can group a (possibly large) block of statements that go together in a try/except block and only deal with all errors once.
The second point is that (if possible) you want to solve failures close to the point where they occur. E.g. you are reading data from a network and get zero bytes. In that case it is usually perfectly allright to wait and try again.
The last point is that sometimes an error is so big that it cannot be handled at a low level. E.g. if you are trying to open a file that not exist, it wil fail. And your program cannot do whatever it was going to do with the contents of the file. It is best to handle that at the top level of the program and ask the user for another file name or maybe quit the program.

I think the point is that exceptions should be used only for 'exceptional' circumstances. That is the meaning behind the use "in calls which will usually succeed". An example of this might be some computation that under some really weird circumstances ends having to do a division by zero. Then you can enclose that in a try / except statement to deal with that possibility.
The catch-all in a main function would be applicable to the same scenario. Say your calculations end up with a division by zero somewhere deep in the call stack. From this point you can't continue, so there's no point in having a try/except statement right at the point of failure. It would make more sense to just have one at a higher level where you can reasonably recover from the error.
The example they give in the documentation is an example of this. When calling 'get_status' you would expect the file to be there. If it doesn't then you have the except statement to deal with that (although, as mentioned, in that particular case the 'with' statement is much better).

The Python philosophy is typically to "ask forgiveness, not permission." But the idea isn't to use a try-except clause to catch ALL possible errors. Ideally, each try-except will only catch the error(s) that are relevant.
BAD (doesn't specify a specific exception type):
a = [1, 2, 3]
i = 3
try:
print a[i]
except:
print "Not found!"
GOOD (only handles the exception we expect to get):
a = [1, 2, 3]
i = 3
try:
print a[i]
except IndexError:
print "Not found!"
The reason this is important is so we don't obscure other possible code errors. What if, in this example, i was 1.8? The first example would print Not Found!, obscuring the real issue, but the second example would return TypeError: list indices must be integers, not float letting us know that there's a logical flaw in our code.

confusing exception behavior in python

I have some django code that resembles this (this is python 2.7.1)
try:
a = some_model.objects.get(some_field='foo') #this could except if for some reason it doesn't exist
method_that_throws_exception() #it won't reach this if we get a DoesNotExist
except some_model.DoesNotExist:
#if it doesn't exist create it then try again
a = some_model.objects.create(....)
try:
method_that_throws_exception() #this time lets say another exception is raised by method
print 'this should not print right?'
except Exception as e:
logging.error("error msg here")
The problem is the "this should not print" line is still being printed. I'm confused by this. I feel like I am probably overlooking something very simple but might be having some tunnel vision at the moment. Thanks in advance.
Update: also if I remove the nested try block the print below the call to the method that throws an exception still prints.

I figured it out, the method had a try block inside of it that Iw asn't raising out. adding raise in my exception handler inside the method_that_throws_exception() fixed my problem.

Trapping MySQL Warnings In Python

I would like to catch and log MySQL warnings in Python. For example, MySQL issues a warning to standard error if you submit 'DROP DATABASE IF EXISTS database_of_armaments' when no such database exists. I would like to catch this and log it, but even in the try/else syntax the warning message still appears.
The try/except syntax does catch MySQL errors (eg, submission of a typo like 'DRP DATABASE database_of_armaments').
I have experimented with <<except.MySQLdb.Warning>> -- no luck. I've looked at the warnings module, but don't understand how to incorporate it into the try/else syntax.
To be concrete, how do I get the following (or something like it) to work.
GIVEN: database 'database_of_armaments' does not exist.
try:
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
except: <<WHAT DO I PUT HERE?>>
print 'There was a MySQL warning.'
<<AND what goes here if I want to get and manipulate information about the warning?>>
UPDATE:
Thanks for the comments. I had tried these and they didn't work -- but I had been using a DatabaseConnection class that I wrote for a connection, and its runQuery() method to execute. When I created a connection and cursor outside the class, the try/except Exception caught the "Programming Error", and except MySQLdb.ProgrammingError worked as advertised.
So now I have to figure out what is wrong with my class coding.
Thank you for your help.

Follow these steps.
Run it with except Exception, e: print repr(e).
See what exception you get.
Change the Exception to the exception you actually got.
Also, remember that the exception, e, is an object. You can print dir(e), e.__class__.__name__, etc.to see what attributes it has.
Also, you can do this interactively at the >>> prompt in Python. You can then manipulate the object directly -- no guessing.

Have you tried something like this?
try:
cursor.execute(some_statement)
except MySQLdb.IntegrityError, e:
# handle a specific error condition
except MySQLdb.Error, e:
# handle a generic error condition
except MySQLdb.Warning, e:
# handle warnings, if the cursor you're using raises them

I think the exception you want to catch is a MySQLdb.ProgrammingError, and to get information about it, just add a variable to store the error data (a tuple) in after that i.e:
try:
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
except MySQLdb.ProgrammingError, e:
print 'There was a MySQL warning. This is the info we have about it: %s' %(e)

I managed to trap the mysql warning like this:
import _mysql_exceptions
try:
foo.bar()
except _mysql_exceptions.Warning, e:
pass

first you should turn on warnings to treat as exceptions, and only then you can catch them.
see "error" warnings filter - https://docs.python.org/3/library/warnings.html#the-warnings-filter

Plain and simple
def log_warnings(curs):
for msg in curs.messages:
if msg[0] == MySQLdb.Warning:
logging.warn(msg[1])
cursor.execute('DROP DATABASE IF EXISTS database_of_armaments')
log_warnings(cursor)
msg[1] example :- (u'Warning', 1366L, u"Incorrect integer value: '\xa3' for column 'in_userid' at row 1")