We Keep Coding

python try statement or if statement

Hello everyone I have my code almost done but I'm trying to add in some sort of check to avoid errors. But I'm not really understanding what statement would be better to use to test the code. I know there are a few options of either using a loop, if-statement, or try. But here is the code in regards to doing captcha. I need it to run the first set of code which if the captcha doesn't pop up I continue on. But if the captcha does pop up solve it then continue on.
Some times captcha doesnt appear and if I run the whole set of code I get an error because we are expecting captcha to pop up.
Or if the captcha does appear to solve it which would.
I would really appreciate any help please as I'm not sure the right statement to use.

try should be used when something would return an error and would otherwise cause your program to stop/crash. An example of this would be:
try:
import pandas
except:
print("Unable to import pandas. Module not installed")
In this example your program will attempt to import the pandas module. If this fails it will then print out a line of text and continue running.
if statements are used to decided when to do something or not based on the returned logic. The key difference is that logic IS returned and not an error.
if x > 10:
print("This is a large number")
else:
print("This is a small number")
With this example, if 'x' did not exist it would produce an error, no more code will be executed, and the program will crash. The main difference between IF and TRY is whether logic is returned as true/false or is something just plains fails.
With your specific example it is important to know if the captcha appearing or not will break your code. Does the logic boil down to captcha = false or does captcha not exist at all and logic fails entirely?

Q: How do you define sometimes captcha doesn't appear (1%, 20%, 50%, ...)?
A: Maybe 5% of the time captcha doesn't appear.
In this case, I prefer to use Exception handling: do stuff and if something goes wrong, fix it
try:
# > 95% of the time the code below works when the captcha appears
except SomeException:
# < 5% of the time the code is called when the captcha doesn't appear
IMHO, you have not really 2 different codes: you have one and a fallback solution, it's really different than:
if x % 2:
...
else:
...

In python, how do I re-run a function if it throws an error

Below are snippets of my codes, what I wanted to know is hypothetically if function main2() throws out an error for some reason, how do I get my exception to run the same function again say 3 times before it breaks?
Just to add here, any of the functions could throw an error (not just main2()) also I might have not just 3 but many more functions
import numpy as np
def main():
np.load('File.csv')
def main1():
np.load('File1.csv')
def main2():
np.load('File2.csv')
for i in range(1, 10):
try:
main()
main2()
main3()
except Exception as e:
print e
else:
break

You could do it with python retry decorator
#retry((Exception), tries=3, delay=0, backoff=0)
def main2():
np.load('File2.csv')
This would work the same way as if you would write:
error_counter = 0
def main2():
try:
np.load('File2.csv')
except:
if error_counter < 3
error_counter += 1
main2()
raise Exception("Will not try again, have tried 3 times")
error_counter = 0
If you want to make it robust and clean, you should go for the first solution. The first solution you can reuse in a large enterprise project and due to the backoff time it can take disk load,user load network issues into consideration with backoff/delay time.
If you don't use a time delay, you will make all the 3 tries in just a second. That is fine for some kind of exceptions but when having network issues, or disk issues you want a more sophisticated solution.
Also, consider to not catch all exceptions, it is a bad practice to cache all. More info, why it is bad

Here's an idiom you can try:
for _ in range(3): # try 3 times
try:
main2()
break # as soon as it works, break out of the loop
except Exception as e:
print e
continue # otherwise, try again
else: # if the loop exited normally, e.g. if all 3 attempts failed
pass
# do_something...
Note the indentation. The else here is attached to the for, not the try.

Unfortunately, implementing a custom retry decorator can often be a bit of a pain. If you want to tweak their logic or adjust them, it can actually get quite complicated quite fast. There's a Python library out there called Backoff-Utils that supports very robust and easily-extensible retry / backoff strategies (full disclosure: I'm biased, since I'm the author of that library).
In your hypothetical question, you could use the library in a decorator-based strategy:
from backoff_utils import backoff, apply_backoff, strategies
#apply_backoff(strategies.Fixed, max_tries = 3, catch_exceptions = [type(ValueError)])
def main2():
# np.load('File2.csv')
raise ValueError
print("In main2")
or you could use it in a function-based strategy when calling main2():
result = backoff(main2,
max_tries = 3,
catch_exceptions = [type(ValueError)],
strategy = strategies.Fixed)
Of course, the code snippet above is specifically designed to do exactly what you described up above. It uses a linear strategy (just retrying 3 times, with a default delay of 1 second between attempts).
Using the library, you can employ any number of other retry / delay strategies, including Exponential Backoff, Fibonnaci, Linear Progression, and Polynomial. You can also customize and create your own delay strategies, as well. And you can incorporate custom success / failure handlers, and different alternative paths for different types of situations.
Of course, all of this flexibility is overkill for your particular use case - you don't need that much. But it may be easier than worrying about copying/pasting/maintaining your own retry decorator and gives you additional built-in options if you find you need more sophisticated retry strategies down the road.
In case it's helpful, you can find some pretty detailed documentation here: https://backoff-utils.readthedocs.io/en/latest/index.html
Hope this helps!

You can try
for i in range(1, 10):
error_max = 3
error_counter = 0
try:
main()
try:
main2()
except Exception as e:
counter += 1
if counter == 3:
raise e
else:
continue
main3()
except Exception as e:
print e
else:
break
This code will run function main2() until it will get 3 errors and on the first 2 it will make the loop run again.

You should handle all the errors within specific functions, otherwise if handling errors of all the functions together, the function coming prior to other functions will throw error and the control will execute the except block skipping rest of the code below it in the try block.
Try it yourself:
def main():
# np.load('File.csv')
raise ValueError
print("In main")
def main1():
# np.load('File1.csv')
raise ValueError
print("In main1")
def main2():
# np.load('File2.csv')
raise ValueError
print("In main2")
for i in range(1, 10):
try:
main()
main2()
main3()
except Exception as e:
print(e)
else:
break
Try commenting raised errors in the functions in different order.
And when the errors are handled within each function then every function is executed without any skipping of the remaining functions in the loop

How to avoid program freezing when connecting to server

I have a little script which filters those domain names which are not registred yet. I use pywhois module. The problem is that it suddenly freeze and do nothing after several (sometimes hundreds) of requests. I think it is not a ban because I can run the program right after freeze and it works.
I would like to avoid this freezing. My idea is to count runtime of the function and if the time cross some line (for example 10 seconds) it repeats the code.
Do you have any advice how to avoid the freezing? Or the better way to check domains?
Here is the code:
for keyword in keywords:
try:
details = pythonwhois.get_whois(keyword+'.com')
except Exception as e:
print e
continue
if 'status' not in details.keys():
print 'Free domain!'
print keyword

This method is prone to change (if the underlying library changes), however, you can call internal socket functions to set a timeout for all pythonwhois network calls. For example:
TIMEOUT = 5.0 # timeout in seconds
pythonwhois.net.socket.setdefaulttimeout(TIMEOUT)
pythonwhois.get_whois("example.com")

Maybe you could try dnspython. It looks like you just want to check if a domain name is registered. For example:
import dns.resolver
for keyword in keywords:
try:
dns.resolver.query(keyword+'.com')
except dns.resolver.NXDOMAIN:
print(keyword+'.com is available!')
DNS resolver has a default timeout of 2 seconds. If you want to change that, you can make a new instance of dns.resolver.Resolver with a different timeout.
To make it multithreaded, a thread pool would be the best choice if you can use python3:
from multiprocessing import Pool
def check_keyword(keyword):
try:
dns.resolver.query(keyword+'.com')
except dns.resolver.NXDOMAIN:
# You probably want to change this to a return
print(keyword+'.com is available!')
if __name__ == '__main__':
keywords = [...]
p = Pool(5)
print(p.map(check_keyword, keywords))

How to call code inside try block one more time if exception is thrown?

I am new to python and I am writing code that uses OAuth to authenticate and when the token expires after 60 minutes, it needs to get a new one.
try:
if uploadedContent is not None:
thing.action(uploadedContent)
except LoginOrScopeRequired:
print("Logging in...")
set_access_credentials({"identity", "submit"}, get_access_token())
I currently have this code to handle getting a new token if it expires, but the problem is that if there was an exception it skips over the action it needed to do. I understand that I could take what was inside the try block and append it to end of the except block, but it there a more elegant way to do this?
Some of my research led to the with statement, but I didn't understand with well enough to know if it would solve my problem. So is appending it to the end the best solution or is there something better?

It is considered Idiomatic Python to do this with a function decorator/wrapper:
Example:
#!/usr/bin/env python
from functools import wraps
def retry_on_error(ntries=1):
"""
A decorator that returns a wrapper function that calls
the wrapped function repeatedly up to ntries if an
exception is encountered.
"""
def decorator(f): # wrapping the original function
#wraps(f) # make the wrapped function look like the original
def wrapper(*args, **kwargs): # our function wrapped that calls the original
for i in xrange(ntries):
try:
return f(*args, **kwargs)
except Exception as e:
print("Error executing {0:s} retrying {1:d}/{2:d}".format(f.__name__, i, ntries))
print("Error was {0:s}".format(e))
return wrapper
return decorator # returning the new wrapped function
#retry_on_error()
def f():
n = getattr(f, "n", 0)
try:
if not n:
raise ValueError("n < 0")
finally:
setattr(f, "n", n + 1)
Output:
$ python -i foo.py
>>> f()
Error executing f retrying 0/1
Error was n < 0
>>> f()
>>>
See: Python Decorators for other examples.
Update: There is also a nice library that implements this functionality with a few more features: retrying as well as several other related/similar questions How to retry after exception in python? and Pythonic way of retry running a function
Update #2: I've commented the decorator a bit so hopefully you can understand what's going on in each step of the process. Admittedly decorators aren't that easy to understand at first so I recommend you read Understanding Python Decorators in 12 easy step

Some languages like ruby let you put a retry statement in the exception catch block that makes this incredibly simple. Unfortunately in Python you will need to wrap this in a while statement:
success = False
while not success
try:
if uploadedContent is not None:
thing.action(uploadedContent)
success = True
except LoginOrScopeRequired:
print("Logging in...")
set_access_credentials({"identity", "submit"}, get_access_token())
Note that the line success = True will only be reached if no exception occurs.
EDIT
You will also want to keep track of the number of attempts in a counter to ensure this doesn't loop forever and exits after 3 retries for example.

Python: How to try again when experiencing a timeout

I have a Python program that connects to a server to send it some commands, but occasionally I get this error:
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
In previous situations like this, I would use something like this:
try:
Do something
except KeyError:
do something else
Could I do the same thing in this same situation? I.e.,
try:
Do something
except TimeoutError:
Do something again
And if so, what would I do after the except TimeoutError? Would I just do the same command again?

Could I do the same thing in this same situtation
Yes! You can use try/except for any exception, and TimeoutError is nothing special.
and if so what would I do after the except TimeoutError? would I just do the same command again?
If you only want to retry once, and let a second timeout count as a real error, yes:
try:
do something
except TimeoutError:
do something
(If "do something" is more than a simple statement, you probably want to factor out the code so you don't repeat yourself.)
However, if you want to retry multiple times, you probably want a loop:
for _ in range(max_retries):
try:
do something
break
except TimeoutError:
pass
You may want to add an else clause to the for to distinguish the two cases (succeeded, and did a break, vs. never succeeded and just ran out of tries).
Since the idea here is usually to deal with possibly-transient errors, there are additional things you might want to add, such as:
Re-raising the error, or raising a different one, after max_retries.
Logging at progressively higher levels (e.g., a debug message for a single failure, but a warning for `max_retries).
Retrying with exponential backoff (wait 1 second, then 2, 4, 8, …).
Pushing the URL to the end of the work queue instead of retrying immediately. You can use (URL, retry_count) pairs if you also want max_retries logic, (URL, timestamp) pairs if you also want exponential backoff, or both if you want both. (Of course this only works if you don't care about the order of responses, or can reorder them at the end.)
Different rules for different exceptions (e.g., 500, 502, 503, and 504 errors can all be caused by overload on a server or proxy, but the best retry logic may be different—and the best heuristics for 2018 may be different from 2010 or 2025).
For complicated combinations, a retry decorator, like the one linked in jterrace's helpful answer, is a great way to wrap up the behavior.

You can catch the TimeoutError like you mentioned:
import socket
import sys
try:
dosomething()
except socket.TimeoutError:
print >> sys.stderr, 'Retrying after TimeoutError'
dosomething()
You could also use the retry decorator pattern on a function:
#retry(socket.TimeoutError)
def dosomething():
# code that causes a TimeoutError
...

def f():
pass #insert code here
To repeat once after the error:
try:
f()
except TimeoutError:
f()
Or to loop until success:
while True:
try:
f()
break
except TimeoutError:
pass
Or with a limited number:
attempts = 3
while attempts:
try:
f()
break
except TimeoutError:
attempts -= 1

Please run each example. They are ready!
Example 1
import sys
try:
incorrect.syntaxThatIJustMadeUP()
except:
print((sys.exc_info()[0])) #Now you know what to except and CATCH
else:
print("You will never see this message")
Example 2
import sys
try:
incorrect.syntaxThatIJustMadeUP()
except NameError:
print("There is a problem with your SYNTAX, Dude!")
except:
print((sys.exc_info()[0])) # In case another uncontrollable network problem occurs, User rages-Snaps IJ45
else:
print("You will never see this message unless TRY succeeds")
print("Why not put this try in a loop?")