I have try/except block which handles API requests to some client.
while attempts < 10:
try:
r = requests.post(server, data=contents,
auth=HTTPBasicAuth(service_userid, service_pswd))
r.raise_for_status()
except requests.exceptions.HTTPError as errh:
print ('Http Error:',errh)
attempts += 1
if attempts == 10:
body = 'Http Error: ' + str(errh)
subject = 'Failure'
sendEmailMessage(SMPTHOST, fromEmailAddr, toEmailAddr, subject, body)
except requests.exceptions.ConnectionError as errc:
print ('Error Connecting:',errc)
attempts += 1
if attempts == 10:
body = 'Error Connecting: ' + str(errh)
subject = 'Failure'
sendEmailMessage(SMPTHOST, fromEmailAddr, toEmailAddr, subject, body)
except requests.exceptions.Timeout as errt:
print ('Timeout Error:',errt)
attempts += 1
if attempts == 10:
body = 'Timeout Error: ' + str(errh)
subject = 'Failure'
sendEmailMessage(SMPTHOST, fromEmailAddr, toEmailAddr, subject, body)
except requests.exceptions.RequestException as err:
print ('Unidentified error: ',err)
attempts += 1
if attempts == 10:
body = 'Unidentified error: ' + str(errh)
subject = 'Failure'
sendEmailMessage(SMPTHOST, fromEmailAddr, toEmailAddr, subject, body)
How can I simplify above code?
In general, I would like to handle HTTP response error codes. I want to send an e-mail with specific error information only in case I get at least 10 error codes for the same call.
Since the action to perform is the same in each case, just group the exceptions into a single one, then customize the messages according to the error class / class name:
except (requests.exceptions.HTTPError,requests.exceptions.ConnectionError,requests.exceptions.RequestException,requests.exceptions.Timeout) as err:
error_message = "{}: ".format(err.__class__.__name__,err)
print (error_message)
attempts += 1
if attempts == 10:
body = error_message
subject = 'Failure'
sendEmailMessage(SMPTHOST, fromEmailAddr, toEmailAddr, subject, body)
If you need an indirection, just create a dictionary class name => string/action to perform/whatever.
Related
I've recently been trying to create a torrent client in python, and have just got the UDP announce protocol to work.
The tracker accepts my connect request just fine but only returns my IP and port as the peer list when I announce to it...
I've tried to look at the same torrents in other torrent clients and they have multiple working peers while my request only shows me my computer (I've tried this on many torrents, all return just my IP and port)
Here's the code for the sending function itself:
async def announce_udp(self, try_num = 1):
self.sock.settimeout(15)
answer = {}
inner_while = False
while try_num < 4:
while try_num < 4:
try:
print("trying to send")
sended = self.send(1, self.announce_payload())
print("sending the following packet: {0}".format(sended))
print(self.url)
inner_while = True
break
except Exception:
print("problem in sending")
try_num += 1
if not inner_while:
break
try:
answer = self.interpret(15)
break
except Exception:
print("problem in receiving")
try_num += 1
print("announce answer is: {0}".format(answer))
return answer
here's the code for the make payload function:
def announce_payload(self, downloaded = 0, left = 0, uploaded = 0, event = 0, key = get_transaction_id()):
payload = [self.torrent.get_torrent_info_hash_decoded(), get_peer_id().encode(), downloaded,
self.torrent.get_torrent_size(), uploaded, event, 0, key, -1, 6988]
p_tosend = None
try:
p_tosend = struct.pack('!20s20sqqqiIIiH', *payload)
except Exception as e:
print("there was an error: {0}".format(e))
return p_tosend
here's the code for the interpret + process function:
def interpret(self, timeout=10):
self.sock.settimeout(timeout)
print("got to interpret")
try:
response = self.sock.recv(10240)
print("answer recieved")
except socket.timeout:
print("no answer, try again")
raise TrackerResponseException("no answer", 0)
headers = response[:8]
payload = response[8:]
action, trans_id = struct.unpack('!ll', headers)
try:
trans = self.transactions[trans_id]
except KeyError:
raise TrackerResponseException("InvalidTransaction: id not found", trans_id)
try:
trans['response'] = self.process(action, payload, trans)
except Exception as e:
trans['response'] = None
print("error occured: {0}".format(e))
trans['completed'] = True
del self.transactions[trans_id]
#print(trans)
return trans
def process_announce(self, payload, trans):
response = {}
info = payload[:struct.calcsize("!lll")]
interval, leechers, seeders = struct.unpack("!lll", info)
print(interval, leechers, seeders, "noamsssssss")
peer_data = payload[struct.calcsize("!lll"):]
peer_size = struct.calcsize("!lH")
num_of_peers = int(len(peer_data) / peer_size)
print("the number of peers is: {0} and the peer data is: {1}".format(num_of_peers, peer_data))
print()
peers = []
for peer_offset in range(num_of_peers):
off = peer_size * peer_offset
peer = peer_data[off:off + peer_size]
addr, port = struct.unpack("!lH", peer)
peers.append({
'addr': socket.inet_ntoa(struct.pack('!L', addr)),
'port': port,
})
print(payload)
return dict(interval=interval, leechers=leechers, seeders=seeders, peers=peers)
I'm sorry if any of this is irrelevant, but I want to give you all of the code incase it tells you something.
(get_peer_id() returns a random peer id per the tracker protocol specification, and the get_transaction_id() returns random.randint(0, 1 << 32 - 1))
EDIT:
Alright, I've found the problem and now I'm feeling pretty dumb...
turns out even in the udp tracker whenever you send the info hash it has to be SHA1 encoded.
Hopefully this can help someone if they are stuck in the same problem :)
I'm using Python 3.4 on an RaspberryPi to read and upload data to Weather Underground. It works great most of the time, but occasionally either my internet connections is poor or Weather Underground servers are slow The other day I got this error:
socket.timeout: _ssl.c:584: The handshake operation timed out
I have try/except code, but it didn't match with any of the exceptions. I assumed the last "except:" would have caught the error, but I guess not. Should I just add "except socket.timeout:"?
try:
r = requests.get(full_URL, timeout=5) # send data to WU
# If uploaded successfully, website will reply with 200
if r.status_code == 200:
return(True)
else:
print('Upload Error: {} {}'.format(r.status_code, r.text))
return(False)
except requests.exceptions.ConnectionError:
print("Upload Error in upload2WU() - ConnectionError")
return(False)
except requests.exceptions.NewConnectionError:
print("Upload Error in upload2WU() - NewConnectionError")
return(False)
except requests.exceptions.MaxRetryError:
print("Upload Error in upload2WU() - MaxRetryError")
return(False)
except socket.gaierror:
print("Upload Error in upload2WU() - socket.gaierror")
return(False)
except:
print("Upload Error in upload2WU() - other")
return(False)
I do have two other places I'm using requests.get(), but they both use try: and except:
try:
response = requests.get(getUrl, timeout=5).json()
if len(response) > 1:
if isNumber(response['current_observation']['precip_today_in']):
daily_rain = float(response['current_observation']['precip_today_in'])
print('Suntec station daily rain={}'.format(daily_rain))
return(daily_rain)
return(ERR_INVALID_DATA)
except:
print("Error in WU_download.py getDailyRain() - failed get() request")
return(ERR_FAILED_GET)
Here's the other one:
try:
response = requests.get(getUrl, timeout=5).json()
if len(response) > 1: # valid response returns 3, if there's an error, the len() is 1
if isNumber(response['current_observation']['pressure_in']):
nearby_pressure = float(response['current_observation']['pressure_in'])
nearby_last_update_time = int(response['current_observation']['observation_epoch'])
if(nearby_pressure) > 25: # a pressure less than 25 inHg isn't gonna be valid
return(nearby_pressure)
# Didn't get a valid pressure. Try the next station in WU_STATIONS tuple
print("Couldn't get pressure data from {}".format(WU_STATIONS[i]))
nearby_pressure = ERR_INVALID_DATA
nearby_last_update_time = 0
i = i + 1
time.sleep(10)
except:
print("Error in WU_download.py getPressure(), failed get request for station {}".format(WU_STATIONS[i]))
i = i + 1
if (i >= len(WU_STATIONS)):
return(ERR_FAILED_GET)
I wrote a hiscore checker for a game that I play, basically you enter a list of usernames into the .txt file & it outputs the results in found.txt.
However if the page responds a 404 it throws an error instead of returning output as " 0 " & continuing with the list.
Example of script,
#!/usr/bin/python
import urllib2
def get_total(username):
try:
req = urllib2.Request('http://services.runescape.com/m=hiscore/index_lite.ws?player=' + username)
res = urllib2.urlopen(req).read()
parts = res.split(',')
return parts[1]
except urllib2.HTTPError, e:
if e.code == 404:
return "0"
except:
return "err"
filename = "check.txt"
accs = []
handler = open(filename)
for entry in handler.read().split('\n'):
if "No Displayname" not in entry:
accs.append(entry)
handler.close()
for account in accs:
display_name = account.split(':')[len(account.split(':')) - 1]
total = get_total(display_name)
if "err" not in total:
rStr = account + ' - ' + total
handler = open('tried.txt', 'a')
handler.write(rStr + '\n')
handler.close()
if total != "0" and total != "49":
handler = open('found.txt', 'a')
handler.write(rStr + '\n')
handler.close()
print rStr
else:
print "Error searching"
accs.append(account)
print "Done"
HTTPERROR exception that doesn't seem to be working,
except urllib2.HTTPError, e:
if e.code == 404:
return "0"
except:
return "err"
Error response shown below.
Now I understand the error shown doesn't seem to be related to a response of 404, however this only occurs with users that return a 404 response from the request, any other request works fine. So I can assume the issue is within the 404 response exception.
I believe the issue may lay in the fact that the 404 is a custom page which you get redirected too?
so the original page is " example.com/index.php " but the 404 is " example.com/error.php "?
Not sure how to fix.
For testing purposes, format to use is,
ID:USER:DISPLAY
which is placed into check.txt
It seems that total can end up being None. In that case you can't check that it has 'err' in it. To fix the crash, try changing that line to:
if total is not None and "err" not in total:
To be more specific, get_total is returning None, which means that either
parts[1] is None or
except urllib2.HTTPError, e: is executed but e.code is not 404.
In the latter case None is returned as the exception is caught but you're only dealing with the very specific 404 case and ignoring other cases.
I am lost here, I have an API that streams prices, I am trying to compare the second to last price with the last price, for instance, if x > y then do something. I cannot figure out how to compare the last to the second to the last price when the prices are streaming. Could someone please shed some light on how this may work? Thanks in advance!
my stream:
def stream_to_queue(self):
response = self.connect_to_stream()
if response.status_code != 200:
return
for line in response.iter_lines(1):
if line:
try:
msg = json.loads(line)
except Exception as e:
print "Caught exception when converting message into json\n" + str(e)
return
if msg.has_key("instrument") or msg.has_key("tick"):
price = msg["tick"]["ask"]
print price
This prints a price like 1.23004 and then continues to loop and print more prices. I have tried to save the current price in a variable outside the loop and then reference it when a new price comes in but it's not working..
my attempt:
def stream_to_queue(self):
response = self.connect_to_stream()
if response.status_code != 200:
return
oldLine = ''
for line in response.iter_lines(1):
if line:
try:
msg = json.loads(line)
except Exception as e:
print "Caught exception when converting message into json\n" + str(e)
return
if msg.has_key("instrument") or msg.has_key("tick"):
price = msg["tick"]["ask"]
oldLine = price
newLine = oldLine
if newLine > oldLine:
print newLine
Couple of things:
1- Your indentation is a bit off as the comparison should be done inside the 'for' loop. In your case, the comparison is only being made when the streaming is complete.
2- You are comparing oldLine with newLine which are equal, so nothing will happen. Instead you should compare newLine with price.
Consider the following code:
for line in response.iter_lines(1):
if line:
try:
msg = json.loads(line)
except Exception as e:
print "Caught exception when converting message into json\n" + str(e)
return
if msg.has_key("instrument") or msg.has_key("tick"):
price = msg["tick"]["ask"]
oldLine = price
newLine = oldLine
if newLine > price:
print newLine
I'm encountering this error message:
TypeError: add_header() takes exactly 3 arguments (2 given)
when using these parameters:
testService("SomeServiceName", "POST", "[redacted valid url]", ('Content-type','application/json'), [redacted valid json])
Normally this error means I'm not passing "self" as a parameter, but seeing as this method is not being called in a class, I'm not sure what to do. I've tried passing self in as a parameter in both the parameters and inside the method. And I've tried wrapping the header in brackets and parentheses. When I pass "self" I get the error message that self is undefined, and when I use the brackets instead of parentheses, I get the same error as above.
Anyone with magical Python debugging skills out there? Thanks so much for taking the time to check this out!
def testService(name, verb, url, header="", requestBody=""):
#Log out the name of the request we're testing
if (name is not None) or (name.strip() is not ""):
print "Checking " + name + "\n\n"
# Make URL with StoreNumber
if (url is not None) or (url is not ""):
testUrl = url
# If specified verb is GET
if verb.strip().upper() == "GET":
# Create request
req = urllib2.Request(testUrl)
print "Making request with URL: " + testUrl + "\n\n"
# Send request
try:
response = urllib2.urlopen(req)
# If service returns 200 Okay
print "Connection to " + name + " Service successful. Returned with code " + str(response.code) + "\n\n"
# Log response
print "Response: " + response.read() + "\n\n"
# Handle exceptions
# If HTTP Error
except HTTPError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print e.code
elif hasattr(e, 'message'):
print e.message
pass
# If URL was the problem
except URLError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
if str(e.reason) == "[Errno 11004] getaddrinfo failed":
print "[Errno 11004] getaddrinfo failed with bad url: " + testUrl + "\n\n"
else:
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'Error code: ', e.code
elif hasattr(e, 'message'):
print e.message
pass
# If specified verb was POST
elif verb.strip().upper() == "POST":
# Check for None requestBody
if (requestBody is not None) or (requestBody.strip() is not ""):
data = urllib.urlencode(requestBody)
# Create request
req = urllib2.Request(testUrl, data)
# Check for header
if (header is not None) or (header.strip() is not ""):
req.add_header(header)
# YO YO THE BELOW CODE IS INCOMPLETE PLEASE FINISH
# Log request with URL and Data
print "Making request with URL: " + testUrl + " and data: THIS PART IS UNFINISHED PLEASE FINISH ME \n\n"
try:
response = urllib2.urlopen(req)
# If service returns 200 Okay
print "Connection to " + name + " Service successful. Returned with code " + str(response.code) + "\n\n"
# Log response
print "Response: " + response.read() + "\n\n"
# Handle exceptions
# If HTTP Error
except HTTPError as e:
if hasattr(e, 'code'):
print e.code
elif hasattr(e, 'message'):
print e.message
elif hasattr(e, 'reason'):
print name + ' failed to reach a server.'
print 'Reason: ', e.reason
pass
except URLError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
if str(e.reason) == "[Errno 11004] getaddrinfo failed":
print "[Errno 11004] getaddrinfo failed with bad url: " + url + "\n\n"
else:
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'Error code: ', e.code
elif hasattr(e, 'message'):
print e.message
pass
# Header non-existent in testService call
else:
print "Service header not provided. Exiting program"
sys.exit()
# Requesty Body not present in testService call
else:
print "Service request body not provided in code. Exiting program"
sys.exit()
# If specified verb is not supported (Currently only GET and POST are supported)
else:
print name + " Service written with HTTP verb other than GET or POST. Exiting program"
sys.exit()
else:
print "Service url not provided in code. Exiting program"
sys.exit()
else:
print "Service name not provided in code. Exiting program"
sys.exit()
From the documentation, add_header takes two arguments. You are calling it with one argument, a tuple with two values.
What you should do:
req.add_header(key, value)
What you are currently doing because you are getting the header as a tuple:
req.add_header((key, value,)) # aka passing a tuple with both arguments to the key parameter
You need to unpack the tuple:
req.add_header(header[0], header[1])
Or even better, using the splat operator (*):
req.add_header(*header) # Does the same thing as above
Also, you are using an empty string as the default argument for header, when when it is supplied it is a tuple. You should probably change the default value to a tuple or None.
Your header is a 2-tuple:
('Content-Type', 'application/json')
You're trying to do this:
req.add_header('Content-Type', 'application/json')
But in reality you're doing this:
req.add_header(('Content-Type', 'application/json'))
Notice that you're only passing one argument - a tuple - instead of two, a key and a value.
To fix, unpack your header when you pass it with the * (informally, 'splat') operator:
req.add_header(*header)
Take a look at the documentation: http://docs.python.org/2/library/urllib2.html#urllib2.Request.add_header.
While the function expects a key and a value, you're passing only a single object. Since you're calling this on the req object, that is the implicit "self" that's being passed as well.
You could call this function in two ways:
req.add_header(key, value)
urllib2.Request.add_header(req, key, value) # explicitly passing the reference instead of self
I'm not sure whether you're expecting the string you pass to be treated as the key or the value, but adding another paramater (or making the header parameter take a dict and then splitting appropriately in a for loop) should solve the issue. For example (with irrelevant code removed):
def testService(name, verb, url, header=None, requestBody=""):
if header is None:
header = {}
for key, value in header.iteritems():
req.add_header(key, value)