We Keep Coding

How to Catch Exception in for loop and return exception only after loop is done?

Hi I have a function that parses the first 60 lines in a file and is supposed to alert a user if there are lines that are entirely whitespace. However these can occur anywhere in those 60 lines so I want the script to parse the entire 60 lines mainly because I need data from a couple of the lines for my error reporting. And we might want to know where those errors occur in the future. I wrote this:
def header_data(data):
dictionary = {}
datalen = len(data)
itrcntr = 0
try:
for line in data:
itrcntr += 1
if line.isspace():
raise Exception('File has badly formatted header data line(s)')
else:
linesplit = line.rstrip().split(":")
if len(linesplit) > 1:
dictionary[linesplit[0]] = linesplit[1].strip()
return dictionary
except Exception as e:
errmsg = str(e)
if itrcntr == datalen:
return (dictionary, errmsg)
else:
pass
With this function, I'd expect if it sees that itrcntr is not equal datalen, it would pass and go back to the try block and move on to the next line. But this doesn't happen. INstead it breaks out of the function and continues in the next line in the function caller. How can I make it continue looping till it reaches to the end of the loop in which it will then return the dictionary along with the error message? Or can this not be done with try catch exception handlers?

Unless you want to catch exceptions other than the situation of line.isspace, I wouldn't use a try block at all. Just collect your errors in a list, eg:
errors = []
for line in data:
itrcntr += 1
if line.isspace():
errors.append('File has badly formatted header data at line %d.' % itrcntr)
# then at the end:
if errors:
# do something about it...

If any exception occurred, try clause will be skipped and except clause
will run.
If you raise an exception anywhere in the Try, everything else will be skipped. So if you want the loop to continue, then just don't use Try Except.
Just collect every error message and then return it.

In Python, will raising an Exception inside of a while loop lead to memory leaks?

I have the following code snippet that got me thinking
# Utlizing api_sess from the /token response
# api_sess is a requests.Session() with Authorization header set with session token
products_resp = api_sess.get(
f'{API_BASE}/products',
params = {'type': 'Item'}
)
if products_resp.ok:
while True:
current_page = products_resp.headers.get('X-Current-Page', 1)
total_pages = products_resp.headers.get('X-Total-Pages', 0)
products_json = products_resp.json()
for product in products_json:
print(product['groupCode'], product['sku'], product['productStatus'])
# Handle pagination
if current_page < total_pages:
products_resp = api_sess.get(
f'{API_BASE}/products',
params = {'type': 'Item', 'page': current_page + 1}
)
if not products_resp.ok:
# Break out of while loop before raising error
break
else:
break
# Raise exception outside of while loop to prevent memory leak
if not products_resp.ok:
raise ValueException(f'Request Error {products_resp.status_code}')
else:
raise ValueException(f'Request Error {products_resp.status_code}')
Originally I was going to raise the exception inside of the while where I check the response of the next page request like so:
if current_page < total_pages:
products_resp = api_sess.get(
f'{API_BASE}/products',
params = {'type': 'Item', 'page': current_page + 1}
)
if not products_resp.ok:
raise ValueException(f'Request Error {products_resp.status_code}')
else:
break
At first I didn't think anything of it, but I started to wonder if resource cleanpup would take place since the error is raised inside an infinite while loop due to the True condition. Thus, why I inevitably reworked my logic to break out of the while loop first, then recheck response status.
In my opinion, this cautious method isn't as elegant as I'd like seeing as I'm duplicating the checking of the response status. Which leads me to my question:
Is it safe to keep the Exception raising inside the while loop or will it lead to memory leaks?
I know the question and scenario might seem trivial, but I'd like to stick with best practices in all aspects whenever possible. And upon my googling (while I didn't spend hours atleast) of the matter did not turn out any applicable explanations of how this scenario is handled.

Whether you leave the current scope via return or an uncaught exception, the reference count on the object will be decreased. A simple example:
import sys
a = "hello"
def foo(b):
x = a
print(f'x: {sys.getrefcount(x)}')
if b:
return
else:
raise Exception()
print(f'a: {sys.getrefcount(a)}')
foo(True)
print(f'a: {sys.getrefcount(a)}')
try:
print(f'a: {sys.getrefcount(a)}')
foo(False)
except Exception:
pass
print(f'a: {sys.getrefcount(a)}')
The exact output may vary, but you should observe the reference count for x being one greater than the reference count for a, which remains constant before and after the calls to foo.
In both cases, the reference count for the global a increases when foo is called, and is decreased after foo exits (or after the try statement which catches the exception completes). If the exception is never caught, the interpreter
exits anyway, making the question of a memory leak moot.

How to skip the Index

#main program
while True:
ReadValue = Func03Modbus(1,70,40);#slave,start,number of registers
x3 = struct.pack('>HH',abs(ReadValue[3]),abs(ReadValue[2]))
pressure = struct.unpack('>f', x3)
print pressure[0]
c3 = struct.pack('>HH',abs(ReadValue[5]),abs(ReadValue[4]))
purity = struct.unpack('>f', c3)
print purity[0]
hrs = int(ReadValue[30])
mins= int(ReadValue[31])
timein =float(str(ReadValue[30])+"."+str(ReadValue[31]))
print timein
r=requests.get("http://api.thingspeak.com/update api_key=5RMT************&field4="+str(pressure[0])+"&field5="+str(purity[0])+"&field1="+str(ReadValue[i])+"&field2="+str(mins)+"&field3="+str(timein)))
print str(ReadValue[30])
time.sleep(15)
While running the above program it stops running with returning following error:
Traceback (most recent call last): File "/home/pi/v1.py", line 123,
in
x3 = struct.pack('>HH',abs(ReadValue[3]),abs(ReadValue[2])); IndexError: tuple index out of range
I want my program to run continuously even when it returns error. I want to skip the error and to run the program continuously. How can I do that ?

In theory you could wrap the code in an exception handler like:
while True:
try:
what you want to do
except Exception as e:
print("Something bad happened:", e)
finally:
# reset device here
time.sleep(15)
But this seems like a really bad idea if you're interacting with hardware, since you can't be sure what state you're leaving it in. Ideally, you'd want to make sure you're doing a proper reset of the device (or reconnect? depends what you're talking to) on every cycle.
Alternatively, if you want to explicitly verify that the values you get back are available, you can do:
ReadValue = Func03Modbus(1,70,40);#slave,start,number of registers
if len(ReadValue) < 32:
print("Got incomplete result")
time.sleep(15)
continue
The language reference/tutorial has more information about handling errors here: https://docs.python.org/3/tutorial/errors.html

In order to continue in the event of this kind of error, simply place the part you wish to ignore exceptions in within an appropriate try: ... except ...
while True:
try:
<body of work>
except IndexError:
<you might want to log the error>
pass
In this case, we continue only in the event of IndexError.

python: function occasionally returns Nonetype when returning one of its paramters?

I'm writing a scrolling list window widget in pygame (I know, its stupid, I admit it. But I had fun doing it) Anyway this function is threaded by an file-viewer object which passes an instance of itself along with an absolute path, addr, to display the contents of.
it starts by collecting the names of the files, and storing them in a 2 member array with the name of the file as the first instance and a formatted message to display on screen as the second member:
example: ["videos","[34]: D -- videos"]
the functions busy waits for the user to click on one of the tiles as his/her input: obj.waitingForReply. obj.Answer stores the tuple of the panel the user clicked on. If it stores a directory, the function is supposed to call itself to view the contents of the directory, if its a file, the function exits and the thread ends by setting obj.Quit to true. This also kills the instance of fileviewer, as its purpose is to provide a GUI for the user to pick a file to read into the program. It immediately crashes whenever I go into another directory, but then choose to leave and return to the previous parent directory by choosing: Back: [1]: D -- Desktop As displayed when I choose another directory to view.
I'm looking into using a queue for the semaphores but I don't think that is my problem
code:
def listdirectory(obj,addr,prev=False,type="D",error=0):
var=0
choose = 1
yesnext=False
yesprev=False
next=0
print "in listdirectory"
while(obj.Quit==False):
mylist = []
if(prev!=False):
yesprev=True
what = "F"
if(os.path.isdir(prev)):
what="D"
m = re.split("\/",prev)
mylist.append([prev, format1(choose,what,m[(len(m)-1)],1)])
choose+=1
if(yesnext):
what = "F"
if(os.path.isdir(next)):
what="D"
m = re.split("\/",next)
mylist.append([next, format1(choose,what,m[(len(m)-1)],0)])
choose+=1
for items in os.listdir(addr):
what = "F"
check = concat(concat(addr,"/"),items)
if(os.path.isdir(check)):
what="D"
m = re.split("\/",items)
mylist.append([items, format2(choose,what,m[(len(m)-1)])])
choose+=1
print "listdirectory, currDrawing spinlock"
while obj.currDrawing:
var = (var+1)%2
obj.currDrawing=True
obj.loaddir(mylist)
obj.currDrawing=False
obj.waitingForReply=True
while obj.waitingForReply:
var = (var+1)%2
if(obj.Answer!=False):
chkprev = re.split("\/",addr)
apath = concat(concat(addr,"/"),obj.Answer[0])
if(obj.Answer[0]==chkprev[(len(chkprev)-2)]):
return addr
elif(os.path.isdir(apath)):
yesnext=True
next = listdirectory(obj,apath,addr)
print "value of next: %r" % next
next = re.split("\/",next)
next = next[(len(next)-1)]
else:
obj.Quit=True
choose=1
when it crashes:
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py", line 486, in __bootstrap_inner
self.run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/threading.py", line 446, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/Jason/Desktop/cup-robot-project/cup-robot-3/src/fileopener.py", line 524, in listdirectory
next = re.split("\/",next)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/re.py", line 165, in split
return _compile(pattern, 0).split(string, maxsplit)
TypeError: expected string or buffer
Just to clarify: The return addr is for use of the calling previous listdirectory function. When it exits the function isn't supposed to return anything, as the desired variable is stored in self.Answer which gets accessed through one of the objects member functions. the entire function is technically a daemon: while self.Quit==False IDK if that changes anything

The only return statement in the function is:
if(obj.Answer[0]==chkprev[(len(chkprev)-2)]):
return addr
If this condition is not true, and the following statement os.path.isdir(apath) is not True, it will reach the line obj.Quit=True, leave the for loop, then reach the end of the function, where there is no return. At that point it will return None.

WebElement is being returned as NoneType after calling driver.refresh()

I'm writing a script in Python to do a smoke test on a social network, that implements a post feed.
I wrote a method that looks for the topmost post, and returns it (it's the class "media"). You'll see that there are some time.sleep() and refresh() calls, and that's because the server we use is horrible, to say the least, and the loading fails very often, only rendering partial content, making a refresh necessary.
Here's where the problem is: When, and only when the br.refresh() is called, the object returned is NoneType. If the page loads correctly, and the refresh() is not called, the object returned is correct.
Does anyone have any idea why this might happen? I tried implementing the method without the use of exceptions (in case this is what broke the return, somehow) without any success.
PS: Curiously enough, if instead of waiting for the br.refresh() to be called, I manually go and press the Refresh button on the "driven" browser, the object is returned perfectly.
Here's the code:
def getLastPost (br, count = 0):
try:
elapsed = 0
while(br.find_elements_by_class_name("media") == []) and elapsed < 15:
if elapsed % 5 == 0:
log("Waiting...","w")
time.sleep(0.5)
elapsed += 0.5
if(br.find_elements_by_class_name("media") == []):
raise NoSuchElementException
return br.find_elements_by_class_name("media")[0]
except NoSuchElementException:
if(count >= 5):
raise Exception("Element not found after 5 page reloads.")
log("Element not loaded! Retrying.","w")
count += 1
br.refresh()
time.sleep(count) # Wait a bit more each time.
getLastPost(br, count)
And the error that gives when trying to read the returned object:
Traceback (most recent call last):
File "Smoke.py", line 37, in <module>
assert ("MESSAGE") in getLastPost(br).text
AttributeError: 'NoneType' object has no attribute 'text'

refresh() reloads the browser page. After that, all WebElements are invalid and you must locate them again.
Background: Selenium / WebDriver doesn't remember how you got the element, they ask the browser for a unique internal ID of the element and when you, say, click on an element, they sand a message to the browser "click on 34987563424563.34675".
A reload invalidates all internal IDs.
The reason why you get None in the assert is because there is no return statement in the except clause (last line). Without an explicit return, all Python functions return None. Try return getLastPost(br, count)