We Keep Coding

python UnboundLocalError referenced before assignment

I see that this seems to be a common error, but I'm not seeing the answer for my case.
UnboundLocalError: local variable 'tfstate_dict' referenced before assignment
#!/usr/bin/env python
import json
def main():
get_sfleet_id()
def get_sfleet_id():
try:
f=open("terraform_remote.tfstate", "r")
contents =f.read()
tfstate_dict = json.load(contents)
except:
print("error loading %s" % f)
print(contents)
print(tfstate_dict)
if __name__ == '__main__': main()

As I wrote in my comment, tfstate_dict does not come in to existence until you exit your try block. But that's not to say that it applies to all the preceding code; it simply applies to tfstate_dict because it happens to be the very last line.
This is easily testable with the following:
try:
a = int(2)
b = int(3)
c = int('hi')
except:
print(locals())
print()
print(locals().get('a'))
You should see that 'a' and 'b' are both defined and can be accessed (depending on how you're running this code, there could a lot of stuff in locals() too). So, the existence of 'a' and 'b' gives you no assurance that 'c' exists.
There's two issues with your current exception handling:
There's probably too much going on in your try block to be handled the way you currently do. This code will fail if the file cannot be located, but you wouldn't necessarily know that was happening. And if your code originally failed only on tfstate_dict = json.load(contents) you're now scratching your head why you're getting a NameError on print(contents) all of a sudden.
You don't want to be catching these issues with blanket except. At a minimum you'll want to use except Exception as e:, which allows you to print e too.
Here's a hypothetical situation where you handle the file not existing, and you also give a shot at parsing JSON.
import json
from json.decoder import JSONDecodeError
try:
with open('something.json') as infile:
try:
#data = json.load(infile) # This is what you'd really use
data = json.loads("{hi: 2}") # But let's make it fail
except JSONDecodeError:
print("Not valid JSON, try something else")
data = infile.read()
except FileNotFoundError:
print("Can't find file")
data = ''

how to properly use try/except/with inside functions and main

I am a relative python newbie and I am getting confused with how to properly handle exceptions. Apologies for the dumb question.
In my main() I iterate through a list of dates and for each date I call a function, which downloads a csv file from a public web server. I want to properly catch exceptions for obvious reasons but especially because I do not know when the files of interest will be available for download. My program will execute as part of a cron job and will attempt to download these files every 3 hours if available.
What I want is to download the first file in the list of dates and if that results in a 404 then the program shouldn't proceed to the next file because the assumption is if the oldest date in the list is not available then none of the others that come after it will be available either.
I have the following python pseudo code. I have try/except blocks inside the function that attempts to download the files but if an exception occurred inside the function how do I properly handle it in the main() so I can make decisions whether to proceed to the next date or not. The reason why I created a function to perform the download is because I want to re-use that code later on in the same main() block for other file types.
def main():
...
...
# datelist is a list of date objects
for date in datelist:
download_file(date)
def download_file(date):
date_string = str(date.year) + str(date.strftime('%m')) + str(date.strftime('%d'))
request = HTTP_WEB_PREFIX+ date_string + FILE_SUFFIX
try:
response = urllib2.urlopen(request)
except urllib2.HTTPError, e:
print "HTTPError = " + str(e)
except urllib2.URLError, e:
print "URLError = " + str(e)
except httplib.HTTPException, e:
print "HTTPException = " + str(e)
except IOError:
print "IOError = " + str(e)
except Exception:
import traceback
print "Generic exception: " + traceback.format_exc()
else:
print "No problem downloading %s - continue..." % (response)
try:
with open(TMP_DOWNLOAD_DIRECTORY + response, 'wb') as f:
except IOError:
print "IOError = " + str(e)
else:
f.write(response.read())
f.close()

The key concept here is, if you can fix the problem, you should trap the exception; if you can't, it's the caller's problem to deal with. In this case, the downloader can't fix things if the file isn't there, so it should bubble up its exceptions to the caller; the caller should know to stop the loop if there's an exception.
So let's move all the exception handling out of the function into the loop, and fix it so it craps out if there's a failure downloading the file, as the spec requires:
for date in datelist:
date_string = str(date.year) +
str(date.strftime('%m')) +
str(date.strftime('%d'))
try:
download_file(date_string)
except:
e = sys.exc_info()[0]
print ( "Error downloading for date %s: %s" % (date_string, e) )
break
download_file should now, unless you want to put in retries or something like that, simply not trap the exceptions at all. Since you've decoded the date as you like in the caller, that code can come out of download_file as well, giving the much simpler
def download_file(date_string):
request = HTTP_WEB_PREFIX + date_string + FILE_SUFFIX
response = urllib2.urlopen(request)
print "No problem downloading %s - continue..." % (response)
with open(TMP_DOWNLOAD_DIRECTORY + response, 'wb') as f:
f.write(response.read())
f.close()
I would suggest that the print statement is superfluous, but that if you really want it, using logger is a more flexible way forward, as that will allow you to turn it on or off as you prefer later by changing a config file instead of the code.

From my understanding of your question... you should just insert code into the except blocks that you want to execute when you encounter the particular exception. You don't have to print out the encountered error, you can do whatever you feel is necessary when it is raised... provide a popup box with information/options or otherwise direct your program to the next step. Your else section should isolate that portion, so it will only execute if none of your exceptions are raised.

Safest way to open a file in python 3.4

I was expecting the following would work but PyDev is returning an error:
try fh = open(myFile):
logging.info("success")
except Exception as e:
logging.critical("failed because:")
logging.critical(e)
gives
Encountered "fh" at line 237, column 5. Was expecting: ":"...
I've looked around and cannot find a safe way to open a filehandle for reading in Python 3.4 and report errors properly. Can someone point me in the correct direction please?

You misplaced the :; it comes directly after try; it is better to put that on its own, separate line:
try:
fh = open(myFile)
logging.info("success")
except Exception as e:
logging.critical("failed because:")
logging.critical(e)
You placed the : after the open() call instead.
Instead of passing in e as a separate argument, you can tell logging to pick up the exception automatically:
try:
fh = open(myFile)
logging.info("success")
except Exception:
logging.critical("failed because:", exc_info=True)
and a full traceback will be included in the log. This is what the logging.exception() function does; it'll call logging.error() with exc_info set to true, producing a message at log level ERROR plus a traceback.

Python: Reverse sentence and write it in another file

I'm trying to reverse the sentence and write it in another file but when I run it, it creates an empty reversed.txt file.
The following two lines are in file input:
Hello World
How is everyone doing?
The file output will have the lines:
dlroW olleH
?gniod enoyreve si woH
My code:
#!/usr/local/bin/python
name = raw_input("Enter name of file: ")
def Reverse(name):
try:
input_file=open(name,"r")
reversed = "reversed.txt"
output_file=open(reversed,"w")
list=input_file.readlines()
for i in range(0,len(list)):
d = int(len(list) - i)
output_file.write(list[d])
except IOError:
print("Cannot open file")
except:
print("Other errors")
else:
print("success")
finally:
print("prints always")
input_file.close()
output_file.close()
#returns reversed
Reverse(name)
When I run this it prints "Other errors". It creates the new file reversed.txt but it`s empty.

There are a couple of things wrong with your code snippet. But this should work better:
#!/usr/local/bin/python
name = raw_input("Enter name of file: ")
def Reverse(name):
try:
input_file=open(name,"r")
reversed = "reversed.txt"
output_file=open(reversed,"w")
for line in input_file.readlines():
output_file.write(line[-2::-1] + "\n")
except IOError:
print("Cannot open file")
except:
print("Other errors")
else:
print("success")
finally:
print("prints always")
input_file.close()
output_file.close()
#returns reversed
Reverse(name)
Here we iterate over the read lines and then use the fact that strings can be viewed as arrays to reverse each line.
EDIT
By using line[-2::-1] we avoid having the ending new line at the start of the line we write to the file. Then we append the "\n" so that each line is properly separated by the new line character.

You can try something like this:
lines=input_file.readlines()
for line in lines:
line = ''.join(list(reversed(line)))
output_file.write(line)

If I were to tell you what's wrong, without improving your implementation (which is definitely possible ;) ) I'd like to suggest a few things:
Indents are super important:
In your code just after the def you have a blank line. This means your function ends before it starts, and your try excepts are just 'floating'.
Your excepts aren't very helpful: You just print small information when catching exceptions. You have no idea why the exception is thrown. A better way to do this is print the exception with your own custom message. Something like:
try:
someCode()
except IOError as ioE:
print "Can't open file! - ", ioE # Change to print() for py3.x
except Exception as otherE:
print "Unknown error! - ", otherE
Initialize variables you intend to use: If you are using a variable, define it! Make sure any flow of loops/try-excepts will end up initializing your vairables. When in doubt, initialize as None (and check for None when using it). In your code, input_file is being used in finally. But if the input_file=open(name,"r") raises an exception, input_file will remain uninitialized causes another exception in finally.
Other than that your implementation can be improved from the other answers.
Hope this helps!

Searching for equivalent of FileNotFoundError in Python 2

I created a class named Options. It works fine but not not with Python 2.
And I want it to work on both Python 2 and 3.
The problem is identified: FileNotFoundError doesn t exist in Python 2.
But if I use IOError it doesn t work in Python 3
Changed in version 3.3: EnvironmentError, IOError, WindowsError, VMSError, socket.error, select.error and mmap.error have been merged into OSError.
What should I do ???(Please do not discuss my choice of portability, I have reasons.)
Here s the code:
#!/usr/bin/python
#-*-coding:utf-8*
#option_controller.py
#Walle Cyril
#25/01/2014
import json
import os
class Options():
"""Options is a class designed to read, add and change informations in a JSON file with a dictionnary in it.
The entire object works even if the file is missing since it re-creates it.
If present it must respect the JSON format: e.g. keys must be strings and so on.
If something corrupted the file, just destroy the file or call read_file method to remake it."""
def __init__(self,directory_name="Cache",file_name="options.json",imported_default_values=None):
#json file
self.option_file_path=os.path.join(directory_name,file_name)
self.directory_name=directory_name
self.file_name=file_name
#self.parameters_json_file={'sort_keys':True, 'indent':4, 'separators':(',',':')}
#the default data
if imported_default_values is None:
DEFAULT_INDENT = 2
self.default_values={\
"translate_html_level": 1,\
"indent_size":DEFAULT_INDENT,\
"document_title":"Titre"}
else:
self.default_values=imported_default_values
def read_file(self,read_this_key_only=False):
"""returns the value for the given key or a dictionary if the key is not given.
returns None if it s impossible"""
try:
text_in_file=open(self.option_file_path,'r').read()
except FileNotFoundError:#not 2.X compatible
text_in_file=""#if the file is not there we re-make one with default values
if text_in_file=="":#same if the file is empty
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
try:
option_dict=json.loads(text_in_file)
except ValueError:
#if the json file is broken we re-make one with default values
self.__insert_all_default_values()
text_in_file=open(self.option_file_path,'r').read()
option_dict=json.loads(text_in_file)
if read_this_key_only:
if read_this_key_only in option_dict:
return option_dict[read_this_key_only]#
else:
#if the value is not there it should be written for the next time
if read_this_key_only in self.default_values:
self.add_option_to_file(read_this_key_only,self.default_values[read_this_key_only])
return self.default_values[read_this_key_only]
else:
#impossible because there is not default value so the value isn t meant to be here
return None
else:
return option_dict
def add_option_to_file(self,key,value):#or update
"""Adds or updates an option(key and value) to the json file if the option exists in the default_values of the object."""
option_dict=self.read_file()
if key in self.default_values:
option_dict[key]=value
open(self.option_file_path,'w').write(\
json.dumps(option_dict,sort_keys=True, indent=4, separators=(',',':')))
def __insert_all_default_values(self):
"""Recreate json file with default values.
called if the document is empty or non-existing or corrupted."""
try:
open(self.option_file_path,'w').write(\
json.dumps(self.default_values,sort_keys=True, indent=4, separators=(',',':')))
except FileNotFoundError:
os.mkdir(self.directory_name)#Create the directory
if os.path.isdir(self.directory_name):#succes
self.__insert_all_default_values()
else:
print("Impossible to write in %s and file %s not found" % (os.getcwd(),self.option_file_path))
#demo
if __name__ == '__main__':
option_file_object=Options()
print(option_file_object.__doc__)
print(option_file_object.read_file())
option_file_object.add_option_to_file("","test")#this should have no effect
option_file_object.add_option_to_file("translate_html_level","0")#this should have an effect
print("value of translate_html_level:",option_file_object.read_file("translate_html_level"))
print(option_file_object.read_file())

If FileNotFoundError isn't there, define it:
try:
FileNotFoundError
except NameError:
FileNotFoundError = IOError
Now you can catch FileNotFoundError in Python 2 since it's really IOError.
Be careful though, IOError has other meanings. In particular, any message should probably say "file could not be read" rather than "file not found."

You can use the base class exception EnvironmentError and use the 'errno' attribute to figure out which exception was raised:
from __future__ import print_function
import os
import errno
try:
open('no file of this name') # generate 'file not found error'
except EnvironmentError as e: # OSError or IOError...
print(os.strerror(e.errno))
Or just use IOError in the same way:
try:
open('/Users/test/Documents/test') # will be a permission error
except IOError as e:
print(os.strerror(e.errno))
That works on Python 2 or Python 3.
Be careful not to compare against number values directly, because they can be different on different platforms. Instead, use the named constants in Python's standard library errno module which will use the correct values for the run-time platform.

The Python 2 / 3 compatible way to except a FileNotFoundError is this:
import errno
try:
with open('some_file_that_does_not_exist', 'r'):
pass
except EnvironmentError as e:
if e.errno != errno.ENOENT:
raise
Other answers are close, but don't re-raise if the error number doesn't match.
Using IOError is fine for most cases, but for some reason os.listdir() and friends raise OSError instead on Python 2. Since IOError inherits from OSError it's fine to just always catch OSError and check the error number.
Edit: The previous sentence is only true on Python 3. To be cross compatible, instead catch EnvironmentError and check the error number.

For what it's worth, although the IOError is hardly mentioned in Python 3's official document and does not even showed up in its official Exception hierarchy, it is still there, and it is the parent class of FileNotFoundError in Python 3. See python3 -c "print(isinstance(FileNotFoundError(), IOError))" giving you a True. Therefore, you can technically write your code in this way, which works for both Python 2 and Python 3.
try:
content = open("somefile.txt").read()
except IOError: # Works in both Python 2 & 3
print("Oops, we can not read this file")
It might be "good enough" in many cases. Although in general, it is not recommended to rely on an undocumented behavior. So, I'm not really suggesting this approach. I personally use Kindall's answer.