Why catching an exception close the open file, with and without context? - python

I was working on an issue from a small script I have written. It's a hook between two application. The issue is one application have been updated and now it use yaml instead of json as config file.
Minimal exemple
import os
import yaml
import json
config = {
'version': "2.0.2",
'journals': {
"default": "/test/yaml/bhla"
},
'editor': os.getenv('VISUAL') or os.getenv('EDITOR') or "",
'encrypt': False,
'template': False,
'default_hour': 9,
'default_minute': 0,
'timeformat': "%Y-%m-%d %H:%M",
'tagsymbols': '#',
'highlight': True,
'linewrap': 79,
'indent_character': '|',
}
with open("jrnl.yaml", 'w') as f:
yaml.safe_dump(config, f, encoding='utf-8', allow_unicode=True, default_flow_style=False)
This will create a yaml file where you will have run the code.
The problem
I first wrote this simple patch to allow my hook to work with both (json and yaml).
JRNL_CONFIG_PATH = "jrnl.yaml"
with open(JRNL_CONFIG_PATH, "r") as f:
try:
JRNL_CONFIG = json.load(f)
except json.JSONDecodeError:
JRNL_CONFIG = yaml.load(f, Loader=yaml.FullLoader)
TAGS_SYMBOL = JRNL_CONFIG.get("tagsymbols", "#")
But as a big suprise, when error is catched, f is closed because JRNL_CONFIG will return None and produce this error :
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-141-bc1ef847563b> in <module>()
----> 1 JRNL_CONFIG.get("tagsymbols", "#")
AttributeError: 'NoneType' object has no attribute 'get'
Questions
Why catching an exception close the open file, with and without context?
What is the best method to catch the json error and still be able to parse the file as yaml ?
Tested
Can't pass the file by name, as the config file may not have extension (.json, .yaml)
This work but it's far from elegant ...
try:
f = open(JRNL_CONFIG_PATH, "r")
JRNL_CONFIG = json.load(f)
except json.JSONDecodeError:
f = open(JRNL_CONFIG_PATH, "r")
JRNL_CONFIG = yaml.load(f, Loader=yaml.FullLoader)
finally:
f.close()
Edit 1
Question 1 : Why catching an exception close the open file, with and without context?
Have been awser by #jedwards
Question 2 : What is the best method to catch the json error and still be able to parse the file as yaml ?
Have been awsered by #chepner

The issue isn't that the file is being closed (it's not), it's that the file pointer is no longer at the expected place (the beginning of the file) when you try to use your fallback:
with open("some.yaml") as f:
try:
print("before", f.tell())
data = json.load(f)
except json.JSONDecodeError:
print("after", f.tell())
print("is closed:", f.closed)
Here, the .tell() method returns the location of the file pointer.
One solution would be to reset the file pointer inside the except block:
with open("some.yaml") as f:
try:
JRNL_CONFIG = json.load(f)
except json.JSONDecodeError:
f.seek(0)
JRNL_CONFIG = yaml.load(f, Loader=yaml.FullLoader)

What about:
with open(JRNL_CONFIG_PATH, "r") as f:
data = f.read()
try:
JRNL_CONFIG = json.loads(data)
except json.JSONDecodeError:
JRNL_CONFIG = yaml.load(data, Loader=yaml.FullLoader)
TAGS_SYMBOL = JRNL_CONFIG.get("tagsymbols", "#")

There's no need to try both json.load and yaml.load, because YAML is a superset of JSON and yaml.load will parse anything json.load can.
JRNL_CONFIG_PATH = "jrnl.json"
with open(JRNL_CONFIG_PATH, "r") as f:
JRNL_CONFIG = yaml.load(f, Loader=yaml.FullLoader)
TAGS_SYMBOL = JRNL_CONFIG.get("tagsymbols", "#")

Related

Do not exit if the file is not found python3

Hi I have a list of filenames
When reading a file, it comes out with an error
Traceback (most recent call last):
File "test.py", line 229, in <module>
f = open(i, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'directory_home.json'
How can I skip the error and complete the rest of the files?
code
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)
Something like this. Note that 'pass' goes to next iteration. 'Continue' would repeat iteration (in this case useless). 'Break' would end iteration.
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
try:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)
f.colse()
except FileNotFoundError:
print("File does not exist")
pass
Two options.
Option 1: os.path.isfile
import os
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
if os.path.isfile(file):
f = open(i)
data = json.load(f)
for i in data['name']:
print(i)
else:
continue # or whatever you want to do.
Option 2: Exception handling
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
try:
f = open(i, 'r')
except FileNotFoundError:
continue # or whatever
data = json.load(f)
for i in data['name']:
print(i)
You should pur your code in a try-except statement, like this
try:
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)
except FileNotFoundError as e:
# log exception and do whatever you want
pass
In this way, if your code raises an exception because the file si not present, you are able ti handle It.
Please remember that in the code above, just the exception due to a not existing file si catch. This means that if another exception occurs the code will raise a traceback

save error compile message in txt file in python3

Could you help me that when I compile my program and if I have an error how can I redirect this error message in text file, before compilation terminate?
I wrote this code, But my problem is i want when i have an error after that my COMP_ERR.txt file create and write error inside this file.
but in my code this create before
import urllib.request
import os
import tempfile
import sys
import fileinput
original_stderr = sys.stderr
f_error = open("COMP_ERR.txt", "w")
sys.stderr = f_error
try:
url = sys.argv[1]
path_snip_file = sys.argv[2]
#url = 'http://pages.di.unipi.it/corradini/Didattica/AP-17/PROG-ASS/03/ClassWithTest.java'
#path_snip_file = "C:/Users/user/AppData/Local/Programs/Python/Python36/snip1.java"
path_remote_file_inOurComp = "C:/Users/user/AppData/Local/Programs/Python/Python36/ClassWithTest.java"
path_remote_file_inOurCom = "C:/Users/user/AppData/Local/Programs/Python/Python36/ClassWithTest1.java"
gt_url = urllib.request.urlretrieve(url)
print("the URL is: ")
print(gt_url)
link = input("input the path of file: ")
"""
f = open(link, "r")
for line in f:
print(line, end='')
f.close()
"""
#--------------------]
#copy snipper java file inside remote file
#[--------------------
with open(path_remote_file_inOurComp, 'w') as modify_file:
with open (link, 'r') as r_file:
for line in r_file:
if " public static void main(" not in line:
modify_file.write(line)
else:
with open(path_snip_file, 'r') as snip_file:
for lines in snip_file:
modify_file.write(lines)
modify_file.write('\n'+line)
#-------------------------------------------------------------
#refactor
#-------------------------------------------------------------
with open(path_remote_file_inOurCom, 'w') as ft:
with open(path_remote_file_inOurComp, 'r') as file_n:
for line in file_n:
line = line.replace("System.out.println(", "System.out.println(insertLength(")
line = line.replace(";", ");")
ft.write(line)
except IndexError:
print("Not enough input! ! !")
sys.stderr = original_stderr
f_error.close()
Check if it is useful to you
import sys
try:
raise
except Exception as err:
**exc_type, exc_obj, exc_tb = sys.exc_info()** # this is to get error line number and description.
file_name = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1] # to get File Name.
error_string="ERROR : Error Msg:{},File Name : {}, Line no : {}\n".format(err,file_name,exc_tb.tb_lineno))
file_log = open("error_log.log", "a")
file_log.write(error_string)
file_log.close()
logf = open("download.log", "w")
for download in download_list:
try:
# code to process download here
except Exception as e: # most generic exception you can catch
logf.write("Failed to download {0}: {1}\n".format(str(download), str(e)))
# optional: delete local version of failed download
finally:
# optional clean up code
pass
apart from this approach, you can use logging library to save each and every logs of your application.
Following is the method 2 of the problem of saving logs.
import logging
logging.basicConfig(filename='gunicon-server.log',level=logging.DEBUG,
format='[%(levelname)s]: [%(asctime)s] [%(message)s]', datefmt='%m/%d/%Y %I:%M:%S %p')
try:
# code to process download here
except Exception as e: # most generic exception you can catch
logging.error(str(e))
finally:
# optional clean up code
pass
import sys
sys.stderr = open('errorlog.txt', 'w')
# do whatever
sys.stderr.close()
sys.stderr = sys.__stderr__
see this for more details : https://ubuntuforums.org/showthread.php?t=849752

Write into a file using 'file = file_name' in print statement in python

In book headfirstpython in chapter4 they have used the syntax
print(list_name, file= output_file_name)
For them it's working fine, but for me it's giving syntax error on file = output_file_name. The python version is same i.e. 3.
code:
import os
man = []
other = []
try:
data = open('sketch.txt')
for each_line in data:
try:
(role, line_spoken) = each_line.split(':', 1)
line_spoken = line_spoken.strip()
if role == 'Man':
man.append(line_spoken)
elif role == 'Other Man':
other.append(line_spoken)
except ValueError:
pass
data.close()
except IOError:
print('The datafile is missing!')
try:
man_file = open('man_data.txt', 'w')
other_file = open('other_data.txt', 'w')
print(man, file=man_file)
print(other, file=other_file)
except IOError:
print('File error.')
finally:
man_file.close()
other_file.close()
As per the help of print function indicates
file: a file-like object (stream); defaults to the current
sys.stdout.
So the input is not supposed to be file-name but rather a file-like object. If you want to write into (say) a text file, you need to first open it for writing and use the file handle.
f = open("output.txt",'w')
print(list_name, file=f)

Pickle: TypeError: a bytes-like object is required, not 'str' [duplicate]

This question already has answers here:
Using pickle.dump - TypeError: must be str, not bytes
(3 answers)
Closed 4 years ago.
I keep on getting this error when I run the following code in python 3:
fname1 = "auth_cache_%s" % username
fname=fname1.encode(encoding='utf_8')
#fname=fname1.encode()
if os.path.isfile(fname,) and cached:
response = pickle.load(open(fname))
else:
response = self.heartbeat()
f = open(fname,"w")
pickle.dump(response, f)
Here is the error I get:
File "C:\Users\Dorien Xia\Desktop\Pokemon-Go-Bot-Working-Hack-API-master\pgoapi\pgoapi.py", line 345, in login
response = pickle.load(open(fname))
TypeError: a bytes-like object is required, not 'str'
I tried converting the fname1 to bytes via the encode function, but It still isn't fixing the problem. Can someone tell me what's wrong?
You need to open the file in binary mode:
file = open(fname, 'rb')
response = pickle.load(file)
file.close()
And when writing:
file = open(fname, 'wb')
pickle.dump(response, file)
file.close()
As an aside, you should use with to handle opening/closing files:
When reading:
with open(fname, 'rb') as file:
response = pickle.load(file)
And when writing:
with open(fname, 'wb') as file:
pickle.dump(response, file)
In Python 3 you need to specifically call either 'rb' or 'wb'.
with open('C:\Users\Dorien Xia\Desktop\Pokemon-Go-Bot-Working-Hack-API-master\pgoapi\pgoapi.py', 'rb') as file:
data = pickle.load(file)
You need to change 'str' to 'bytes'. Try this:
class StrToBytes:
def __init__(self, fileobj):
self.fileobj = fileobj
def read(self, size):
return self.fileobj.read(size).encode()
def readline(self, size=-1):
return self.fileobj.readline(size).encode()
with open(fname, 'r') as f:
obj = pickle.load(StrToBytes(f))
I keep coming back to this stack overflow link, so I'm posting the real answer for the next time I come looking for it:
PickleDB is messed up and needs to be fixed.
Line 201 of pickledb.py
From:
simplejson.dump(self.db, open(self.loco, 'wb'))
to:
simplejson.dump(self.db, open(self.loco, 'wt'))
Problem solved forever.

Zero size length on json object in Python

I'm not sure why this happen, there are zero length in my json file.
0
I t supposedly to be like this,
1000
I'm afraid the comma thing after each json object cause this issue. (My current json format)
{ A:"A"},{ B:"B"),...
The correct way is like this
{ A:"A"} { B:"B"),...
So how I can calculate all the length without removing the comma?
My code
import json
githubusers_data_path = 'githubusers.json'
githubusers_data = []
githubusers_file = open(githubusers_data_path, "r")
for line in githubusers_file:
try:
data = json.loads(line)
githubusers_data.append(data)
except:
continue
print len(githubusers_data)
Sample
{
"login": "datomnurdin"
}, {
"login": "ejamesc"
},...
I think you're getting an exception that you're suppressing with try-except, because of the commas.
One solution would be to convert your file to a string first, stick a '[' and ']' around the string to convert it into a valid json format, then use json.loads to convert the string.
import json
githubusers_data_path = 'githubusers.json'
githubusers_file = open(githubusers_data_path, "r")
githubusers_string = ''.join(line for line in githubusers_file)
githubusers_string = '[{}]'.format(githubusers_string)
githubusers_data = json.loads(githubusers_string)
print len(githubusers_data)
githubusers_file.close()
there is an exception in your code:
import json
githubusers_data_path = 'githubusers.json'
githubusers_data = []
githubusers_file = open(githubusers_data_path, "r")
for line in githubusers_file:
try:
data = json.load(githubusers_file) # exception from here
githubusers_data.append(data)
except Exception, e:
print e
print len(githubusers_data) # so githubusers_data is always []
Mixing iteration and read methods would lose data

Categories