Given a report (which is just a dictionary) and a filename, I want to be able to write into the supplied file name all of the contents of the report. I want to make sure I don't overwrite anything in the filename. This is what I have:
def write_report(r, filename):
input_filename=open(filename, "a")
new_report= input_filename.append(r)
filename.close()
return new_report
But I get this error when I test it:
AttributeError: '_io.TextIOWrapper' object has no attribute 'append'
How do I append something into a file?
Use json module to write dictionary to a file;
>>> import json
>>> d = dict.fromkeys('abcde')
#Write
with open('abc.json', 'w') as f:
json.dump(d, f)
#Read
with open('abc.json') as f:
print (json.load(f))
...
{'a': None, 'b': None, 'c': None, 'd': None, 'e': None}
There's two errors there.
The method to write to a file is write(), not append()
You're calling close() on a string, you should close() the file object input_filename.
Also, you may want to rename input_filename to output_file.
Related
I have list which is saving in file while while returning I need to
(https://docs.python.org/3.8/library/ast.html#ast.literal_eval ) do ast.literal_eval method to getting as element
my list is having combination lst = FirstName, empid,age,salary,filename
example my list which is saved in file is ['Joe',101,31,99292,'/home/Joe/Joe.txt'], If I need to pass this to return I need to use ast.literal_eval
How to save with the list in to file with help of pickle and how to return it?
Here's a way to save pickle into a file, and then load it again:
Save:
s = ['Joe',101,31,99292,'/home/Joe/Joe.txt']
with open("my_file.pcl", "wb") as f:
p = pickle.dumps(s)
f.write(p)
Load:
with open("my_file.pcl", "rb") as f:
p = f.read()
s = pickle.loads(p)
print(s)
Result:
['Joe', 101, 31, 99292, '/home/Joe/Joe.txt']
I have a file folder of 1000+ json metadata files. I have created a list of the file paths and I'm trying to:
for each file path, read json file
pull in only the key value pairs I'm interested in
store it in a variable or save it in a way that I can insert into
mongodb using pymongo
I have been successful listing the file paths to a variable and loading ONE json doc (from one file path). The problem is I need to do over a thousand and I get an error when trying to incorporate list of file paths and loop.
Here's what I've tried so far:
import pymongo
import json
filename = r"C:\Users\Documents\FileFolder\randomFile.docx.json"
with open(filename, "r", encoding = "utf8") as f:
json_doc = json.load(f)
new_jsonDoc = dict()
for key in {'Application-Name', 'Author', 'resourceName', 'Line-Count', 'Page-Count', 'Paragraph-Count', 'Word-Count'}:
new_jsonDoc[key] = json_doc[0][key]
Sample output:
{'Application-Name': 'Microsoft Office Word',
'Author': 'Sample, John Q.',
'Character Count': '166964',
'Line-Count': '1391',
'Page-Count': '103',
'Paragraph-Count': '391',
'Word-Count': '29291',
'resourceName': 'randomFile.docx'}
Now when I add the loop:
for file in list_JsonFiles: # this is list of file paths created by os.walk
# if I do a print(type(file)) here, file type is a string
with open(file, "r") as f:
# print(type(file)) = string, print(type(f)) = TextIOWrapper
json_doc = json.loads(f)
### TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper ###
How can I get my loop working? Is my approach wrong?
Figured the TypeError out:
for file in list_JsonFiles:
with open(file, "r", encoding = "utf8") as f:
json_doc = json.load(f)
new_jsonDoc = dict()
for key in {'Application-Name', 'Author', 'resourceName', 'Line-Count', 'Page-Count', 'Paragraph-Count', 'Word-Count'}:
if key in json_doc[0]:
new_jsonDoc[key] = json_doc[0][key]
I'm trying to be able to load a class instance from a JSON file. For some reason, the JSON data can be read from the file (see the print line) but can't be set to a variable.
JSON file contents:
{"key": 1}
with open(json_path) as json_file:
print(json.load(json_file)) # prints {'key': 1}
class_dict = json.load(json_file)
I get this error:
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
I tried json.load and json.loads with a string value. I tried adding additional parameters to the open function. None of it works. I validated the JSON here: https://jsonlint.com/.
I agree with #Scott Hunter, you already read the file with the first json.load statement. If you need to assign the contents directly you could rewind the file.
with open(json_path) as json_file:
print(json.load(json_file))
json_file.seek(0)
class_dict = json.load(json_file)
print class_dict
You're loading the file twice by
with open(json_path) as json_file:
print(json.load(json_file)) # prints {'key': 1}
class_dict = json.load(json_file)
The first json.load(json_file) will load the open file fully, so the second time there is nothing more to read.
If you want to print it as well as assign it, assign it first, and then print it:
with open(json_path) as json_file:
class_dict = json.load(json_file)
print(class_dict) # prints {'key': 1}
I want to do the following:
1- Check if a pkl file with a given name exists
2- If not, create a new file with that given name
3- Load the data into that file
if not os.path.isfile(filename):
with open(filename,"wb") as file:
pickle.dump(result, file)
else:
pickle.dump(result, open(filename,"wb") )
However, this rises an error even though I have checked the file exists (shouldnt even enter the if!!) with the given path:
Traceback (most recent call last):
with open(filename_i,"wb") as file:
IsADirectoryError: [Errno 21] Is a directory: '.'
Thanks!
You can do it like this:
import os
import pickle
if not os.path.isfile("test_pkl.pkl"):
with open("test_pkl.pkl",'wb') as file:
pickle.dump("some obejct", file)
So first it checks if file exists, if not create the file ("wb") and then dump some object to it via pickle pickle.dump
Maybe this is more clear:
Imports
import os
import pickle
Create pickle and save data
dict = { 'Test1': 1, 'Test2': 2, 'Test3': 3 }
filename = "test_pkl.pkl"
if not os.path.isfile(filename):
with open(filename,'wb') as file:
pickle.dump(dict, file)
file.close()
Opening the pickle file
infile = open(filename,'rb')
new_dict = pickle.load(infile)
infile.close()
Test the data
print(new_dict)
print(new_dict == dict)
print(type(new_dict))
Output
{'Test1': 1, 'Test2': 2, 'Test3': 3}
True
<class 'dict'>
I am exporting data from an item to a json file with srapy's jsonitemexporter. Now I would like to add some basic information about the data to the file, e.g. partner name or pagename.
Putting this code into
class BidPipeline(object):
file = None
def open_spider(self, spider):
self.file = open('data/'+ datetime.datetime.now().strftime ("%Y%m%d") + '_' + spider.name + '.json', 'wb')
self.exporter = JsonItemExporter(self.file)
# trying to add partner info
a = {'partner': 3}
line = json.dumps(a) + "\n"
self.file.write(line)
self.exporter.start_exporting()
Results in traceback:
yield self.engine.open_spider(self.spider, start_requests)
builtins.TypeError: a bytes-like object is required, not 'str'
My goal is to add some info to the json file before starting the export of the items, so later while processing the data one can determine e.g. the source.
What would be the best way to achieve this?
There error is pretty self explanatory here:
a bytes-like object is required, not 'str'
You open file to write bytes (wb) and you try to write string:
def open_spider(self, spider):
self.file = open(..., 'wb')
^^^^^
...
a = {'partner': 3}
line = json.dumps(a) + "\n"
^^^^
self.file.write(line)
To resolve this either open file as string file (just w instead of wb) or encode your line before writing it to file:
self.file.write(line.encode())
Preferably you should always use w when writing text and wb when writting bytes (e.g. image data)