Suppose, I want to read the complete file using pickle.load(), not just a single line. I know I can use try - except but is there any other method to read it?
I am using this:
import pickle
d = {}
for i in range(2):
roll_no = int(input("Enter roll no: "))
name = input("Enter name: ")
d[roll_no] = name
f = open("test.dat", "ab")
pickle.dump(d, f)
f.close()
f = open("test.dat", "rb")
while True:
try:
print(pickle.load(f))
except EOFError:
break
The official Python library does not support this within a single instruction. You can define your own helper function though:
import io
import pickle
from typing import List
def unpickle(file: io.IOBase) -> List[object]:
result = []
while True:
try:
result.append(pickle.load(file))
except EOFError:
break
return result
You can then call it like this
with open('data.pickle', 'rb') as f:
objects = unpickle(f)
objects will contain all the objects that have been serialized in data.pickle here.
You can use file.tell to see if you are at EOF
f = open("test.dat", "rb")
# go to end of file and get position
size = f.seek(0, 2)
# now return to the front and pull pickle records
f.seek(0)
while f.tell() < size:
print(pickle.load(f))
Related
I add to my dictionary 2 different meals and 2 different unit amounts, which then saves to my text file perfectly fine, but then when I exit the program and reload the file, I get a syntax error. (Also some functions look like they are not completed but that's just because Stack Overflow told me not to add all the code in.
Tried making the dictionary not start as an empty dictionary but that didn't solve anything.
import pprint
import time
pretty = pprint.PrettyPrinter(width = 20)
meals = {}
command = ""
condition = False
f = open("meals.txt", "r+")
f.write(str(meals))
f.close()
def save_dict_to_file(meals):
f = open("meals.txt", "r+")
f.write(str(meals))
f.close()
def load_dict_from_file():
f = open("meals.txt", "r+")
data = f.read()
f.close()
return eval(data)
load_dict_from_file()
def add_entry():
meal = input("Enter name of meal: ").lower()
units = int(input("Enter units needed: "))
meals[meal] = units
pretty.pprint(meals)
save_dict_to_file(meals)
def remove_entry():
def help():
def view_dict():
def ending_message():
while True:
command = input("> ").lower()
if command == "help":
help()
elif command == "add":
add_entry()
elif command == "exit":
save_dict_to_file(meals)
ending_message()
time.sleep(3)
break
elif command == "remove":
remove_entry()
elif command == "view":
view_dict()
After adding multiple entries to the dictionary, I'm expecting to be able to quit the program, load back up and when I type view to look at my entries, see what I have previously added in.This is what I get -
PS C:\Users\Alex\Desktop\Python\own_projects\mosh> python diabetes.py
Traceback (most recent call last):
File "diabetes.py", line 24, in <module>
load_dict_from_file()
File "diabetes.py", line 22, in load_dict_from_file
return eval(data)
File "<string>", line 1
{}lasagne': 12, 'fish and chips': 16}
^
SyntaxError: invalid syntax
The problem is probably caused by not adding newlines to your file. A simple fix would be: f.write(str(meals) + "\n")
But writing code in a textfile and then evaluating it is a bad idea:
The file will only ever be readable by your python program
It is prone to syntax errors (as in your question)
It is very unsafe. Malicious code could end up in your file.
As long as you only store text, numbers and true/false in your dictionary, it can be represented very cleanly by a JSON file. JSON has the advantage that it can be read by basically any programming language:
import json
data = {
"A": 1,
"B": {"C": 2}
}
# write data to file
with open("file.txt", "w") as file:
file.write(json.dumps(data))
# read data from file
with open("file.txt", "r") as file:
data = json.load(file)
If you store more complex objects in your file, maybe instances of classes etc, then you should look at pickle. That's another built-in library and a very convenient way to store almost everything from your python program. As Klaus D. has pointed out in a comment, pickle is not safer than your approach. You should never load a pickle object from an origin that you don't trust.
import pickle
with open("file.txt", "wb") as file:
pickle.dump(data, file)
with open("file.txt", "rb") as file:
data = pickle.load(file)
I don't entirely remember how pretty print works, but this way of accomplishing this is very fragile. The likelihood of something not printing in a way you expect it to is pretty high, which could break things as it is now.
I recommend using a more standard data transmission format like CSV or JSON. If your data is flat, I recommend CSV. If your data is more complex, I recommend JSON.
I'll edit this answer momentarily with examples of both methods.
CSV example:
import csv
# This is a list of dictionaries
data = [
{
'meal': 'breakfast',
'units': 20
},
{
'meal': 'lunch',
'units': 40
},
{
'meal': 'dinner',
'units': 50
}
]
# If you wanted the values for lunch, you could do data[1]['units']
# This says access the second value inside the data list,
# and get the value for the units in that dictionary
def write_data(file_path='meals.csv'):
# Open the file for writing without adding extra new lines
# The with syntax will automatically close it for us
with open(file_path, 'w', newline='') as f:
# Create a dictionary writer, telling it what columns to expect
writer = csv.DictWriter(f, ['meal', 'units'])
writer.writeheader()
writer.writerows(data)
def read_data(file_path='meals.csv'):
new_data = []
# open the file for reading
with open(file_path) as f:
# Create a dictionary reader. It will figure things out automagically
reader = csv.DictReader(f)
for row in reader:
print(row)
new_data.append(row)
return new_data
print('writing data...')
write_data()
print('reading data...')
print(read_data())
JSON example:
import json
data = {
'lunch': 10,
'breakfast': 20,
'dinner': 30
}
def write_data(file_path='meals.json'):
# Open the file for writing without adding extra new lines
# The with syntax will automatically close it for us
with open(file_path, 'w', newline='') as f:
json.dump(data, f)
def read_data(file_path='meals.json'):
# open the file for reading
with open(file_path) as f:
new_data = json.load(f)
return new_data
print('writing data...')
write_data()
print('reading data...')
print(read_data())
this need to work:
import pprint
import time
import os
pretty = pprint.PrettyPrinter(width = 20)
meals = {}
command = ""
condition = False
n = False
if not os.path.isfile('meals.txt'):
f = open("meals.txt", "w+")
f.write(str(meals))
f.close()
else:
n = True
def save_dict_to_file(meals):
f = open("meals.txt", "w+")
f.write(str(meals))
f.close()
def load_dict_from_file():
f = open("meals.txt", "r+")
data = f.read()
f.close()
return eval(data)
if n:
meals = load_dict_from_file()
def add_entry():
meal = input("Enter name of meal: ").lower()
units = int(input("Enter units needed: "))
meals[meal] = units
pretty.pprint(meals)
save_dict_to_file(meals)
import os
def remove_entry():
os.remove('meals.txt')
def help():
pretty.pprint('help')
def view_dict():
pretty.pprint(load_dict_from_file())
def ending_message():
pretty.pprint('done')
while True:
command = input("> ").lower()
if command == "help":
help()
elif command == "add":
add_entry()
elif command == "exit":
save_dict_to_file(meals)
ending_message()
time.sleep(3)
break
elif command == "remove":
remove_entry()
elif command == "view":
view_dict()
In book headfirstpython in chapter4 they have used the syntax
print(list_name, file= output_file_name)
For them it's working fine, but for me it's giving syntax error on file = output_file_name. The python version is same i.e. 3.
code:
import os
man = []
other = []
try:
data = open('sketch.txt')
for each_line in data:
try:
(role, line_spoken) = each_line.split(':', 1)
line_spoken = line_spoken.strip()
if role == 'Man':
man.append(line_spoken)
elif role == 'Other Man':
other.append(line_spoken)
except ValueError:
pass
data.close()
except IOError:
print('The datafile is missing!')
try:
man_file = open('man_data.txt', 'w')
other_file = open('other_data.txt', 'w')
print(man, file=man_file)
print(other, file=other_file)
except IOError:
print('File error.')
finally:
man_file.close()
other_file.close()
As per the help of print function indicates
file: a file-like object (stream); defaults to the current
sys.stdout.
So the input is not supposed to be file-name but rather a file-like object. If you want to write into (say) a text file, you need to first open it for writing and use the file handle.
f = open("output.txt",'w')
print(list_name, file=f)
I have to create a save function and a load function that saves a dictionary in the format of:
123;Kalle;
123;Maria;
321;Anna;
321;Olle;
My dictionary is supposed to look like a phonebook, with the key being the name and the value is the phonenumber:
telebook = {"jacob":"8472923777", "nisse":"092563243"}
How can I write a function that saves my phonebook in the format mentioned? It should look like this:
8472923777;jacob;
This is my current code:
def save(lista, telebook):
import pickle
filename = lista[1]
f = open(filename, "w")
pickle.dump(telebook, f)
f.close()
print telebook
def load(lista, telebook):
import pickle
try:
filename = lista[1]
f = open(filename, "r")
telebook_1 = pickle.load( f )
telebook.clear()
telebook.update(telebook_1)
f.close()
print telebook
except:
print "This file doesn't exist"
EDIT:
My save function was easier than I thought, managed to solve it on my own. Not sure how to get the load function to work though.
book = raw_input("telebook> ").lower()
lista = book.split()
def save(lista, telebook):
filename = lista[1]
f = open(filename, "w")
for name, num in telebook.items():
f.write(num+";"+name+";"+"\n")
f.close()
print telebook
My load is the same as before but obviously I can't use that one anymore.
def save(telebok, filepath):
with open(filepath, 'w') as outfile:
for name,num in telebok.items():
outfile.write("{};{};\n".format(num, name))
And to get it back:
import csv
def load(filepath):
with open(filepath) as infile:
telebok = dict((v,k) for v,k,_ in csv.reader(infile, delimiter=';'))
return telebok
I am iterating a .dat file save on a http website using
import urllib2
test_file = urllib2.urlopen('http://~/file.dat')
And then, I have a function which iterates the file
def f(file):
while True:
iter = file.readline()
if iter == "":
break
print iter
If I want to call this function twice without opening the test_file again:
f(test_file)
f(test_file)
then what should I add into the f function?
Update:
Since I am not allowed to change anything outside the function, I finally came up a silly but effective solution:
def f(file):
while True:
iter = file.readline()
if iter == "":
break
print iter
global test_file
test_file = test_file = urllib2.urlopen('http://~/file.dat')
Thanks for the guys who answered my questions!
f.seek(0)
returns you to the start of the file. The argument is the byte position in the file.
So the best thing for you to do is to save the output of your f.read() to a var and then push through via the StringIO.readline() method that will work similarly to f.readline() but within memory.
import urllib2
import StringIO
t_fh = urllib2.urlopen('http://ftp.cs.stanford.edu/pub/sgb/test.dat')
test_data = t_fh.read()
def f(data):
buf = StringIO.StringIO(data)
while True:
line = buf.readline()
if line == "":
break
print line
f(test_data)
I'd like to figure out how I should use a class to read input from a file so that I can use that data in other classes. If I read input from a file into a list, should I pass that to another class that needs that to use that information?
Right now I have:
import sys
class FileReader:
"""Reads a file"""
def __init__(self):
input = ''
try:
with open(sys.argv[1], 'r') as inFile:
input = inFile.readline()
print(input)
except IndexError:
print("Error - Please specify an input file.")
sys.exit(2)
def main():
x = FileReader()
if __name__ == "__main__":
main()
I thought about making some kind of list to hold strings from the file, but I'm not sure whether that should be global or not.
If all you're trying to do is read the file line by line, something like the following would work just fine (exception handling omitted).
>>> path = '/path/to/file.txt'
>>> with open(path, 'r') as f:
... lines = [l for l in f]
You can then pass around lines as necessary.