EOFerror in unpickling Python dictionary from an empty pickle file - python

Very newbie programmer, sorry if this is stupid or if my English is wrong. So, I have this command line address book that I am writing. It consists of a dictionary that holds an object with the key as the name variable, and each object has variables associated with it like the name of the person, the email, etc... It works, but now I'm trying to make it store the dictionary persistenly in memory using pickle.
def create_person():
"""Adds an instance object of the Person class to the dictionary persons. persons is a global variable, that has been created previously. DATA is a variable that points to a file named test.data that exists in the same directory as the script."""
name = raw_input("Enter the person's name here: ")
email = raw_input("Enter the person's email here: ")
phone = raw_input("Enter the person's phone here: ")
address = raw_input("Enter the person's address here: ")
f = open(DATA, "rb")
persons = pickle.load(f) #assign whatever is saved in the dictionary in persistent memory to global variable persons, which is empty at this point in the beginning
f.close()
persons[name] = Person(name, email, phone, address)
f = open(DATA, "wb")
pickle.dump(persons, f)
f.close()
However, I'm getting this error:
Traceback (most recent call last):
File "testpickle.py", line 85, in <module>
main()
File "testpickle.py", line 40, in main
create_person()
File "testpickle.py", line 20, in create_person
persons = pickle.load(f)
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 1378, in load
return Unpickler(file).load()
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/home/pedro/anaconda/lib/python2.7/pickle.py", line 880, in load_eof
raise EOFError
EOFError
I don't understand this. I had actually already written this program, and it was working with memory saving, but I accidently deleted it. What is happening?

pickle hitting EOF is a sure tell the file is corrupt (probably truncated, maybe a write operation failed).
If you upload it somewhere, we might be able to deduce what exactly is wrong with it. In the worst case, you'll have to peek inside and deduce the data that was there to re-input it by hand.
That's your price for using an unreadable format and not paying attention to its integrity (e.g. writing to another file and only moving it over the original one after saving succeeded).
UPDATE:
If you want to start from a new, "empty" file, then handle the case when the file is missing and produce an empty dict. After all, the file is supposed to be missing initially, isn't it?
An empty file is not valid pickle data (it has to at least contain information about the object's type).
Here's "one obvious way" ((c) Python Zen) to handle a missing file:
import errno
try: f = open(DATA, "rb")
except IOError,e:
if e[0]==errno.ENOENT: persons={}
else: raise
else:
persons = pickle.load(f)
f.close()
del f

I don't see anything obviously wrong with your code, but if you just want to store key/value pairs like this, you probably ought to use the shelve module instead of a home-brewed solution.
import shelve
persons = shelve.open(DATA)
def create_person():
"""Adds an instance object of the Person class to the dictionary persons. persons is a global variable, that has been created previously. DATA is a variable that points to a file named test.data that exists in the same directory as the script."""
name = raw_input("Enter the person's name here: ")
email = raw_input("Enter the person's email here: ")
phone = raw_input("Enter the person's phone here: ")
address = raw_input("Enter the person's address here: ")
persons[name] = Person(name, email, phone, address)
while more_people:
create_person()
persons.close()

Related

How do I save the outputs of a code to a text file for logging (python)

Hello I am really new to programming and im trying to make like some form-like thing where you input stuff and it gets saved to a text file how do I input the outputs of my code to a text file?
date = input("The Date [Month/Day/Year]: ")
print("")
Name = input("Name: ")
print("")
Age = int(input("Age: "))
print("")
Gender = input("Gender: ")
print("")
Nationality = input("Nationality: ")
p = Age + 10
print(p)
print(date, Name, Age, Gender, Nationality)
I tried adding
sys.stdout = open("test.txt", "w")
(my code here)
sys.stdout.close()
I got from google but instead what it did when I run it is leave the console completely blank. I'll appreciate help, thank you!
By default when your program is printing stuff normally, it prints it to a special file named "stdout" (AKA "standard output", which is your console). The code you got updates stdout to instead reference a file named "test.txt" in write mode (hence the "w"). Therefore, everything you print normally will be rerouted to that new file. There should be a file named "test.txt" in the directory where you are executing your python program, and it should have the output of your program.
Another way of writing to files that I often see is:
with open('test.txt', 'w') as file:
# your code here
# the file will save and close after you exit this block scope
# ex. write:
file.write('hi')
Create a file in which you want to store your information. Collect all your data in a list or dict. I would prefer dict because information is stored systematically. After that, open the file in python in append mode. Remember that your program and your file must be at the exact same location. If not, mention the file location while opening it. I'm recommending append mode because everytime you write, the data should not be lost. If you open in write mode, everytime you write in a file, it writes from starting which may result in losing of data. Now write the dict in the file. Your code:
date = input("The Date [Month/Day/Year]: ")
print("")
Name = input("Name: ")
print("")
Age = int(input("Age: "))
print("")
Gender = input("Gender: ")
print("")
Nationality = input("Nationality: ")
p = Age + 10
print(p)
di={"Date":date,"Name":Name,"Age":Age,"Gender":Gender,"Nationality": Nationality}
print(date, Name, Age, Gender, Nationality)
f=open("test.txt","a")
f.write(di)
f.close()

Python Pickle, Loading Issue when returning to list

I am using pickle to serialize a list of objects, encrypting the serialized string and then using pickle to dump it into a file so that I can be read out in reverse later. I understand this may be a round about way of doing things but in theory I cant seem to get my head around why when reloading the data it continues to fail. Let me first apologize for my horrendous naming conventions one day these will improve.
I have dumped the code below, the error only occurs at the very end when loading the data back into the list and fails with the following error:
Traceback (most recent call last):
File "servertest.py", line 453, in <module>
mylist = mary1.ReadFromFile()
File "servertest.py", line 142, in ReadFromFile
mrylist = pickle.loads(datastring)
File "C:\Python27\lib\pickle.py", line 1388, in loads
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 864, in load
dispatch[key](self)
KeyError: '\xa7'
I have printed the data out at each step when calling both the write and read functions and it appears to be fine:
This is rawdata
]q ((c__main__
User
q☺oq☻}q♥(U☻idq♦U♦daveq♣U♥pinq♠U☻22qub(h☺o}q (h♦U♦maryq
h♠U♥434q♂ube.
this is encrypt data
'▼ES╬NZ┴w▲sæô▄Hó^┴☻½K◄6►ëΘ░¡√╡▼/ⁿ♀*a]ì╡
5åα¬δ⌡D╓£o¶α{ô √■♣í♠.|H^E«√☼Hwñ/cúw╠Σ√↔↔╞╞┼3
this is enc data
'▼ES╬NZ┴w▲sæô▄Hó^┴☻½K◄6►ëΘ░¡√╡▼/ⁿ♀*a]ì╡
5åα¬δ⌡D╓£o¶α{ô √■♣í♠.|H^E«√☼Hwñ/cúw╠Σ√↔↔╞╞┼3
unencrypted data
]q ((c__main__
User
q☺oq☻}q♥(U☻idq♦U♦daveq♣U♥pinq♠U☻22qub(h☺o}q (h♦U♦maryq
h♠U♥434q♂ube.
The following shows the two functions I'm using:
def WriteToFile(self, mrlist):
self.mrlist = mrlist
rawdata = pickle.dumps(mrlist, 1)
#print 'rawdata1\n' + rawdata
#encrypt pickled list (call on my encryption functions)
encrydata = self.encryptmessage.EncrMsg(rawdata)
print('this is encrypt data \n ' + encrydata)
#Open and dump encrypted string to file
file=open(self.fn,"wb")
pickle.dump(encrydata, file, 1)
file.close()
#Function to call when reading the file
def ReadFromFile(self):
mrylist = []
#open file and load
file = open(self.fn, "rb")
encdata = pickle.load(file)
file.close
print('this is enc data*********\n ' + encdata)
#Decrypt the string
datastring = self.encrytmessage.DeCryptMsh(encdata)
print 'unencrypted data' + datastring
mrylist = pickle.loads(datastring)
return mrylist`
Edit:
The following shows the simple user class for those interested in seeing what the list contains:
class User:
def __init__(self, id, pin):
self.id = id
self.pin = pin
def getpair(self):
return ' %s " | " %s ' % (self.id, self.pin)
def edit_pin(self, pin1):
self.pin = pin1
Solution
I managed to get this sorted, my original way of doing this was working fine and it wasn't actually an issue with pickle at all. It appears that my initialisation vector when calling my decryption function differed from when it was originally encrypted.
This still produced an identical string when decryption took place but must have included some differences i was unable to see. I've rewritten the ReadWrite class to make use of Read and Write functions rather than using pickle to dump the serialised/encrypted list into the file. I have also started writing the IV to an external so that it can be read by the decrypt method the next time the program loads.
Thank you for any help and suggestions given.

Check input python so i can close a file?

file=open('ClassA1.txt','a')
file=open('ClassB1.txt','a')
file=open('ClassC1.txt','a')
print('hello welcome to maths 2000')
Class=input('please enter your class '+"\n")
name=input('please enter your name '+"\n")
if Class==(int(input"A1")):
file.close('ClassB1')
file.close('ClassC1')
file.write(name+"/n")
file.close
How do I get it to check user input so it can close the files?
Ok first point :
This line:
file=open('ClassA1.txt','a')
opens file 'ClassA1.txt' for appending, assign the file object to the name file (which eventually shadows the builtin type file but that's not relevant here)
Then the second line:
file=open('ClassB1.txt','a')
opens file 'ClassB1.txt' for appending, assign the file object to the name file, sus replacing the binding to the previously opened file "ClassA1.txt". Since there's no other name referencing this previously opened file, it's lost. In the best case, the underlying file pointer will be closed when the object gets garbage-collected (CPython) but this is NOT garanteed by thye language's specification and another implementation might not free the file pointer correctly.
In all cases you can not access 'ClassA1.txt' anymore at this point.
Now the third line:
file=open('ClassC1.txt','a')
does the same thing - reassigning the name file to a new file object etc.
At this point, you have to possibly opened, possibly not, and in both case unreachable (and possibly already garbage collected) file objects and the name file points to the third one - which means any write operation on file will write to file "ClassC1.txt".
If you want to keep all three files opened, you have to keep references to them, either by binding each to a distinct name, ie:
file1=open('ClassA1.txt','a')
file2=open('ClassB1.txt','a')
file3=open('ClassC1.txt','a')
or by storing them in a list:
files = []
files.append(open('ClassA1.txt','a'))
files.append(open('ClassB1.txt','a'))
files.append(open('ClassC1.txt','a'))
so you can now acces them by index, ie files[0], files[1], files[2]
or in a dict:
files = {}
files["A1"] = open('ClassA1.txt','a')
files["B1"] = open('ClassB1.txt','a')
files["C1"] = open('ClassC1.txt','a')
so you can now acces them by key, ie files["A1"], files["A2"], files["A3"]
BUT : why would you
open three files,
ask the user which file he wants to write to,
close the two other files,
write to the selected file
close it
when you could more simply:
ask the user which file he wants to write to,
open it
write to it
close it
Since your files are named after the class name, you can easily build the filename from the class name:
cls = input("please enter your class\n")
filename = "Class{}.txt".format(cls)
f = open(filename, "a")
f.write("whatever")
f.close()
or even more safely (this will ensure the file WILL be closed whatever happens):
cls = input("please enter your class\n")
filename = "Class{}.txt".format(cls)
with open(filename, "a") as f:
f.write("whatever")
Note that in this case you don't have to call f.close()
A couple other points:
Class=input('please enter your class '+"\n")
=> 'cls' or 'class_', not 'Class' - by convention, capitalized names are for class (in the OO meaning) names.
=> Python is not PHP: 'please enter your class \n' just works
if Class==(int(input"A1"))
I don't know what you expect this line to do, but it sure looks you don't know either... One thing is sure : a string won't be equal to an integer. Never...
file.close('ClassB1')
Have you read the documentation at all ? It's here (well, for a starter at least) : https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files
file.close
You want file.close() here. The parens are NOT optional - if you want to call the method at least.
You're doing a long winded way, you could just open the file after the user gives their answer.
import os
print('hello welcome to maths 2000')
yourClass=input('please enter your class '+"\n")
filename = 'Class{}.txt'.format(yourClass)
name=input('please enter your name '+"\n")
if os.path.exists(filename):
f = open(filename, 'a')
file.write(name+"/n")
file.close()
else:
print ("Class not found")
I just set it up so the input determines the filename it tries to open, and if that file exists it opens it and appends their name.
When you execute these
file=open('ClassA1.txt','a')
file=open('ClassB1.txt','a')
file=open('ClassC1.txt','a')
file variable contains "ClassC1.txt", you are re-assigning the object again and again.
so after whatever check if you execute this:
file.close()
last file will be closed.
Instead I would recommend you first take the input of what file is to be opened and then open that file.
file_name = input("Enter file name")
file = open(file_name, 'a')
#do your work
file.close()
fd = open("Student_info.txt", "a+")
class_info = []
print "Hello, welcome to maths 2000"
class_to_be = raw_input("Please enter your class: ")
name = raw_input("Please enter your name: ")
student = name + " " + class_to_be
class_info.append(student)
print class_info
for students in class_info:
fd.write("%s" %(students))
fd.close()
results:
jester112358#ubuntu:~$ python stackhelp.py
Hello, welcome to maths 2000
Please enter your class: Python-class
Please enter your name: Greenie245
['Greenie245 Python-class']
and writes the content of your list to Student_info.txt
I think it's better to have all the information in one file, but obviously you can have a file for every class if you want.
If you want class for every file, consider using:
for students in class_info:
spl = students.split()
if spl[1] == "A1":
A1=open('ClassA1.txt','a')
A1.write("%s" %(students))
A1.write("\n")
A1.close()
elif ... # add anothor classes here

Opening a file with a name error

I am trying to build a function that asks the user to input the name of a file, opens the file, reads its contents, prints the contents of the file on the screen, and closes the file. If no such file exists, it's okay if the script crashes. When I run the function, it gives me: NameError: name 'myInput' is not defined, and I'm not sure how to fix it.
Here is what I have so far:
print(input('Please enter the name of a file that you want to open.' + myInput))
with open(r"C:\Python32\getty.txt", 'r') as infile:
data = infile.read()
print(data)
Help if you can..
myInput is an undefined variable, and I can't fathom what you had in mind by using it.
Maybe where you show the code...:
print(input('Please enter the name of a file that you want to open.' + myInput))
with open(r"C:\Python32\getty.txt", 'r') as infile:
you actually meant something very different, e.g like...:
myInput = input('Please enter the name of a file that you want to open.')
with open(myInput, 'r') as infile:
...?
In your first line, you have:
print(input('Please enter the name of a file that you want to open.' + myInput))
do you have myInput defined? You need to define it. If you don't have it defined before that line, your script will crash.
This can be gleaned from your helpful error message:
NameError: name 'myInput' is not defined
Which means that the variable myInput is not defined, so the compiler doesn't know what to put there.
I think something like this would solve your problem
fileName = raw_input("Please enter the name of a file that you want to open. ")
fileObject = open(fileName, "r")
fileText = fileObject.read()
print(fileText)

Splitting database throwing errno 9 bad file descriptor with CON file

I've done some research and from what I can tell this normally happens when a file is closed before it's finished being used?
But that doesn't make sense for what is happening here.
Here is my code:
import csv
dicto = {}
name = ""
with open(input("enter filepath here: "), "r") as mainfile:
reader = csv.reader(mainfile)
for row in reader:
name = row[8].lstrip("'")
name = name.lstrip("\xa0")
name1 = name
name = name.upper()
if not name[:3] in dicto:
dicto[name[:3]] = [name[:3]+".js", 0]
with open(dicto[name[:3]][0], "w") as file1: #here is the problem line
file1.write("tags=[")
else:
dicto[name[:3]][1] += 1
if name[:1] == "#":
print(name)
with open(dicto[name[:3]][0], "a") as file2:
if dicto[name[:3]][1]>0:
file2.write('various spam')
else:
file2.write('various eggs')
for key in dicto.keys():
with open(dicto[key][0], "a") as file3:
file3.write("\n];")
I'm running through a large database and splitting it up into JS files which are named after the first three letters of the data's label. It seems to run ok at first (there are 44k entries to go through so it takes a few seconds to finish). Overall I currently have 309 files generated, although none are complete. However Once it gets to the combo "CON" an error occurs:
Traceback (most recent call last):
File "C:\Users\SarbickiN\Documents\Codes\Python\schools\schools.py", line 16, in <module>
with open(dicto[name[:3]][0], "w") as file1:
OSError: [Errno 9] Bad file descriptor: 'CON.js'
and this shuts down the programme. Is there any reason why this would have happened? I've made a comment next to the line causing the issue.
Edit: solution (or lack thereof)
CON is a reserved name for files in windows along with a few others so would need to be replaced with something else. Check here for more details.
Self answering
CON is a reserved name for files in windows along with a few others so would need to be replaced with something else. Check here for more details.

Categories