I was trying to load data by using this repository (uses some Python 2 originally): https://github.com/hashbangCoder/Text-Summarization
However I got an pickling error (using Python 2.7, I tried also Python2.6 with the same result):
>>> import cPickle as pickle
>>> pickle.load(open('train.bin', 'rb'))
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
cPickle.UnpicklingError: invalid load key, '?'.
I tried also with Python3 but without success (same for _pickle):
import pickle
pickle.load(open(path, 'rb'))
Error:
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
<ipython-input-9-0129e43fa781> in <module>()
----> 1 data = pickle.load(open(path, 'rb'), encoding='utf8')
UnpicklingError: invalid load key, '\xd9'.
There are plenty of questions out there dealing with this error, but I haven't found anything that solves my problem.
I tried also on different systems and downloaded it twice to be sure that the file wasn't corrupted during the download. I'm also getting similar errors for the other files.
So I guess it may be some kind of version or encoding problem here?
Any idea what I can try to load the file?
Thanks in advance!
I recently had this issue when trying to unpickle a file... try using joblib instead:
fname = 'Path_to_filename.pkl'
model = joblib.load(open(fname, 'rb'))
Otherwise - it is likely a corrupted file.
I had this issue.
I transferred files using disk.
They were not properly saved.
The issue disappeared after I verified the save to disk.
i also had the same problem coz file was not correctly stored on the disk it was corrupted redownloaded it error was gone
I think you should use a file with PKL extension then it gonna work
train_data = pickle.load(open('train_data.pkl','rb'))
Related
path of file is:
"C:\Users\deana\OneDrive\Marlon's files\Programming\Python\PITT\PITT_LIbrary\Lists\test.txt"
lines of code are:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt'))
the error is this:
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
exec(open('test.txt'))
TypeError: exec() arg 1 must be a string, bytes or code object
also if I try on one line as such:
exec(open(r"C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists/test.txt"))
i'ts the same error. (with and without r)
super frustrationg as it reads like i'm not inputting string... but it is string!?!
also I've done this litteraly the same way before, restarted IDLE shell, no difference.
ugh! I always get stupid errors with file paths.
I should have been using os.startfile() to open this.
It was confusing by using .open(). as I was attempting to open in default app.
before, i've used exec.open() to open .py files and guess I got them confused.
exec is just used to open other scripts... need stronger coffee next time.
Try this:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt', 'rb'))
You can convert the txt file to bytes by opening it with rb (read bytes).
I was pickling some variables and the process was interrupted. How do I recover data from the partially pickled file?
Pickled two lists of lists using:
import pickle
import sys
sys.setrecursionlimit(5000) # to get around max depth recursion error
with open('level4_half.pkl', 'wb') as f:
pickle.dump([level4_url, level4_desc], f)
Checked from windows explorer that the file is not empty (158MB)
Tried to unpickle file using:
with open('level4_half.pkl','rb') as f:
level4_url, level4_desc = pickle.load(f)
and encountered the error:
Traceback (most recent call last):
File "<ipython-input-18-32ed3a0e79d4>", line 2, in <module>
level4_url, level4_desc = pickle.load(f)
EOFError
(Previously I've tried to pickle and unpickle (fully pickled) files successfully using the commands above.)
I found a similar question here but I didn't use dill and am not sure if a partially pickled file is considered corrupted. My current technical skill is not so honed as to be able to quickly implement the solution there: "Read through the Python module's source code and you can probably find a way to hook all of the load_ methods to give you more information." If this turns out to be the same solution to my question, it will be great to get guidance on how I can "hook all of the load_ methods".
Thank you.
My program was killed while serializing data (a dict) to disk with dill. I cannot open the partially-written file now.
Is it possible to partially or fully recover the data? If so, how?
Here's what I've tried:
>>> dill.load(open(filename, 'rb'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "lib/python3.4/site-packages/dill/dill.py", line 288, in load
obj = pik.load()
EOFError: Ran out of input
>>>
The file is not empty:
>>> os.stat(filename).st_size
31110059
Note: all data in the dictionary was comprised of python built-in types.
The pure-Python version of pickle.Unpickler keeps a stack around even if it encounters an error, so you can probably get at least something out of it:
import io
import pickle
# Use the pure-Python version, we can't see the internal state of the C version
pickle.Unpickler = pickle._Unpickler
import dill
if __name__ == '__main__':
obj = [1, 2, {3: 4, "5": ('6',)}]
data = dill.dumps(obj)
handle = io.BytesIO(data[:-5]) # cut it off
unpickler = dill.Unpickler(handle)
try:
unpickler.load()
except EOFError:
pass
print(unpickler.stack)
I get the following output:
[3, 4, '5', ('6',)]
The pickle data format isn't that complicated. Read through the Python module's source code and you can probably find a way to hook all of the load_ methods to give you more information.
I can't comment on the above answer, but to extend Blender's answer:
unpickler.metastack worked for me, dill v0.3.5.1 (though you could do it without dill, afaik). stack did exist, but was an empty list.
Also, with dill I got a UnpicklingError rather than EOFError. This could also be partly because of how my file got corrupted (ran out of disk space)
I am trying to figure out the basics of importing DICOM files in Python using pydicom. While trying really simple code, I get following errors:
For code:
import dicom
filePath="C:\Python34\Lib\site-packages\dicom\testfiles"
ds=dicom.read_file(filePath[0])
I get error:
C:\Python34\python.exe C:/Users/041213/PycharmProjects/D/Deki.py
Traceback (most recent call last):
File "C:/Users/041213/PycharmProjects/D/Deki.py", line 4, in
ds=dicom.read_file(filePath[0])
File "C:\Python34\lib\site-packages\dicom\filereader.py", line 589, in read_file
fp = open(fp, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'C'
I am using Python 3.4, pydicom 0.9.9 and JetBrains PyCharm Community Edition 2016.3.2
If anyone can help me with this, or even just help me how to load a DICOM file in general, I would appreciate it a lot.
you're passing the first char of the string (C) instead of the full string. Just do:
ds=dicom.read_file(filePath)
next error you'll stumble into: use raw prefix or \t gets interpreted as a tabulation character:
filePath=r"C:\Python34\Lib\site-packages\dicom\testfiles"
^
Use and \ escape character to avoid issues with tab and other special characters. Also remember when you do filePath[0] on a string it returns the first character
filePath="C:\\Python34\\Lib\\site-packages\\dicom\\testfiles"
ds=dicom.read_file(filePath)
Okay, so I just started this tutorial using Anaconda in PyCharm. I imported the correct libraries, but when I try to upload my image using this code:
img = mpimg.imread('FileName.png')
I get this error message:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File ".../matplotlib/image.py", line 1323, in imread
with open(fname, 'rb') as fd:
IOError: [Errno 2] No such file or directory: 'FileName.png'
I tried putting in the file location like this:
img=mpimg.imread('FilePath/FileName.png')
But I got the same error. I want to follow the tutorial exactly so I'm not sure why this isn't working! Any help is appreciated!
Make sure that there is that file in the folder...
Try this command to list the files in that folder just to make sure that python atleast recognizes or reads the files.
import os
print (os.listdir('your path'))
you should give an absolute path for the image. for example
r'C:\Users\HP\Desktop\stinkbug.png' adding r to remove the Unicode error
I was providing the full path: "C:/user/username/Downloads/folder/file.ext"
When I changed it to relative, it worked for me: "Downloads/folder/file.ext"