I often use pickle files to store my dataset. Currently I encounter a trouble. In my local desktop, the python version is python 3.8. However, In the remote server, the python version is python 3.7. I need to work in the remote server. So
I want to know how to read the pickle files created by the python 3.8?
import pandas as pd
df=pd.read_pickle('FUND_AREACLASS.pkl')
The report error is here:
File "C:\ProgramData\Anaconda3_new\lib\site-packages\pandas\io\pickle.py", line 181, in read_pickle
return pickle.load(f)
ValueError: unsupported pickle protocol: 5
Can anybody help me figure it out? Thanks in advance.
Related
I have a python script which uses pickle that I've been running for weeks. I recently installed dropbox so that I could run the script on machine A, pickle the data to dropbox, and then load the data from dropbox onto machine B. So, I used to write to a file in the path of the script, now I write to a separate file synced by dropbox.
When I go to load the data, I get the following error:
data = pickle.load(f)
ModuleNotFoundError: No module named 'pandas._libs'
However, this prior line works fine:
import pandas as pd
In fact, if I run the script that's dumping (rather than loading) data, it also runs successfully.
I've also verified the path is correct using sys.path.
What could be the problem?
As mentioned in comments, this is a pandas version issue. Your pickle file was created by pickling objects containing a newer version or pandas, and the system unpickling that file contains an older version of pandas.
To be more precise, pandas._libs first appeared in:
commit 648ae4f03622d8eafe1ca3b833bd6a99f56bece4
Author: Jeff Reback
Date: Tue Mar 7 18:21:18 2017 -0500
BLD: consolidate remaining extensions
moves extensions to pandas/_libs, which holds the extension code
and also the generated builds (as its importable).
... which first appears in version 0.20. It stands to reason that your pickle file was created with a version of pandas >= 0.20, and the unpickle system has version < 0.20.
I am trying to save a sklearn model on a Windows server using sklearn.joblib.dump and then joblib.load the same file on a linux server (centOS71). I get the error below:
ValueError: non-string names in Numpy dtype unpickling
This is what I have tried:
Tried both python27 and python35
Tried the built in open() with 'wb' and 'rb' arguments
I really don't care how the file is moved, I just need to be able to move and load it in a reasonable amount of time.
Python pickle should run between windows/linux. There may be incompatibilities if:
python versions on the two hosts are different (If so, try installing same version of python on both hosts); AND/OR
if one machine is 32-bit and another is 64-bit (I dont know any fix so far for this problem)
Here is my code.
Line number 65
ma=cv2.imread(str(files1[x]),1)
The result of a cv2.imread() is always a None.
I have done all basic checks.
The file which i'm trying to read exists.
There are no other variables or functions called cv2 or imread.
I'm using the same version of python in all cases.
I have only one version of opencv installed.
I'm using ubuntu 14.04 and the folder has read and write permissions for all users.
I have also tested it on Pycharm with a new python file and it works.
I have only problem with this program.
Please let me know if u have any ideas about this problem.
I'm using virtualenv with my application, and I've installed gdata, jira, and gspread using env/bin/pip install <lib name> in terminal under my project folder. I'm following the documentation from the Google API but it is not working?
In the documentation, in order to do error handling you need to do:
from gdata import errors
And in order to create an instance of the Drive API service (in order to later on create a file) you need to do:
from gdata.discovery import build
However the files are different, there is no "discovery" or "errors" and when I run env/bin/python run.py I get this error:
Traceback (most recent call last):
File "run.py", line 3, in <module>
from gdata import errors
ImportError: cannot import name errors
(same with discovery)
I thought that maybe they mean from apiclient import errors literally in the documentation, so I tried pip installing apiclient and replacing gdata with apiclient but it still does not work.
I downloaded the gdata.zip file and unzipped it and looked through the sample code (especially for spreadsheet since that's what I'm trying to create) and they take a very different approach than the documentation and I'm very confused. My goal is to use their API to just create a spreadsheet from the code, but I do not plan on using their API to edit the spreadsheet itself, I plan on using gspread (Github).
I've done a lot of research and I've been directed to a lot of different places and I might have perhaps mixed up the code? Does anyone know what I did wrong/have a fix? A huge thanks in advance.
This kind of import error is usually caused by the user installing another module of the same name. Do you by any chance have a gdata.py somehwere on your Python path?
You can verify whether this is causing the issue by:
import gdata
print gdata.__file__
This tells you where the interpreter is loading the code from.
Context
Steps taken:
Environment Setup
I've installed protobufs via Home Brew
I've also followed the steps in the proto-bufs python folder's readme on installing python protobufs - namely running the python setup.py install command
I've using the protobuf-2.4.1 files
Coding
I have a python file (generated from a .proto file I compiled) that contains the statement, among other import statements, but I believe this one is the one causing issues:
from google.protobuf import descriptor_pb2
The above python file, I'm importing in another python file, it's
this python file that I want to write up logic for parsing the
protobufs data files I receive
Error received
I get this error when running that file:
Steps taken to fix
Searched google for that error - didn't find much
Looked at this question/answer Why do I see "cannot import name descriptor_pb2" error when using Google Protocol Buffers?
I don't really understand the above questions selected answer,I tried to run the command in the above answer protoc descriptor.proto --python_out=gen/ by coping and pasting it in the terminal in different places but couldn't get it to work
Question
How do I fix this error?
What is the underlying cause?
How do I check if the rest of my protobuf python compiler/classes are set up correctly?
I've discovered the issue. I had not run the python install instructions the first time I tried to compile this file. I recompiled the file and this issue was fixed.