We Keep Coding

Pandas Import : ModuleNotFoundError: No module named 'pandas._libs.tslib'

Whenever I try to import pandas, whether inside a virtualenv or otherwise I am always getting this error.
Python 3.6.2 |Anaconda custom (64-bit)| (default, Sep 19 2017, 08:03:39) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
Traceback (most recent call last):
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 26, in <module>
from pandas._libs import (hashtable as _hashtable,
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\_libs\__init__.py", line 3, in <module>
from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
ModuleNotFoundError: No module named 'pandas._libs.tslib'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I tried the follwoing solutions:
Cloning pandas from git and running SETUP.py (on an instance of python 3.6 installed directly into my win10 os)
Using anaconda as python distribution and conda to install pandas
Updating microsoft visual c++ 2017 redistributable
Updating C:\ProgramData\Anaconda3\Lib\site-packages\PyInstaller\hooks\hook-pandas.py
None of these seem to work. Please help me understand what the issue here is.

This exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first
suggests that pandas was not built properly during installation.
The latter sentence:
If you want to import pandas from the source directory, you may need to run python setup.py build_ext --inplace --force to build the C extensions first
Is really only something you ought to be doing if you are contributing to pandas source code (e.g. to fix a pandas bug or add a feature to pandas itself) to the pandas-dev github repository*.
Most likely you shouldn't be building from source in your project.
Generally anaconda is pretty good at installing pandas correctly, and so my guess/comment was:
My guess is the virtual env is not using anaconda, and the install of pandas is messed up (perhaps created before installing anaconda?). I would delete this directory C:\Users\ishan\AppData\Roaming\Python\Python36 and see if that helps
The reason I suggested that directory was because it was in the error message AND it doesn't look like somewhere I expect anaconda's installation of pandas to be (either generally or as a virtualenv).
* Note: this is something fun to do, to give back to the pandas community: there's some low-hanging fruit, typos or code changes, so I recommend investigating whether there's any way you can contribute.

If you are using a Conda distribution (e.g., AnaConda, MiniConda), as it seems to be the case, uninstalling and reinstalling Pandas may help.
Run the following commands on the cmd console:
conda uninstall pandas
conda install pandas

I run into same error when setting up python, keras and anything between.
Background:
I installed anaconda and followed instructions by https://www.youtube.com/watch?v=z0qhKP2liHs and instruction to downgrade to python 3.6 by http://docs.anaconda.com/anaconda/user-guide/faq/#how-do-i-get-the-latest-anaconda-with-python-3-5
Running from Jupyter I run into same problem as author
I was able to solve my problem by:
- uploading pandas version for python 3.6 per https://docs.anaconda.com/anaconda/packages/py3.6_win-64/
then I run python from command line .. it worked
then I tested with PyCharm .. it worked
Appears that either Anaconda&Jupyter combination did not work or selecting pandas version did the job.

since tslib has been deprecated for the latest version of pandas. try to remove the pd.tslib.Dataframe and replace with pd.DataFrame where ever tslib is present in ggplot library. it works !!
you can find the packages in the lib file of ggplot folder.
Thank you!

Not sure if this is anything like an exhaustive answer, but seems related.
I came by this question as I was building Python from source (were you too perhaps?) and using that then to build an installer for my app - at some point I managed to get the exception from the OP:
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
What I then did was trying to import the module in question from a newly opened Python REPL (i.e. import pandas._libs.tslib). It turned out to be a useful effort as what I there got was an error likely about a missing _bz2.
By then I had already been through installing a good number of Linux packages that needed to be present prior to running the Python's ./configure so that they get included (for pip to operate well, for instance), and it seemed obvious that I just missed yet another package.
Indeed, as found to be suggested here in response to that error, a simple
sudo apt-get install libbz2-dev
and re-running ./configure, make, make install put me into a situation when finally pandas was willing to be loaded.
However, now it complains for the missing lzma extension :) such is life:
UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
From my perspective this finally seems like a soft error (app starts yay!)
My speculative explanation is that due to the lack of bzip2 available for Python at the time building and then this being missing, Pandas gives a slightly misleading error and fails to load, whether or not it correctly assesses that to be a show-stopper error. Possibly it's such a rare situation that from a time when it was commonplace/intuitive to fix, they may have stopped maintaining it and now it isn't telly enough?
Update: in case anyone wondered, the lzma dependency warning can be alleviated via sudo apt-get install liblzma-dev see UserWarning: Could not import the lzma module. Your installed Python
is incomplete and rebuilding in my case Python and then the PyInstaller packaged app, by the way somewhere along the sequence this includes reinstalling Pandas.

Fedora: NO module named RPM when using YUM

I removed numpy and scipy contents from the system's Python 2.6.4 in order to install different versions for them and now I broke YUM. The error I get is the following:
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
No module named rpm
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.6.4 (r264:75706, Jun 4 2010, 18:20:31)
[GCC 4.4.4 20100503 (Red Hat 4.4.4-2)]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://yum.baseurl.org/wiki/Faq
Also when I import YUM from inside the python console I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 23, in <module>
import rpm ImportError:
No module named rpm
Python is working fine and I managed to install numpy however I want to install scipy using YUM and I can't. Also when I run:
rpm --version
I get: RPM version 4.8.1.
Is there a way to solve this issue? I would be grateful for any advice or pointers towards solving this issue.

It looks like you don't have rpm-python installed. If yum was previously working, then it is difficult to see how the rpm module is not installed. Take a look in /usr/lib/python2.6/site-packages/rpm - that directory should exist, and should contain a couple of python files (__init__.py and transaction.py) and a shared library for rpm.
If the /usr/lib/python2.6/site-packages/rpm directory or the files are missing then you could try to install it. First check whether it is in the RPM database using rpm -q rpm-python. If it not installed then you can get the rpm from somewhere like this mirror and install it (rpm -ivh http://mirror.as24220.net/pub/fedora/linux/releases/20/Fedora/i386/os/Packages/r/rpm-python-4.11.1-7.fc20.i686.rpm). You will need to find the correct version for your Fedora and machine architecture.
More likely it will be (apparently) installed, in which case you could try force a reinstallation. Just be certain that you have access to the correct rpm of the same identical version, download it to be sure. If you are confident, then install with rpm -ivh --force rpm_file.
Then run python and see if you can import rpm and import yum.

how to install lpsolve module for Python on Linux Ubuntu 10.04?

I am using Ubuntu 12.04 64-bit, and started learning python today.(I tried to install a pirate version of MATLAB but failed...)
I have a linear programming problem to solve, and I want to use lp_solve module for Python.
I tried for 1~2 hours to find the download file and install the module.
I am not sure if I downloaded a right thing, and I could not install it until now.
How can I install this?
There is no download link in http://lpsolve.sourceforge.net/, and it tells me to run a command
python setup.py install
but there is no setup.py file in anywhere, including the lpsolve source file I downloaded somewhere.
If you know where to download it, and install it, could you teach me how to do them, step by step?
I am not sure about the version of my Python.
Thank you.

Adding few more details to the answer provided by dnozay.
Download the following two files from http://sourceforge.net/projects/lpsolve/files/lpsolve/
lp_solve_5.5.2.0_dev_ux64.tar.gz - contains the .so files
lp_solve_5.5.2.0_Python2.5_exe_ux64.tar.gz - contains the python wrapper scripts for lpsolver, which helps to invoke the native library from .so files.
Unzip the above downloaded files, where each directory formed by unzip will have an lpsolve55.so file, though at different locations.
Specify the paths to lpsolve55.so file in each directory by setting the following two environment variables:
export LD_LIBRARY_PATH=/usr/local/lib:/home/xxx/lp_solve_dev/
export PYTHONPATH=/home/xxx/usr/lib/python2.5/site-packages
To test if lpsolver is configured as expected :
[xx-xxxx#ip-xx-x-x-xx ~]$ python
>>>Python 2.7.9 (default, Apr 1 2015, 18:18:03)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>from lpsolve55 import *
>>>lpsolve()
lpsolve Python Interface version 5.5.0.9
using lpsolve version 5.5.2.0
Usage: ret = lpsolve('functionname', arg1, arg2, ...)
P.S.: make sure you have installed python-dev (if not, type sudo apt-get install python-dev at the command line) before you do this all.

The download link is:
http://sourceforge.net/projects/lpsolve/, or
http://sourceforge.net/projects/lpsolve/files/lpsolve/ for the files tab.
Once you have it installed, you may need to tweak your PYTHONPATH.
You also may want to look into cvexp:
http://pypi.python.org/pypi/cvexp

Too many different Python versions on my system and causing problems

During the past years, I have installed many Python libraries with various Python versions. To make them ready to work immediately, I installed them blindly without control. Currently they're causing problems when I tried to install pynest which invokes numpy, scipy and matplotlib. After struggling, I am going to clean and reinstall Python and the libraries.
After investigation, I found Python 2.5/2.6/2.7/3.2 on my system, and each of them has some copies or other things at: (my OS == Mac OS X 10.7.5 Lion)
/Library/Frameworks/
/opt/local/Library/Frameworks/
/opt/local/bin/
/Applications/
/usr/local/bin/
/usr/bin/
/System/Library/Frameworks/
I know I'm crazy to have these. Now I have removed all these except the things in /System/Libarary/Frameworks (I never remove any thing from /System/Library/). After the clean work, which python now gives /usr/bin/python which links to /System/Library/Frameworks.
Now, is it a clear environment for me to reinstall python? How to double check that there's no other versions existing? How should I reinstall them to guarantee that they and their libraries won't be everywhere and have many copies again?
I want to install a clean Python 2.7 onto a proper location, and make my system know exactly where it is and never install any libraries somewhere else. Please give me some advice that how to manage it like in a professional way.
For your information, here is my current $PATH, I think it should be modified:
/opt/local/bin:/opt/local/sbin:/opt/nest/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/django/bin:/usr/X11/bin:/opt/local/bin:/opt/local/sbin:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.2/bin
Please let me know If you need more information. Thank you!
UPDATE:
I'm rethinking profoudly why it becomes so crazy. I believe it's because I installed things via:
easy_install / macports / homebrew / fink / pip sometimes;
.dmg sometimes;
.pkg sometimes;
compile source code sometimes;
and they made things at different locations. I wonder what's the mechanism behind these ways? How do they choose target location? How to prevent them from messing things up?

Why did it get messed up?
There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.
What problem does it cause?
I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.
How to avoid this?
I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.
Unless you intend to, don't install the same thing twice via different
ways. (If you intend to do it for python, you might want to check out virtualenv)
Keep an eye on the path order in your PATH and consider if it's
correct.
When installing modules, be clear which python (or pip) is
running and where the module is installed.
So, how did I solve my own case?
Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.

Here is what was confusing me and how I solved it.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ ls /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory
So notice I didn't have a HomeBrew installation of python2.7, but did have the python3 installation. The version under /usr/bin/python is using the system default. You can tell based on the module search path:
$ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
`enter code here`Type "help", "copyright", "credits" or "license" for
more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.7/...
Notice the '/Library/Python'... that's Mac OS's version of python. But I want to stay strictly on a user installed version (i.e. HomeBrew).
So here's what I did to fix this:
$ brew install python
...
Warning: python 2.7.13 is already installed, it's just not linked.
You can use `brew link python` to link this version.
$ brew link --overwrite python
$ which python
/usr/local/bin/python
$ python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/2.7.13...
Its no longer /Library/.. but /usr/local.
Now its finding all of my pip installed modules! Problem solved!
UPDATE:
After updating brew to version 1.5.4, it seems the symbolic links were removed. And now you have to add this to your path:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Read the Caveats section in 'brew info python':
==> Caveats
This formula installs a python2 executable to /usr/local/bin.
If you wish to have this formula's python executable in your PATH then add
the following to ~/.bash_profile:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Pip and setuptools have been installed. To update them
pip2 install --upgrade pip setuptools
You can install Python packages with
pip2 install <package>
They will install into the site-package directory
/usr/local/lib/python2.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python.html

In order to install a python distributions into specific folder, you can use the --prefix scheme during python installation. Using the prefix scheme, you can for example install Python 2.7 into the folder /opt/py27. Now, in order to use the new installed Python distribution you have to: cleanup you PATH and LD_LIBRARY_PATH:
Remove all 'old' Python paths and
configure (according to my example) the environment variables like this:
PATH: Add /opt/py27/bin
LD_LIBRARY_PATH: Add /opt/py27/lib
That's it.
(In case you need multiple environments of Python installed at the same time, I'd suggest to have a look at virtualenv)

tl;dr
brew install python
Symptoms
I had similar issues with python programs not finding dependencies.
My python3 version was a broken symlink.
My pip was pointing to a python 3.8
And my pip3 was pointing to 3.9
python -V was outputting some python 2.7 version
python3 -V was outputting some python3.8 version
Solution
I ran brew install python and it fixed all my problems.

Python and Yum issue CentOS 5.4

I'm new to Linux/CentOS and I'm not having the most fun using it. I was developing something locally on my Mac using python 2.7 when I ran it on the server I ran into issues which I put down to being the outdated version of Python (2.4.3) I then updated it, which wasn't a very good idea. I built it from source and then ran into issues, Yum and other modules disappeared.
I'm not back using Python 2.4.3 but I still don't have Yum back.
[root#issues ~]# yum Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback There was a problem importing one of the
Python modules required to run yum. The error leading to this problem
was:
No module named yum
Please install a package which provides this module, or verify that
the module is installed correctly.
It's possible that the above module doesn't match the current version
of Python, which is:
2.4.3 (#1, Jun 18 2012, 08:55:31) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
If you cannot solve this problem yourself, please go to the yum faq
at: http://wiki.linux.duke.edu/YumFaq
Searching loads of forums everyone says that it's because of a different version of Python, this is no longer the case for me, I think it's something to do with Yum being mapped to a different directory.
I found this command somewhere else and ran this:
[root#issues ~]# rpm -V python
missing /usr/bin/python2
missing /usr/bin/python2.4
Does anyone know what to do or where to go for help? I really need to get this fixed as soon as possible!
Thanks
Actually just to add, If I run python I get this output.
[root#issues ~]# python
Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>

Yes on Linux systems, it is not a good idea to upgrade default python which is 2.4.x version and Linux package management and many other tools require it to function correctly. It is always better to install other python versions separately and use it directly (like: /usr/bin/python2.7)
Python executable itself is not working correctly as it is not able to load default module 'site' during start-up.
The python installation is broken.
Since you can not use "Yum" or "RPM" to bootstrap your packages, I suggest that you use your "OS" installation CD or ISO image to install the rpm/yum packages directly. It will fix the python installation as dependency.
I had gone through this issue and after various trial, this was only way I could fix it.
[Edit:]
See: Python 2.x multiple version issues regarding PYTHONPATH
Also the error is that it is not able to find the package directory.
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
See the paths that I have. Check out the path of packages and can you set "PYTHONHOME".
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/lib64/python2.4/site.pyc matches /usr/lib64/python2.4/site.py
import site # precompiled from /usr/lib64/python2.4/site.pyc
# /usr/lib64/python2.4/os.pyc matches /usr/lib64/python2.4/os.py
import os # precompiled from /usr/lib64/python2.4/os.pyc
If you are not able to locate similar package location, can you use RPM to directly install python2.4 again
rpm --force -hUv url_for_python_rpm