Whenever I try to import pandas, whether inside a virtualenv or otherwise I am always getting this error.
Python 3.6.2 |Anaconda custom (64-bit)| (default, Sep 19 2017, 08:03:39) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
Traceback (most recent call last):
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 26, in <module>
from pandas._libs import (hashtable as _hashtable,
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\_libs\__init__.py", line 3, in <module>
from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
ModuleNotFoundError: No module named 'pandas._libs.tslib'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I tried the follwoing solutions:
Cloning pandas from git and running SETUP.py (on an instance of python 3.6 installed directly into my win10 os)
Using anaconda as python distribution and conda to install pandas
Updating microsoft visual c++ 2017 redistributable
Updating C:\ProgramData\Anaconda3\Lib\site-packages\PyInstaller\hooks\hook-pandas.py
None of these seem to work. Please help me understand what the issue here is.
This exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first
suggests that pandas was not built properly during installation.
The latter sentence:
If you want to import pandas from the source directory, you may need to run python setup.py build_ext --inplace --force to build the C extensions first
Is really only something you ought to be doing if you are contributing to pandas source code (e.g. to fix a pandas bug or add a feature to pandas itself) to the pandas-dev github repository*.
Most likely you shouldn't be building from source in your project.
Generally anaconda is pretty good at installing pandas correctly, and so my guess/comment was:
My guess is the virtual env is not using anaconda, and the install of pandas is messed up (perhaps created before installing anaconda?). I would delete this directory C:\Users\ishan\AppData\Roaming\Python\Python36 and see if that helps
The reason I suggested that directory was because it was in the error message AND it doesn't look like somewhere I expect anaconda's installation of pandas to be (either generally or as a virtualenv).
* Note: this is something fun to do, to give back to the pandas community: there's some low-hanging fruit, typos or code changes, so I recommend investigating whether there's any way you can contribute.
If you are using a Conda distribution (e.g., AnaConda, MiniConda), as it seems to be the case, uninstalling and reinstalling Pandas may help.
Run the following commands on the cmd console:
conda uninstall pandas
conda install pandas
I run into same error when setting up python, keras and anything between.
Background:
I installed anaconda and followed instructions by https://www.youtube.com/watch?v=z0qhKP2liHs and instruction to downgrade to python 3.6 by http://docs.anaconda.com/anaconda/user-guide/faq/#how-do-i-get-the-latest-anaconda-with-python-3-5
Running from Jupyter I run into same problem as author
I was able to solve my problem by:
- uploading pandas version for python 3.6 per https://docs.anaconda.com/anaconda/packages/py3.6_win-64/
then I run python from command line .. it worked
then I tested with PyCharm .. it worked
Appears that either Anaconda&Jupyter combination did not work or selecting pandas version did the job.
since tslib has been deprecated for the latest version of pandas. try to remove the pd.tslib.Dataframe and replace with pd.DataFrame where ever tslib is present in ggplot library. it works !!
you can find the packages in the lib file of ggplot folder.
Thank you!
Not sure if this is anything like an exhaustive answer, but seems related.
I came by this question as I was building Python from source (were you too perhaps?) and using that then to build an installer for my app - at some point I managed to get the exception from the OP:
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
What I then did was trying to import the module in question from a newly opened Python REPL (i.e. import pandas._libs.tslib). It turned out to be a useful effort as what I there got was an error likely about a missing _bz2.
By then I had already been through installing a good number of Linux packages that needed to be present prior to running the Python's ./configure so that they get included (for pip to operate well, for instance), and it seemed obvious that I just missed yet another package.
Indeed, as found to be suggested here in response to that error, a simple
sudo apt-get install libbz2-dev
and re-running ./configure, make, make install put me into a situation when finally pandas was willing to be loaded.
However, now it complains for the missing lzma extension :) such is life:
UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
From my perspective this finally seems like a soft error (app starts yay!)
My speculative explanation is that due to the lack of bzip2 available for Python at the time building and then this being missing, Pandas gives a slightly misleading error and fails to load, whether or not it correctly assesses that to be a show-stopper error. Possibly it's such a rare situation that from a time when it was commonplace/intuitive to fix, they may have stopped maintaining it and now it isn't telly enough?
Update: in case anyone wondered, the lzma dependency warning can be alleviated via sudo apt-get install liblzma-dev see UserWarning: Could not import the lzma module. Your installed Python
is incomplete and rebuilding in my case Python and then the PyInstaller packaged app, by the way somewhere along the sequence this includes reinstalling Pandas.
Related
Below is simple Line of code to try NumPy
import numpy as np
my_list = [1, 2, 3]
print(my_list)
print(type(my_list))
my_list_arr = np.array(my_list) # Here we are converting python list to numpy array
print(my_list_arr)
print(type(my_list_arr))
My question is: when I execute above lines of code using Jupyter-Notebook , it works well without any package issues BUT Why same line of code throwing NumPy , pandas and other package dependencies related issues when execute on PyCharm as .py file
FYI,
my setup of PyCharm project as;
python interpreter --> pointed to Anaconda provided
created PyCharm virtual environment - venv
Inherit-global-site-packages – Yes
Please help me. I am fed up with installing packages using conda and tired of doing package corrections one by one as and when ERRORs comes one after another package.
Any Better Solution please !!
Below is Error by Pycharm
D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\venv\Scripts\python.exe D:/Mytech/IDE_workspace/PycharmProjects/python_anaconda_base_venv_and_global_site_packages/python_for_data_science/my_NumPyArrays_practice.py
C:\Users\DELL\anaconda3\lib\site-packages\numpy\__init__.py:143: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
Traceback (most recent call last):
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
from . import multiarray
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
from . import overrides
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\python_for_data_science\my_NumPyArrays_practice.py", line 1, in <module>
import numpy as np
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\__init__.py", line 145, in <module>
from . import core
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\venv\Scripts\python.exe"
* The NumPy version is: "1.20.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
Process finished with exit code 1
but when i am executing below code it showing. it working.
import sys
print("******************************")
print("This Code using:")
print("Python Version =", sys.version)
print("Python Interpreter =", sys.base_prefix)
print("*******************************")
print("*******************************")
This Code using:
Python Version = 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)]
Python Interpreter = C:\Users\DELL\anaconda3
print("*******************************")
Process finished with exit code 0
As explained in this answer, the inherit-global-site-packages inherits from your Global python install. This is the install that comes with your machine or is installed "directly" into /usr/bin or the Windows equivalent. I don't think pointing the interpreter to Anaconda will make it inherit from Anaconda, because Anaconda is not considered a global python install.
What is happening is you are inheriting from a python install that you probably did not install anything to. I can think of two solutions:
Install the packages you want in your global python install (running which python or which python3 outside of a conda env should show you where that is; after finding the command that calls the global python, pip install using that command)
Use conda environments, and just set your interpreter to different conda environments, instead of making a PyCharm env. If you are not familiar with this, you can run conda create -n NAME python=X.XX where NAME is what you want to name the environment and X.XX is the version number. From there, you can conda activate NAME, and then either pip or conda install your packages.
I am currently following a beginners introduction to machine learning.
While entering in the command:
import pandas as pd in the python shell in terminal, I get an error reading:
ImportError: Missing required dependencies ['numpy'].
I already looked at the other similar question, tried that solution, but still received the same error.
Looks like you might be running on a Mac and perhaps using the default system python. For whatever reason you don't have a complete installation. you have pandas but not numpy. I'm not sure which packages the tutorial you are following uses, but I would recommend installing the Anaconda python distribution as it includes pandas, all its dependencies and much more, including the scikit-learn package often used for machine learning.
If you want to know more about installing a Python environment for machine learning on a Mac, there is a good tutorial on machinelearningmastery.com.
This doesn't have anything to do with incompatibility. As #Peter mentioned, you simply don't have NumPy and should install through Anaconda. Here is the code within pandas that is giving you the error:
# Let users know if they're missing any of our hard dependencies
hard_dependencies = ("numpy", "pytz", "dateutil")
missing_dependencies = []
for dependency in hard_dependencies:
try:
__import__(dependency)
except ImportError as e:
missing_dependencies.append(dependency)
if missing_dependencies:
raise ImportError("Missing required dependencies {0}".format(missing_dependencies))
del hard_dependencies, dependency, missing_dependencies
Notice there is nothing here about version.
I had a same problem. I don't know what is the cause of the problem, but it seems to deal with how numpy is installed. You can try the following:
Install pandas
Uninstall numpy
Download numpy whl for your needs from here
Install numpy from downloaded whl
That worked for me!
I get the same error message with my Anaconda installation when I forget to activate the environment:
Testcode: import_pandas.py:
import pandas
print('Pandas import succeeded!')
Run import_pandas.py with ImportError:
Microsoft Windows [Version 10.0.16299.1146]
(c) 2017 Microsoft Corporation. All rights reserved.
C:\Users\peter\demo>python import_pandas.py
Traceback (most recent call last):
File "import_pandas.py", line 1, in <module>
import pandas
File "C:\Users\peter\AppData\Local\Anaconda3\lib\site-packages\pandas\__init__.py", line 19, in <module>
"Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['numpy']
However, after activating conda everything works perfectly fine:
C:\Users\peter\demo>activate
C:\Users\peter\demo>conda.bat activate
(base) C:\Users\peter\demo>python import_pandas.py
Pandas import succeeded!
I downloaded fiona today. when I try to import it in Python using 'import fiona', I get the following error:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import fiona
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 72, in <module>
from fiona.collection import Collection, supported_drivers, vsi_path
File "C:\Python27\lib\site-packages\fiona\collection.py", line 7, in <module>
from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: No module named ogrext
I checked in my site-packages folder, and ogrext is a "C" file. I tried commenting out the import to see if it wasn't necessary, but this of course threw another error.
Specifically, how do I resolve this import error?
More generally, how does one resolve errors involving importing C files into a python library?
You can't just install any module by copying all the files to site-packages. Some modules are pure Python, but there are many with extensions written in other languages (C, C++, Fortran, etc.) that need to be compiled and linked into libraries before being used, and fiona is one of them. This compilation can be done at several stages - by the author, before distributing the module as a wheel, during the pip install process, or by downloading the package's source, unzipping/tarring it, and running python setup.py install. Unfortunately, Windows doesn't come with a compiler by default, so you either need to install and configure your system for gcc or Visual Studio, or use another method, such as a precompiled installer. Fortunately, fiona is available from Christoph Gohlke's Python Extension Packages for Windows Repository here. Download the installer for your version and bit-ness of Python, delete the fiona folder in site-packages, then run the installer. This site contains a large number of packages for scientific computing, and is my go-to resource when I need to install a new module, especially if it has extensions.
EDIT
Upon further inspection, it appears that fiona also requires the GDAL module, as well as six, both of which can be downloaded from Gohlke's repository. I first installed fiona only (I already had six installed), and got a missing DLL error. I then installed GDAL, and import fiona worked just fine - I'm not familiar with the module, so I didn't do any further testing, but hopefully everything should work now.
I removed numpy and scipy contents from the system's Python 2.6.4 in order to install different versions for them and now I broke YUM. The error I get is the following:
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
No module named rpm
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.6.4 (r264:75706, Jun 4 2010, 18:20:31)
[GCC 4.4.4 20100503 (Red Hat 4.4.4-2)]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://yum.baseurl.org/wiki/Faq
Also when I import YUM from inside the python console I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 23, in <module>
import rpm ImportError:
No module named rpm
Python is working fine and I managed to install numpy however I want to install scipy using YUM and I can't. Also when I run:
rpm --version
I get: RPM version 4.8.1.
Is there a way to solve this issue? I would be grateful for any advice or pointers towards solving this issue.
It looks like you don't have rpm-python installed. If yum was previously working, then it is difficult to see how the rpm module is not installed. Take a look in /usr/lib/python2.6/site-packages/rpm - that directory should exist, and should contain a couple of python files (__init__.py and transaction.py) and a shared library for rpm.
If the /usr/lib/python2.6/site-packages/rpm directory or the files are missing then you could try to install it. First check whether it is in the RPM database using rpm -q rpm-python. If it not installed then you can get the rpm from somewhere like this mirror and install it (rpm -ivh http://mirror.as24220.net/pub/fedora/linux/releases/20/Fedora/i386/os/Packages/r/rpm-python-4.11.1-7.fc20.i686.rpm). You will need to find the correct version for your Fedora and machine architecture.
More likely it will be (apparently) installed, in which case you could try force a reinstallation. Just be certain that you have access to the correct rpm of the same identical version, download it to be sure. If you are confident, then install with rpm -ivh --force rpm_file.
Then run python and see if you can import rpm and import yum.
I console access to a computer where I do not have root nor sudo rights.
Python version is 2.5.2 and numpy is not available. I cannot use python setup.py install --user nor there are any compilers available on the machine.
Can I somehow use the compiled packages available https://edge.launchpad.net/~scipy/+archive/ppa/+packages without installing them? I tried importing the numpy module directly but it complains:
Python 2.5.2 (r252:60911, Jan 4 2009, 21:59:32)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/home/XXX/temp/python-numpy-1.2.1/numpy/__init__.py", line 121,
in <module>
raise ImportError(msg)
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python intepreter from there.
>>>
Thanks!
Update:
The sysadmin will not install any kind of software in the machine (it's a VPS and my guess is that they have a standard image for deployment). They are crazy paranoic, they won't even tell me what flavor of unix they are running, and even the Apache service has the version number obfuscated! This is all the information I get upon login:
Linux server1 2.4.22 #4 SMP Wed Nov 5 17:44:16 CET 2003 i686 unknown
And for compiling:
python setup.py install --home=~
[...]
RuntimeError: Broken toolchain: cannot link a simple C program
cat /proc/version
Linux version 2.6.32.25-grsec-dh (root#dl345.dinaserver.com) (gcc version 4.3.2
(Debian 4.3.2-1.1) ) #2 SMP Wed Nov 3 13:21:01 CET 2010
If you can resolve all the dependencies, you might be able to install it in your $HOME using dpkg. dpkg doesn't resolve dependencies automatically, so you might have to figure out the right order to install the packages in. Download the .deb files that you're interested in and run the following command for each package:
$ dpkg -i --force-not-root --root=$HOME mypackagename.deb
If you then add the directory with the newly installed Numpy to your $PYTHONPATH, or to sys.path, Numpy might just work.
Alternatively, you might be able to extract the files you need from one of the other binary distributions of Numpy around (such as Sage).
Numpy is quite fussy about what versions of its dependencies it requires though, so you're probably best off downloading the packages for the specific version of Linux that you're using.
Finally, consider asking your administrator whether s/he'll install Numpy for you. You'd be surprised how often a simple request can solve all your problems, especially since it's just one apt-get command.
EDIT: Just as an alternative, if you can get access to another machine running the same version/architecture of Ubuntu/Debian, you might be able to download the numpy source tarball, compile with python setup.py build and then just copy everything in directory_where_you_extracted_the_tarball/build/numpy/lib.OS-arch-PythonVersion (on my system, it is lib.linux-x86_64-2.6/) to a directory of your choice on the target machine. Then, just add that directory to your $PYTHONPATH and you're done. Remember to copy the contents, not the whole directory (tar -jcf np.tar.bz2 /path/to/numpy/build/numpy/lib.OS-arch-PythonVersion/numpy then get the tar.bz2 to the remote machine and extract it in a directory of your choice).
There is some documentation on how to use setuptools here: http://docs.python.org/install/index.html#how-installation-works
Building Numpy by hand is not for the faint of heart though, so this might lead to a lot of head-banging and hair-tearing.
I'm not 100% this will work, but Enthought has a free version of the EPD that has numpy and scipy included, that may not require a compiler to install (since it's just installing binaries as far as I can tell), and doesn't need root access:
http://www.enthought.com/products/epd_free.php
You could try setting up a virtualenv environment on a similar machine with similar architecture. Then install virtualenv locally on the VPS machine and try copying the environment there.
You can use python's distutils (which is what python setup.py runs) install to a local directory, which must be added to your PYTHONPATH. E.G.,
python setup.py install --prefix=~/local
which uses a directory hierarchy ~/local/lib/python2.x. (Or you can use --home=<dir> to avoid the python2.x part)