We Keep Coding

Why NumPy and pandas packages are always problematic when setting up first time Python interpreter as Anaconda provided

Below is simple Line of code to try NumPy
import numpy as np
my_list = [1, 2, 3]
print(my_list)
print(type(my_list))
my_list_arr = np.array(my_list) # Here we are converting python list to numpy array
print(my_list_arr)
print(type(my_list_arr))
My question is: when I execute above lines of code using Jupyter-Notebook , it works well without any package issues BUT Why same line of code throwing NumPy , pandas and other package dependencies related issues when execute on PyCharm as .py file
FYI,
my setup of PyCharm project as;
python interpreter --> pointed to Anaconda provided
created PyCharm virtual environment - venv
Inherit-global-site-packages – Yes
Please help me. I am fed up with installing packages using conda and tired of doing package corrections one by one as and when ERRORs comes one after another package.
Any Better Solution please !!
Below is Error by Pycharm
D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\venv\Scripts\python.exe D:/Mytech/IDE_workspace/PycharmProjects/python_anaconda_base_venv_and_global_site_packages/python_for_data_science/my_NumPyArrays_practice.py
C:\Users\DELL\anaconda3\lib\site-packages\numpy\__init__.py:143: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
Traceback (most recent call last):
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
from . import multiarray
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
from . import overrides
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\python_for_data_science\my_NumPyArrays_practice.py", line 1, in <module>
import numpy as np
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\__init__.py", line 145, in <module>
from . import core
File "C:\Users\DELL\anaconda3\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "D:\Mytech\IDE_workspace\PycharmProjects\python_anaconda_base_venv_and_global_site_packages\venv\Scripts\python.exe"
* The NumPy version is: "1.20.3"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
Process finished with exit code 1
but when i am executing below code it showing. it working.
import sys
print("******************************")
print("This Code using:")
print("Python Version =", sys.version)
print("Python Interpreter =", sys.base_prefix)
print("*******************************")
print("*******************************")
This Code using:
Python Version = 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)]
Python Interpreter = C:\Users\DELL\anaconda3
print("*******************************")
Process finished with exit code 0

As explained in this answer, the inherit-global-site-packages inherits from your Global python install. This is the install that comes with your machine or is installed "directly" into /usr/bin or the Windows equivalent. I don't think pointing the interpreter to Anaconda will make it inherit from Anaconda, because Anaconda is not considered a global python install.
What is happening is you are inheriting from a python install that you probably did not install anything to. I can think of two solutions:
Install the packages you want in your global python install (running which python or which python3 outside of a conda env should show you where that is; after finding the command that calls the global python, pip install using that command)
Use conda environments, and just set your interpreter to different conda environments, instead of making a PyCharm env. If you are not familiar with this, you can run conda create -n NAME python=X.XX where NAME is what you want to name the environment and X.XX is the version number. From there, you can conda activate NAME, and then either pip or conda install your packages.

Running numpy using anaconda and VS Code

I am trying to get VScode to work with anaconda but having issues with numpy. I have managed to get VScode to use the right python environment From VScode (ctrl+shift+P, type Python:Select Interpreter and select the appropriate option).
However, when I type "import numpy" in to my script, I get the following error message:
C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\__init__.py:140: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\[NAME]\Documents\python\sandpit\hello.py", line 2, in <module>
import numpy
File "C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\ProgramData\Anaconda3\envs\sandpit\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.8 from "C:\ProgramData\Anaconda3\envs\sandpit\python.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
I have successfully called the 'import numpy' command from the anaconda command prompt so it seems the issue is just with VS-Code?
I have also tried to reinstall numpy from the environment with the following command conda install numpy --force-reinstall but this has no effect?

Try launching VS Code from your activated conda environment. Errors like this typically come up if you managed to run code before your shell had a chance to do a conda environment activation.

There is one step more
Launch Anaconda 3 powershell
conda init cmd.exe #to allow cmd to run VScode
Lauch VScode by the hitting the command code.

Numpy import failed

On an RPi2, I upgraded all my packages with pip-review and numpy does not work anymore.
I tried to uninstall and reinstall numpy though pip and apt but I'm getting no luck.
Apt installs numpy 1.12.1, if I try to install that it does not install through pip.
Pip installs numpy 1.16.4 fine, but when I go to run it I get the error below.
I have already run: sudo apt install libc6 libatlas-base-dev
Any help is appreciated.
Python 3.7.3 (default, May 8 2019, 18:07:21)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/numpy/core/__init__.py", line 40, in <module>
from . import multiarray
File "/usr/local/lib/python3.7/site-packages/numpy/core/multiarray.py", line 12, in <module>
from . import overrides
File "/usr/local/lib/python3.7/site-packages/numpy/core/overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ImportError: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIBC_2.27' not found (required by /usr/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-arm-linux-gnueabihf.so)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/numpy/__init__.py", line 142, in <module>
from . import core
File "/usr/local/lib/python3.7/site-packages/numpy/core/__init__.py", line 71, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using /usr/local/bin/python3),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIBC_2.27' not found (required by /usr/local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-arm-linux-gnueabihf.so)

pip install numpy --global-option="-mfloat-abi=hard" --force-reinstall was the only answer that worked for me . The explanation is given in a thread on a slightly different problem see this link

With Conda it is solved depending on the version, but must force reinstall.
# versions >= 4.6:
conda install numpy --force-reinstall
# versions < 4.6:
conda install numpy --force

Pandas Import : ModuleNotFoundError: No module named 'pandas._libs.tslib'

Whenever I try to import pandas, whether inside a virtualenv or otherwise I am always getting this error.
Python 3.6.2 |Anaconda custom (64-bit)| (default, Sep 19 2017, 08:03:39) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
Traceback (most recent call last):
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 26, in <module>
from pandas._libs import (hashtable as _hashtable,
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\_libs\__init__.py", line 3, in <module>
from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
ModuleNotFoundError: No module named 'pandas._libs.tslib'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
I tried the follwoing solutions:
Cloning pandas from git and running SETUP.py (on an instance of python 3.6 installed directly into my win10 os)
Using anaconda as python distribution and conda to install pandas
Updating microsoft visual c++ 2017 redistributable
Updating C:\ProgramData\Anaconda3\Lib\site-packages\PyInstaller\hooks\hook-pandas.py
None of these seem to work. Please help me understand what the issue here is.

This exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\ishan\AppData\Roaming\Python\Python36\site-packages\pandas\__init__.py", line 35, in <module>
"the C extensions first.".format(module))
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first
suggests that pandas was not built properly during installation.
The latter sentence:
If you want to import pandas from the source directory, you may need to run python setup.py build_ext --inplace --force to build the C extensions first
Is really only something you ought to be doing if you are contributing to pandas source code (e.g. to fix a pandas bug or add a feature to pandas itself) to the pandas-dev github repository*.
Most likely you shouldn't be building from source in your project.
Generally anaconda is pretty good at installing pandas correctly, and so my guess/comment was:
My guess is the virtual env is not using anaconda, and the install of pandas is messed up (perhaps created before installing anaconda?). I would delete this directory C:\Users\ishan\AppData\Roaming\Python\Python36 and see if that helps
The reason I suggested that directory was because it was in the error message AND it doesn't look like somewhere I expect anaconda's installation of pandas to be (either generally or as a virtualenv).
* Note: this is something fun to do, to give back to the pandas community: there's some low-hanging fruit, typos or code changes, so I recommend investigating whether there's any way you can contribute.

If you are using a Conda distribution (e.g., AnaConda, MiniConda), as it seems to be the case, uninstalling and reinstalling Pandas may help.
Run the following commands on the cmd console:
conda uninstall pandas
conda install pandas

I run into same error when setting up python, keras and anything between.
Background:
I installed anaconda and followed instructions by https://www.youtube.com/watch?v=z0qhKP2liHs and instruction to downgrade to python 3.6 by http://docs.anaconda.com/anaconda/user-guide/faq/#how-do-i-get-the-latest-anaconda-with-python-3-5
Running from Jupyter I run into same problem as author
I was able to solve my problem by:
- uploading pandas version for python 3.6 per https://docs.anaconda.com/anaconda/packages/py3.6_win-64/
then I run python from command line .. it worked
then I tested with PyCharm .. it worked
Appears that either Anaconda&Jupyter combination did not work or selecting pandas version did the job.

since tslib has been deprecated for the latest version of pandas. try to remove the pd.tslib.Dataframe and replace with pd.DataFrame where ever tslib is present in ggplot library. it works !!
you can find the packages in the lib file of ggplot folder.
Thank you!

Not sure if this is anything like an exhaustive answer, but seems related.
I came by this question as I was building Python from source (were you too perhaps?) and using that then to build an installer for my app - at some point I managed to get the exception from the OP:
ImportError: C extension: No module named 'pandas._libs.tslib' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
What I then did was trying to import the module in question from a newly opened Python REPL (i.e. import pandas._libs.tslib). It turned out to be a useful effort as what I there got was an error likely about a missing _bz2.
By then I had already been through installing a good number of Linux packages that needed to be present prior to running the Python's ./configure so that they get included (for pip to operate well, for instance), and it seemed obvious that I just missed yet another package.
Indeed, as found to be suggested here in response to that error, a simple
sudo apt-get install libbz2-dev
and re-running ./configure, make, make install put me into a situation when finally pandas was willing to be loaded.
However, now it complains for the missing lzma extension :) such is life:
UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
From my perspective this finally seems like a soft error (app starts yay!)
My speculative explanation is that due to the lack of bzip2 available for Python at the time building and then this being missing, Pandas gives a slightly misleading error and fails to load, whether or not it correctly assesses that to be a show-stopper error. Possibly it's such a rare situation that from a time when it was commonplace/intuitive to fix, they may have stopped maintaining it and now it isn't telly enough?
Update: in case anyone wondered, the lzma dependency warning can be alleviated via sudo apt-get install liblzma-dev see UserWarning: Could not import the lzma module. Your installed Python
is incomplete and rebuilding in my case Python and then the PyInstaller packaged app, by the way somewhere along the sequence this includes reinstalling Pandas.

Pycuda installation on Ubuntu

I'm having a difficult time installing pycuda. I am running on Ubuntu 12.04. I first installed the Enthought python distribution (even though I already had python on the computer). I added the enthought python location to the path in my .profile (and this works without issue, typing python or ipython from the command line correctly uses the enthought version). I then followed the instructions from http://wiki.tiker.net/PyCuda/Installation/Linux/Ubuntu (skipping steps 0-1 as I already had numpy and cuda installed).
Everything seems to run and install fine. But when I try to actually use pycuda, it fails. For example, here's what happens when I try to run the hello_gpu.py example:
:~/Downloads/pycuda-2012.1$ python examples/hello_gpu.py
Traceback (most recent call last):
File "examples/hello_gpu.py", line 1, in <module>
import pycuda.driver as drv
File "/usr/lib/python_enthought/lib/python2.7/site-packages/pycuda-2012.1-py2.7-linux- x86_64.egg/pycuda/driver.py", line 2, in <module>
from pycuda._driver import *
ImportError: /usr/lib/libboost_python-py27.so.1.46.1: undefined symbol:
PyUnicodeUCS4_FromEncodedObject
Alternatively, when I try to import the pycuda package from idle, I get a different error:
Python 2.7.3 |EPD 7.3-1 (64-bit)| (default, Apr 11 2012, 17:52:16)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "credits", "demo" or "enthought" for more information.
>>> import pycuda
>>> pycuda
<module 'pycuda' from 'pycuda/__init__.pyc'>
>>> import pycuda.driver as drv
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pycuda/driver.py", line 2, in <module>
from pycuda._driver import *
ImportError: No module named _driver
I suspect part of the issue is me somewhere not specifying the correct path to the python interpreter, but I cannot figure out where this error occurs. Any suggestions would be most welcome, I am out of ideas.

This problem would appear to have been caused by conflicts between two different Python installations on the same system. The OP apparently solved this by uninstalling an Enthough Python installation and using only the system Python installation. There might well be other ways to solve this using a non-system Python installation, as suggested here
[This answer was assembled from comments and added as a community wiki entry in the hope it will receive a vote and be removed from the unanswered list for the PyCUDA tag]