Pandas install-Importing the multiarray numpy extension - python

I'm trying to install pandas using pip3, but the install fails. I've reinstalled numpy as the error recommends, but it didn't help. A Google search failed to turn up a solution for me.
I have installed:
Python 3.6.5-1
numpy 1.14.4
I'm using "pip3 install pandas", and it's trying to install pandas 0.23.0. Here's the error:
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: Could not load module /QOpenSys/pkgs/lib/python3.6/site-packages/numpy/core/multiarray.so.
The local-exec model was used for thread-local
storage, but the module is not the main program.
Examine the .loader section header with the 'dump -Hv' command.

Related

Importing the numpy C-extensions fails when importing a package that depends on numpy

When I import a package I built that depends on numpy (via pandas), the numpy import fails with the message below. VS Code is not in the picture.
I'm running macOS Monterey (12.3.1) on a MacBook Pro with an M1 chip.
I installed Python 3.10 from binary.
I'm using virtualenv and VS Code for development, but the failure also occurs when I run a script from Terminal.
I built a package (PyTables) that uses pandas and numpy: PyTables. It runs just fine in that project on my machine.
However, when install that built package in another project -- pip install -i https://test.pypi.org/simple/ pytables -- and then import it -- import pytables as tp -- I get the numpy import error below.
I've isolated the problem to just that import statement.
I installed pandas 1.4.1 and numpy 1.22.1 in this virtualenv using pip3, just as I did in the env where I built the package.
I also have Python 3.9.1 on my system. I need 3.10 for PySimpleGUI.
If I simply comment out the import, the code runs fine.
Import error:
ImportError: Unable to import required dependencies: numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
The Python version is: Python3.10 from
"/Users/alecramsay/.virtualenvs/tstudio/bin/python" * The NumPy
version is: "1.22.1"
and make sure that they are the versions you expect. Please carefully
study the documentation linked above for further help.
Original error was:
dlopen(/Users/alecramsay/.virtualenvs/tstudio/lib/python3.10/site-packages/numpy/core/_multiarray_umath.cpython-310-darwin.so, 0x0002): tried:
'/Users/alecramsay/.virtualenvs/tstudio/lib/python3.10/site-packages/numpy/core/_multiarray_umath.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need
'x86_64'))

error while importing HDF5 and h5py module

I am trying to use predixcan software. In that one of the script utilise h5py module. When I run the script, and it gives the following error:
Could not import HDF5 expression INFO - Could not import h5py module
Does anyone know how to resolve this error?
Install h5py with pip install h5py, if you are using pip, or with conda install h5py if you are using Anaconda.

Numpy Import Error in Anaconda Environment using Spyder (WINDOWS)

BACKGROUND
Just two days ago I was able to run any program that had a numpy dependency. Now when I try to run my code using pandas, matplotlib or any module that depends on numpy, I get the below error:
Traceback (most recent call last):
File "<ipython-input-8-8fcf286af663>", line 7, in <module>
import numpy
File "path\to\Python\Python38\site-packages\numpy\__init__.py", line 140, in <module>
from . import core
File "path\to\Python\Python38\site-packages\numpy\core\__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.7 from "C:\ProgramData\Anaconda3\pythonw.exe"
* The NumPy version is: "1.19.5"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
SETTINGS CHECK
The return message references this site: https://numpy.org/devdocs/user/troubleshooting-importerror.html but nothing there has helped me.
I checked my Path variable to make sure that all the required directory locations are there and there is no issue.
I successfully import numpy when I compile code from cmd using python -c "import numpy; print('done')"
I am using an anaconda environment, the Spyder IDE, and again it was just fine two days ago.
I checked the python version from pythonw.exe and it seems fine as well
What could be wrong with my environment?
ATTEMPTED SOLUTIONS HERE
Also, I've checked the following links with no success:
Importing the numpy c-extensions failed
importing numpy package in Spyder, Python
python Spyder not importing numpy
https://github.com/numpy/numpy/issues/15090
Can't import numpy anaconda
Import error: Anaconda numpy (numpy and Anaconda already installed, virtualenv)
Turns out my problem was very simple. The main solution I was trying was to uninstall and reinstall.
Every time I installed and uninstalled the modules, I did so from the standard command line.
That was wrong since anaconda uses its own virtual environment to store data.
So all I had to do was run this command from the anaconda command prompt:
pip install --upgrade pandas && pip install --upgrade numpy
this command would work just as well:
pip uninstall pandas && pip uninstall numpy && pip install pandas
(since the last install would automatically download any dependencies that pandas has, which is numpy

I get `No module named _multiarray_umath` when using matplotlib

When I run my tests in CI, I get the following error:
ImportError while importing test module '/home/tests/test_process.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
...
.tox/py27/lib/python2.7/site-packages/matplotlib/pyplot.py:31: in <module>
import matplotlib.colorbar
.tox/py27/lib/python2.7/site-packages/matplotlib/colorbar.py:36: in <module>
import matplotlib.contour as contour
.tox/py27/lib/python2.7/site-packages/matplotlib/contour.py:14: in <module>
import matplotlib._contour as _contour
E ImportError: numpy.core.multiarray failed to import
----- Captured stderr -----
ImportError: No module named _multiarray_umath
What's going on here? I haven't made any changes to my code, but all the sudden
my build started failing.
Solution
Install numpy using pip seperately, before installing your sdist.
For tox, add numpy directly to your deps array.
Why did this happen?
Numpy recently published numpy-1.16.0rc2 to pypy, which is what (in conjunction with a bug/oversight in easy_install) broke your build:
pip knows not to install RCs by default, but easy_install (which matplotlib uses to do their builds) does not. If you were to do sdist with a whole bunch of -vvvvvvs, you'd see something like this:
gcc ... -I/tmp/pip-install-Eh8d9d/matplotlib/.eggs/numpy-1.16.0rc2-py2.7-linux-x86_64.egg/numpy/core/include ... -o build/temp.linux-x86_64-2.7/src/_contour.o
In particular, note that matplotlib is being built against numpy-1.16.0rc2-py2.7. But then in another place you might see something like
Successfully installed ... numpy-1.15.4 ...
So then when you try and run your program, matplotlib will try to access modules that don't exist in the non-RC version of numpy, and fail.
If you already have numpy installed, easy_install won't try and fetch its own version, and will instead use the (correct) existing version.
See also
http://numpy-discussion.10968.n7.nabble.com/Issue-with-setup-requires-and-1-16-release-candidates-td46600.html
The solution is that you need to upgrade numpy.
If you are using pip
pip install numpy --upgrade
Hope it helps.

Pandas not properly installed in venv

On ubuntu 16.4.4 with Python 2.7.12, I was trying to install pandas and some other packages with pip in virtualenv. The installation proceeded without any errors. However, when I try to import pandas, I get the following error:
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
Traceback (most recent call last):
...
import pandas as pd
File "/usr/local/lib/python2.7/dist-packages/pandas/__init__.py", line 31, in <module>
"extensions first.".format(module))
ImportError: C extension: umpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.
My system has pandas of the same version(0.18.0) and runs without any problem.
What I tried and did not work:
create a new virtualenv with the --system-site-packages option.
reinstall pandas (in venv)
upgrade pandas (in venv)
upgrade numpy (from 1.11.0 to 1.14.3) (in venv)
uninstall numpy and pandas and install pandas again (in venv)
This looks like a similar problem to this one https://github.com/pandas-dev/pandas/issues/18530 i.e. mismatch between some specific versions of numpy and pandas. The pip install numpy --upgrade didn't work for me but
python -m pip install pandas==0.18.0 --force-reinstall --upgrade --no-deps --no-cache
did the trick in my case (I encountered this problem on fresh Ubuntu 17.10, python2).
Update numpy version:
pip install numpy --upgrade
Should fix the error.

Categories