I want to call Python modules or user-defined functions/scripts I built in python through Matlab. I'm aware of many of the things I need to cover before something like this is possible, I'll make a list of everything I've done so far.
Created Virtual Environment with Anaconda and istalled all the dependencies I needed. Python version 3.5.2
Ran the scripts using PyCharm and all runs good/as expected.
When running pyversion(mypath) in Matlab, though it appears "loaded: 1", I can't call anything on the site-packages module. For example, I can't even call something like numpy (py.importlib.import_module('numpy')) because I get the following error:
Python Error: ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: DLL load failed: The specified module could not be found.
I've verified everything I could so far. Python and Matlab (2016a) are both 64 bits. Although, I can run this small snippet of code with the expected outcome:
T = 'MATLAB(R) is a high-level language'
wrapped = py.textwrap.wrap(T);
whos wrapped
Since I can execute everything properly from PyCharm, and I can't from Matlab's side, I'm wondering if there's any restriction to enable from Matlab or Anaconda (in the configuration).
Is there a setting in the configuration of Anaconda that is maybe not allowing me to do this? Or is this an issue within Matlab? I'm not sure, I'm clueless.
Related
I'm doing some scientific computation in python, and recently switched away from Anaconda since it doesn't yet support python 3.10 (which has some new features I'd like to use). I am now running on python version 3.10.9. I have a pipenv virtual environment set up for this, and in it I've got numpy version 1.23.5 installed. (Trying to upgrade to a newer version fails with a bunch of errors -- if upgrading numpy ends up being necessary I'll need to open a new question to ask about how to get that working.)
Everything initially seems to be working fine, until I try to run a parallelized numpy computation. At that point, I get the error DLL load failed while importing _multiarray_umath: The specified module could not be found. Here's the associated traceback:
Traceback (most recent call last):
File "C:\Users\charles\.virtualenvs\pipenv-WB8juvy1\lib\site-packages\numpy\core\__init__.py", line 23, in <module>
from . import multiarray
File "C:\Users\charles\.virtualenvs\pipenv-WB8juvy1\lib\site-packages\numpy\core\multiarray.py", line 10, in <module>
from . import overrides
File "C:\Users\charles\.virtualenvs\pipenv-WB8juvy1\lib\site-packages\numpy\core\overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.
I can import numpy just fine, and I can use various functions from numpy without issue as well. I only get problems when I specifically want to do some parallelized computations. At that point my console gets flooded with this error (as far as I can tell, it's printing nearly exactly the same thing over and over, presumably once for every worker that it tried to launch).
The following text is also included in the error message:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.10 from "C:\Users\charles\.virtualenvs\pipenv-WB8juvy1\Scripts\python.exe"
* The NumPy version is: "1.24.1"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
While this looks pretty promising, I've read through the documentation on that page and nothing seems to be addressing the particular issue I'm running into here.
Additional details about my setup: The virtual environment was set up with pipenv, using the --site-packages flag (as I don't seem to be able to properly install PyQt5 in the virtual environment, so I'm just sourcing it from the system). I've got numpy installed in both my system's site-packages - that is version 1.24.1 (which checks out with the error messages) - and in the virtual environment's site packages (I didn't explicitly install it there, it was installed as a dependency; the virtual version is 1.23.5). Uninstalling numpy from the system site packages does not fix the issue.
I've also tried completely deleting the virtual environment and reinstalling it from the pipfile, but that also doesn't fix the issue. The version of numpy which is installed in the virtual environment does indeed have a DLL in its .lib folder, so it's not just missing. As far as I can tell there's nothing weird about my path; just in case I tried manually adding the .lib folders where the DLLs live to the path. I added both the virtual and system site packages to the path, and tried sourcing them in various different orders, to no avail.
I'm kinda at the end of my ability to troubleshoot this at this point, hence this request for help. In the meantime I'm resorting to doing all of my computations serially, but (as you might imagine) this is very slow and not very sustainable for the long term. If nobody can help me fix this, I'm going to have to just go back to Anaconda and give up on features from the newer python versions until they support them. (Or maybe I can try working without a virtual environment, but I've messed up systems pretty badly that way before and I'm kind of loathe to take that approach).
I'm using Anaconda. I created an environment called ENGINEERING. In than environment I installed python 3.6, pandas 1.1.3, spyder 3.3.6, numpy 1.19.2, and many more. The base environment has these packages also but not necessarily the same version. Within the ENGINEERING env I created a python script in Spyder that runs without any issues when I run it in spyder. Then, I want to automate that script by creating a .bat file and then using that to automate via windows task scheduler. My .bat looks like this:
"C:\Users\alopo000\Anaconda3\envs\engineering\python.exe" "C:\Users\alopo000\Files\Python\Valid\Portf.py"
pause
when I run the .bat I get an error message:
ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.6 from "C:\Users\alopo000\Anaconda3\envs\engineering\python.exe"
* The NumPy version is: "1.19.2"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: DLL load failed: The specified module could not be found.
I've tried many things to fix this. I went to the website they suggested and couldn't find anything there. I uninstalled pandas and numpy and reinstall them again but still, same error. The one thing I haven't tried is to fix the VARIABLE PATHS, mostly because I've read in many places not to change those. I can change them if that's what it is, but I would like to understand first why it is not recommended. I read it can create issues with other versions of python but not sure if that's what it is.
Do I need to activate the environment in the .bat file? is the problem that the .bat file is trying to run it outside the environment? Any help would be appreciated.
You will need to upgrade/change your base Python & numpy version installations to match those specified (3.6, 1.19.2). I had the same issue and same situation as OP (write/dev program in a virtual Spyder environment "spyder-env", then automate .py file with WTS). I tried copying over & running the 'activate.bat' in the /scripts/ folder of the virtual environment but that didn't work. In my situation, my conda spyder-env was Python = 3.9.12 and numpy = 1.21.5.
I ended up upgrading my base global Python version (conda install) for Python which changed from 3.8.3 and did the same with numpy (had to downgrade). I'm not sure if this is exactly necessary, why this works, or if it may cause problems down the line (I was about to just re-install conda).
The appropriate .bat file should be something like this once you do that:
call "C:\Users\alopo000\Anaconda3\Scripts\activate.bat"
call "C:\Users\alopo000\Files\Python\Valid\Portf.py" %*
REM Uncomment below to debug
REM pause
I need to install and use the Python NumPy module (and then later the Pandas module) in order to process heavy data in Python.
I downloaded and installed ENTHOUGHT, but it wasn't what I wanted all that extra clutter of extra modules (which defeats the purpose of importing Python modules only as needed), but the uninstall did not work properly (i.e. it left garbage folders and ENTHOUGHT remnants all over my computer).
I have tried installing NumPy via EASY_INSTALL and PIP (two package managers if I understand correctly) - but with no success. Every time I try to run my program, I get the error: "no module named numpy".
I have searched the questions here and have tried to alter my ENVIRONMENT VARIABLE as per the following video, but again, no success:
https://www.youtube.com/watch?v=ddpYVA-7wq4
C:\Python34
...still the same error!
I downloaded Anaconda (with all its extra clutter and installed, but I don't like the development environment - I want my Vanilla Python IDLE to run Vanilla NumPy with no extra clutter modules...) and when I tried to again install Numpy I received a message that it was already installed with a path to:
C:\users\yoni\anaconda3\lib\site-packages
....so I ALSO added this PYTHONPATH to the ENVIRONMENT VARIABLE in hopes that it would now recognize where the NumPy installation was (currently with Anaconda3 - but I hoped to be able to import NumPy to my vanilla Python IDLE):
C:\Python34;C:\users\yoni\anaconda3\lib\site-packages
I don't find a clear answer - I see others have the same problem, and nothing is working for me. How can I finish this installation of NumPy so that it works for me when I do a simple import of module?
This is a temporary solution until you can resolve your path issue.
It will be environment specific.
import sys
sys.path.append('C:\users\yoni\anaconda3\lib\site-packages[PackageName]')
import PakcageName
For a schoolproject I need to make use of the aubio library. However, I have a problem compiling it for Windows.
I downloaded the latest source from his git (0.4.0 alpha). Then I compiled it using Cygwin, using the --with-target-platform=win32 to cross-compile it for Windows. It uses waf by the way.
This works without any errors.
Next step is copying the compiled file (libaubio.dll.a) to MinGW library folder. Then I want to compile the Python wrapper for the module, but it shows up the ld.exe cannot find -laubio error. Renaming the libaubio.dll.a to libaubio.a resolves this and compiling succeeds succesfully.
Installing it into the Python folder works perfectly too. But here starts the problem. When trying to import aubio, I get this error: ImportError: DLL load failed: Invalid access to memory location.
I have no clue on how to solve this problem. Can anybody help? Or explain the error to me?
Thanks in advance!
Xander
PS. It compiles perfectly on both OSX and Ubuntu.
Well, this may not be the right solution for you, just a hint. ImportError: DLL load failed: Invalid access to memory location. I encountered the same error when trying to make my own extension of Python programmed in C. Platform: Windows 32bits.
It was a real pain because this error appeared randomly in interactive as well as in non-interactive mode in all Python environments (Spyder, Notebook, plain console...). I compiled my code using MinGW and Python's distutils (command python setup.py install). The compilation gave no warnings or errors and produced pyd file to the correct directory. But when trying to import this module import example pro my Python code it irregularly crashed (usually only one out of five attempts to import the module succeeded).
Strange was that on another computer it worked just fine... Well, finally I found workaround - I downloaded a newer version of MinGW (before I had used the version that comes packed in Qt SDK distribution) and compiled the module again. Then it worked with no more crashes. However I did not find any systematic solution or explanation. So I might have something to do with the compiler (maybe absence of its DLLs? I do not know exactly) that was used to generate the pyd file.
Disclaimer of research:
I have examined the following other StackOverflow questions:
How to import numpy in python shell
How can I use numpy without installing it?
Import a module from a relative path
Perhaps to some, those may answer my question, but according to my knowledge, I still do not understand the situation.
I am trying to import numpy so that matplotlib will work, but upon execution of the __init__.py file in the numpy folder, the following error message is displayed:
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python intepreter from there.
Explain what it means to import something from its source directory as opposed to some other way of importing it. Does it mean that it should not be source code when it is imported? Or does it mean that it literally is just the wrong directory/folder that I am importing. I know that one other StackOverflow answer is:
The message is fairly self-explanatory; your working directory should not be the numpy source directory when you invoke Python; numpy should be installed and your working directory should be anything but the directory where it lives.
However, I don't understand this. Aren't you supposed to import things that you want to work with? I'm assuming that the import command combines the source directory into your current working directory in this statement.
I also read the other answers such as:
Using distutils to install local directories
Using virtualenv to create a virtual system directory
Using Enthought's EPD to have numpy pre-installed in what I believe to be the system directory,
and
Using a command like $ dpkg -i --force-not-root --root=$HOME mypackagename.deb to create what I believe is some kind of sub-system directory that is treated like a system directory.
So, correct me if I'm wrong, but does numpy somehow strongly require to be somehow installed in the main system directory?
Machine status:
I am using Windows machines without administrative privlidges.
They have Python 3.3 Shell as well as matplotlib installed.
When running command prompt, python and python3 are not recognized. I have to run the Python shell from the applications menu.
I can successfull begin importing matplotlib from even my own directory, different from theirs, but it stops upon reaching __init__.py of the numpy module, if it exists and reports the error stated above.
Update:
Luckily, my administrators were able to directly install numpy correctly in the site-packages folder. Thank you for answering my question though. I understand the situation a lot more because of you.
numpy includes extension modules written in C. You will need to build these extension modules before the numpy package is complete. The most robust way to do this is to build it and install it to site-packages like normal. You can also install it to another directory using the standard distutils options for this. However, once you have installed it, you should change your directory out of the source tree. Python starts looking for packages in your current directory, so the presence of the incomplete numpy package (without the necessary built C extension modules) will be picked up first and lead to the error that message that you quote. This happens a lot, so we give a long message explaining what to do.