Since my Macbook with an i7 CPU is currently with AppleCare, I am now working on an older Mac mini with a core duo CPU. I simply connected the Macbook's internal disk via USB to the Mac mini.
Now back at my Python scripts, I ran into a problem which I don't fully understand and do not know how to debug. When I import pandas in Python 2.7.9, Python crashes completely and I get the error Illegal instruction: 4. After some googling I assume, that some packages are compiled for the wrong architecture. But I don't know which ones.
I installed Python, numpy and scipy with homebrew and pandas, etc. with pip into a virtual environment. My system is OS X 10.10.5.
The output of python -vc "import pandas" is very long and given here.
I tried re-installing Python, pandas, numpy, and scipy.
How can I find out which package is causing the error?
Do I need to set an architecture flag or something?
How can I fix this?
Removing the .pyc files might work too.
Since it happens right after the call to
dlopen("/usr/local/lib/python2.7/site-packages/matplotlib/_pabc.so", 2);,
you can try checking the arch type that file was built for with:
file /usr/local/lib/python2.7/site-packages/matplotlib/_pabc.so
then check the arch type of your hardward:
uname -a
If the shared object file (_pabc.so) was not built for that machine you may need to re compile/install/whatever, matplotlib or one of its dependancies.
In my recent experience, this was indeed caused by a linked library being of the wrong architecture as the module's library (as chown suggested).
In particular, a C-compiled python library as part of the python module you're importing (the _mymodule.so file in the module directory) calling a linked system library (eg. libgfortran.dylib), and there being an architecture mismatch between the two.
As aforementioned, you can check the architecture of your system with uname -a and check the arch of an offending dylib via the file /path/to/lib.dylib command.
Related
I want to import fortran files to my python script by using f2py.
For that I compile them via
f2py -c -m my_lib *.f
which produces the file "my_lib.cpython-38-darwin.so" which I import to my script by
import my_lib.
On my Intel-based Macbook that works well. However, running the script on an M1 machine yields the following error:
ImportError: dlopen(./my_lib.cpython-38-darwin.so, 0x0002): tried: './my_lib.cpython-38-darwin.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/my_lib.cpython-38-darwin.so' (no such file), '/usr/lib/my_lib.cpython-38-darwin.so' (no such file)
Same happens when I start my terminal in Rosetta mode.
Any idea how to solve this issue?
I don't know if you could solve the problem but I found a solution to solve it on my Mac.
Running f2py on my Mac caused the same error.
After some time of searching for reasons I tried to use another Python Environment (based on another Base Interpreter).
Till changing I used an Anaconda Environment.
I think the error could be caused by an interpreter that's not running natively on the M1 chip. The compiled Fortran code must run natively on the Mac. So probably there could be problems in compatibility.
Now I'm using the Interpreter given by the "Developer Command Line Tools" (in this case Python 3.8) to compile the source code and to build the .so file.
I also use this Base Interpreter to run my script that has to include the Fortran package.
By doing so I was able to make it work on my Mac.
Could someone help, please? I can't understand why I see this.
I'm trying to use spwd module from Python. I just imported it, but see this, absolutely unexpected, error when running:
Traceback (most recent call last):
File "./library/system_users.py", line 25, in <module>
import spwd
ImportError: No module named pwd
I thought it should be available by default.
Python 2.7.10
pip 8.1.1 from /Library/Python/2.7/site-packages (python 2.7)
OS X El Capitan 10.11.4
I know sometimes pip install --upgrade pip can help. Unfortunately, not now.
Thank you for the advance.
As you guessed in your comment on this question, spwd is a built-in module, so it is usually built as you compile Python. For this reason, I would guess that spwd simply isn't supported on the system you are using—i.e. OS X 10.11—unless whoever built your installed version of Python made a configuration mistake or manually disabled the module.
However, we can do more than guess about this. When building Python, much of the system-specific configuration is handled in a script called setup.py. Support for many built-in modules is determined in the detect_modules method of the PyBuildExt class; the function contains the following code (taken from here).
if (config_h_vars.get('HAVE_GETSPNAM', False) or
config_h_vars.get('HAVE_GETSPENT', False)):
exts.append( Extension('spwd', ['spwdmodule.c']) )
else:
missing.append('spwd')
We can see that the function adds the Extension 'spwd' (which consists of the spwdmodule.c source file) to the lists of extensions only when config_h_var has either HAVE_GETSPNAM or HAVE_GETSPENT. The strings in config_h_vars correspond to the macros that are defined in the pyconfig.h header file that is generated when you run Python's configure script before compiling.
According to comments generated with the pyconfig.h file, HAVE_GETSPNAM is defined (to 1) if your system has the getspname function, and HAVE_GETSPENT is defined (again, to 1) if your system has the getspent function.
The problem is that OS X 10.11 (and, seemingly, newer versions of macOS) have neither of these functions. (The manual for gnulib confirms this, albeit for an older version of Mac OS X. See this page and this page.) Thus, when compiling Python on OS X/macOS, the spwd is not built, and so you see an ImportError when you try to import it on that system.
dooms's answer suggests installing spwd from Anaconda. Of course, this isn't expected to work since spwd is a built-in module, not a package you would expect to install from a package manager. I also don't see a spwd package on PyPI or any Anaconda channels.
However, since Python does not seem to prevent you from installing a module named spwd (at least when you don't have the built-in spwd), it is conceivable that you could install some sort of drop-in replacement spwd made for macOS. macOS and iOS use /etc/master.passwd instead of /etc/shadow, but maybe it might still be possible to make something work.
Unfortunately, as of this writing, it doesn't seem like any such replacement exists for macOS. I would treat any software that requires spwd as incompatible with macOS.
You should use Anaconda.
Anaconda is a completely free Python distribution (including for commercial use and redistribution). It includes more than 400 of the most popular Python packages for science, math, engineering, and data analysis. See the packages included with Anaconda and the Anaconda changelog.
After that, just type :
conda install spwd
For a schoolproject I need to make use of the aubio library. However, I have a problem compiling it for Windows.
I downloaded the latest source from his git (0.4.0 alpha). Then I compiled it using Cygwin, using the --with-target-platform=win32 to cross-compile it for Windows. It uses waf by the way.
This works without any errors.
Next step is copying the compiled file (libaubio.dll.a) to MinGW library folder. Then I want to compile the Python wrapper for the module, but it shows up the ld.exe cannot find -laubio error. Renaming the libaubio.dll.a to libaubio.a resolves this and compiling succeeds succesfully.
Installing it into the Python folder works perfectly too. But here starts the problem. When trying to import aubio, I get this error: ImportError: DLL load failed: Invalid access to memory location.
I have no clue on how to solve this problem. Can anybody help? Or explain the error to me?
Thanks in advance!
Xander
PS. It compiles perfectly on both OSX and Ubuntu.
Well, this may not be the right solution for you, just a hint. ImportError: DLL load failed: Invalid access to memory location. I encountered the same error when trying to make my own extension of Python programmed in C. Platform: Windows 32bits.
It was a real pain because this error appeared randomly in interactive as well as in non-interactive mode in all Python environments (Spyder, Notebook, plain console...). I compiled my code using MinGW and Python's distutils (command python setup.py install). The compilation gave no warnings or errors and produced pyd file to the correct directory. But when trying to import this module import example pro my Python code it irregularly crashed (usually only one out of five attempts to import the module succeeded).
Strange was that on another computer it worked just fine... Well, finally I found workaround - I downloaded a newer version of MinGW (before I had used the version that comes packed in Qt SDK distribution) and compiled the module again. Then it worked with no more crashes. However I did not find any systematic solution or explanation. So I might have something to do with the compiler (maybe absence of its DLLs? I do not know exactly) that was used to generate the pyd file.
I've been trying to run a web server based in Python 2.6 code here at work. The server requires the python LDAP libraries in order to run. Because I'm working on Mac OS X Lion, I needed to run a manual install of python-ldap 2.4.7 in order to get Python to recognize LDAP at all. Python-ldap appeared to intall correctly, but when I try to run the web server I get the following error (I added some line breaks for the sake of clarity):
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/python_ldap-2.4.7-py2.6-macosx-10.7-fat.egg/_ldap.so, 2): Symbol not found: _ber_pvt_opt_on
Referenced from: /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/python_ldap-2.4.7-py2.6-macosx-10.7-fat.egg/_ldap.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/python_ldap-2.4.7-py2.6-macosx-10.7-fat.egg/_ldap.so
I'm using OpenLDAP 2.4.21, and I set the following system variables before running my python-ldap installation:
export ARCHFLAGS="-arch i386"
export CFLAGS="-isysroot /Developer/SDKs/MacOSX10.7.sdk -arch i386"
export MACOSX_DEPLOYMENT_TARGET="10.7"
I should also probably mention that I've had to force everything I've installed to use 32-bit architecture instead of 64-bit in order to work around some issues with Oracle's 64-bit support. Has anyone encountered a similar situation, or do they know the significance of the "_ber_pvt_opt_on" symbol that LDAP was looking for? The number of Google results I was able to come up with was both small and unhelpful. Any light that you could shed on the situation would be greatly appreciated.
I have an algorithm written in C++ that I want to use in ArcGIS. Arc supports python scripting, so I figured I'd write a python wrapper. I built python on my development PC and then built a module (called "delaunay", to calculate Delaunay triangulations) to wrap the algorithm. On that PC I was using cygwin with mingw g++ and put the .dll that I built into /usr/local/lib/python2.6/site-packages. Works fine.
So now I'm trying to get it to work on the PC that I run GIS on. ArcGIS installs python as part of its own installation, so I put the .dll in /Python26/ArcGIS10.0/Lib/site-packages (which is in the sys.path) and tried to import it. I got the following error:
ImportError: No module named delaunay
Renamed delaunay.dll to delaunay.pyd and got this error:
ImportError: DLL load failed: The specified module could not be found.
I have a feeling it'd work if I put the source on the GIS machine and built and installed it with distutils, but I don't really want to install cygwin or Visual Studio on that machine.
Anyway, I have no idea why I get different messages, what the significance of the messages is, or whether this will work at all (i.e. is the ArcGIS distro of Python incompatible with mine 'cause we used different compilers?)
Also, the dev machine has a Jun 12, 2010 revision of Python 2.6.5 and the GIS machine has a Mar 19, 2010 revision. Maybe that is significant?
Help?
Run python -vvv and you'll see import debug info.
You are probably mixing 32-bit and 64-bit binaries. Binaries from different Python builds are incompatible.
Renamed delaunay.dll to delaunay.pyd and got this error:
ImportError: DLL load failed: The specified module could not be found.
This sounds like delaunay.pyd depends on some other dll that is not found.