We are currently running our own private PyPi server, and uploading wheels of our internal Python libraries to speed up installs.
Many of our tools require numpy, scipy, pandas, etc.
We built wheels for all of our dependencies by installing all of our dependencies from source, and then using
pip wheel .
which outputs wheels for all of our dependencies - or so we thought.
We have found that when we install, say, numpy from our private PyPi, where it is available as a wheel, the following happens.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "redacted/lib/python2.7/site-packages/numpy/__init__.py", line 199, in <module>
from . import random
File "redacted/site-packages/numpy/random/__init__.py", line 99, in <module>
from .mtrand import *
ImportError: redacted/lib/python2.7/site-packages/numpy/random/mtrand.so: undefined symbol: PyFPE_jbuf
It is clear to me that these wheels were not created properly. What is not clear to me is how to go around fixing this. We are not distributing these to the public, only for our internal tools.
I am far from being an expert on wheel packaging but here is what I know about wheel building so far.
For OS X and Windows you can build binary wheels and use the to deploy your software.
On Linux this process doesn't always work and that's because many binary wheels will try to use specific versions of OS libraries (.so) and these are different from one distribution to another, or even inside the same distribution.
Here is some further reference:
http://lucumr.pocoo.org/2014/1/27/python-on-wheels/
Related
I'm trying to import pyo in python3.8.8 and I'm not getting any results. When I try to run the command:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in
import pyo
File "C:\Users\andre\AppData\Roaming\Python\Python38\site-packages\pyo_init_.py", line 28, in
from .lib import analysis as analysis
File "C:\Users\andre\AppData\Roaming\Python\Python38\site-packages\pyo\lib\analysis.py", line 32, in
from ._core import *
File "C:\Users\andre\AppData\Roaming\Python\Python38\site-packages\pyo\lib_core.py", line 58, in
from .._pyo import *
ImportError: DLL load failed while importing _pyo: Impossibile trovare il modulo specificato.
How can I solve this?
I assume you are on windows. What version of pyo do you have installed? I saw that it was only version 1.0.1 that python 3.8 support was introduced. Try uninstalling pyo, then make sure you install the newest version. Also, did you install with pip? If you did, try installing from their downloadable releases on github. What code did you write that caused this error? I need to see your code.
Looks like you're using pc032 (032bit) Python. Last PyO version with pc032 (on Win) support is [PyPI]: pyo 1.0.1 (from 191127). But I didn't see the Python 3.8 .whl among files (neither for a couple of older versions).
Just out of curiosity: how did you install PyO?
In order to get things going either:
Switch to pc064 (064bit) Python
Use a (pc032) Python version that prebuilt .whl exists for
I've built v1.0.4 (latest at answer time) and placed the .whls at [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/PyO/v1.0.4. Download and install the preferred one (check [SO]: Installing pygraphviz on Windows 10 64-bit, Python 3.6 (#CristiFati's answer) (at the end) for more details about the process)
I used python xy recently with an old lib of scipy (0.15). Because I wanted to use Spherical Voronoi, I had to install a more recent version of scipy. For this I used pip and the unofficial distributions for python modules with wheel. The install worked without error messages. But now, if I try to import scipy by code, I get this error message below. I already unistalled whole python packages so far and instead installed anaconda. But the error message is still the same. I checked my registry (Windows 10) - can't find anything suspicious.
Any idea? Thank you!
test.py
1.12.1
Traceback (most recent call last):
File "test.py", line 11, in <module>
import scipy
File "C:\Users\Boss\AppData\Roaming\Python\Python27\site-packages\scipy\__init__.py", line 61, in <module>
from numpy._distributor_init import NUMPY_MKL # requires numpy+mkl
ImportError: cannot import name NUMPY_MKL
Your scipy is built with mkl support and requires mkl support in numpy too. Download numpy from the same site you downloaded scipy (probably Christoph Gohlke's builds). Or if you are using Anaconda use Anaconda packages for both.
Updated - Your error is basically that numpy+mkl (numpy with Intel math Kernel library) so this is because when you have installed the scipy by precompiled archive, which requires numpy+mkl but installing numpy using pip won't get you that.
This problem can be easy solved by installation for numpy+mkl from whl file from - http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
Old - This is due to the improper installation of numpy, I've had this error few days ago and then I had to install the numpy again. So, upgrade pip and then try installing the numpy whl again. It just worked for me.
Get it here -https://pypi.python.org/pypi/numpy
even this has almost everything - http://www.lfd.uci.edu/~gohlke/pythonlibs/
I downloaded fiona today. when I try to import it in Python using 'import fiona', I get the following error:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import fiona
File "C:\Python27\lib\site-packages\fiona\__init__.py", line 72, in <module>
from fiona.collection import Collection, supported_drivers, vsi_path
File "C:\Python27\lib\site-packages\fiona\collection.py", line 7, in <module>
from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: No module named ogrext
I checked in my site-packages folder, and ogrext is a "C" file. I tried commenting out the import to see if it wasn't necessary, but this of course threw another error.
Specifically, how do I resolve this import error?
More generally, how does one resolve errors involving importing C files into a python library?
You can't just install any module by copying all the files to site-packages. Some modules are pure Python, but there are many with extensions written in other languages (C, C++, Fortran, etc.) that need to be compiled and linked into libraries before being used, and fiona is one of them. This compilation can be done at several stages - by the author, before distributing the module as a wheel, during the pip install process, or by downloading the package's source, unzipping/tarring it, and running python setup.py install. Unfortunately, Windows doesn't come with a compiler by default, so you either need to install and configure your system for gcc or Visual Studio, or use another method, such as a precompiled installer. Fortunately, fiona is available from Christoph Gohlke's Python Extension Packages for Windows Repository here. Download the installer for your version and bit-ness of Python, delete the fiona folder in site-packages, then run the installer. This site contains a large number of packages for scientific computing, and is my go-to resource when I need to install a new module, especially if it has extensions.
EDIT
Upon further inspection, it appears that fiona also requires the GDAL module, as well as six, both of which can be downloaded from Gohlke's repository. I first installed fiona only (I already had six installed), and got a missing DLL error. I then installed GDAL, and import fiona worked just fine - I'm not familiar with the module, so I didn't do any further testing, but hopefully everything should work now.
I am trying to install a python module with the standard python setup.py install but I get the following error. I am fairly new to python but I have been able to install other modules in this way in the past. I am under the impression this module setuptools is not something I am supposed to have gotten separate from my python installation. Do I need to be in a specific directory or something?
Error:
Traceback (most recent call last):
File "setup.py", line 3 in <module>
from setuptools import setup, find_packages
ImportError: No module named setuptools
Apparently, that package requires you to have setuptools to install it. Setuptools is a module that provides easy installation of Python packages. You can get it on pypi, here.
Surprisingly, setuptools does NOT come pre-packaged with Python, despite how often and casually everyone refers to it. I also had this problem when learning Python.
Setuptools is available on the Python Package Index.
I am trying to install a python library and receive this error after downloading an egg file.
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c7-py2.5.egg
Traceback (most recent call last):
File "setup.py", line 10, in <module>
use_setuptools(min_version=min_version)
File "/Users/tylo/Downloads/Archives/simplejson-2.0.9/ez_setup.py", line 88, in use_setuptools
import setuptools; setuptools.bootstrap_install_from = egg
zipimport.ZipImportError: can't decompress data; zlib not available
I did some research and discovered that zlib is built into OS X.
What could be going wrong here?
Run xcode-select --install to fix installation of Command Line Tools for Xcode working.
It's not the zlib C library that is missing, but the zlib Python module. This is usually caused by compiling Python yourself, and not having the necessary bits (header files, specifically) of zlib available, even when you do have the C library available. Or, sometimes, by the zlib Python module having the wrong permissions; take a look in the directories in sys.path, looking for a zlib.so or zlibmodule.so. If it doesn't exist, the Python installation was built without it, or building it failed. If it does exist, check its permissions (and the directory's permissions.)