I would like to install a python package (third party package written with cython for some numerical calculations).
My previous experience (which were successfully) is that, I only need to perform following command:
python setup.py install --prefix="/destination/path/"
Then, I would expect an additional folder as /destination/path/package
so, I can do
export PYTHONPATH=$PYTHONPATH:/destination/path/
So, I can finally import the package normally in python script.
However, it seems that the default behavior of setup.py (I am using anaconda python3.7) changed. When I use the same (first) command, I encounter an error as:
Checking .pth file support in /destination/path//lib/python3.7/site-packages/
/path/to/python -E -c pass
TEST FAILED: /destination/path/lib/python3.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/destination/path/lib/python3.7/site-packages/
Could someone explain to me what happened and how can I fix this error?
Thank you very much in advance.
Best,
Greg
Related
I am attempting to install from a Python setup. I had this working after a month of hell-ish attempts on 18.04, and now need it on 20.04.
The setup.py while I am using is on this repository ( https://github.com/flashlight/flashlight/tree/main/bindings/python ).
I am using Ubuntu 20.04, fresh install sans a few dependencies I know the system needs.
I have Python3.10 installed locally.
During the install at steps
cd bindings/python
python3.10 setup.py install
I hit the error
CMake Error at cmake/FindMKL.cmake:400. MKL library not found. Please specify the library location by appending the root directory of the MKL installation to the environment variable CMAKE_PREFIX_PATH
Which is weird for a variety of reasons.
I did a sudo apt-get install mkl-devel which took, but also
The setup script claims it defaults to MKL off.
Right prior to this error, I do get a warning.
Cmake Warning at cmake/FindMKL.cmake:387. MKL libraries files are found, but MKL header files are not. You can get them by 'pip install mkl-devel' (I did this btw) if using pip. If build fails with header files available on the system, please make sure that CMake will search the directory containing them by setting CMAKE_INCLUDE_PATH
That last part I did not do, primarily because I'm not sure how to tell where pip install mkl-devel would have put those header files.
That said, when I do a find / "*mkl*" I notice files in two primary locations
.h files : /home/myusername/.local/include
.so files: /home/myusername/.local/lib
So I set the following environment variables in my terminal
LIBRARY_PATH=/home/myusername/.local/lib
LD_LIBRARY_PATH=/home/myusername/.local/lib
KENLM_ROOT=/home/myusername/Flashlight/kenlm
USE_MKL=0
CMAKE_INCLUDE_PATH=/home/myusername/.local/include/
CMAKE_PREFIX_PATH=/home/myusername/.local/lib
However when I try to install again, the process still fails with both of the MKL issues above (Warning and Error)
I'm baffled at what I am supposed to be pointing where to get this to succeed at this point.
I've got a couple of projects here for which I'm preparing documentation at the moment, hosted at readthedocs.org. FYI, all of them use poetry and I use custom .readthedocs.yml files with this entry:
python:
install:
- method: pip
path: .
It works fine for most projects, but it fails for two for different reasons during installation of the project via pip:
The first one uses PyGObject, which failes like this:
Package gobject-introspection-1.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gobject-introspection-1.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gobject-introspection-1.0' found
Command '('pkg-config', '--print-errors', '--exists', 'gobject-introspection-1.0 >= 1.56.0')' returned non-zero exit status 1.
Try installing it with: 'sudo apt install libgirepository1.0-dev'
So it seems that PyGObject cannot be installed without some system packages to be installed. I could rearrange the code so that the import is not top-level. But still I need it in the dependencies. Can I tell pip install to ignore this single package somehow? Any other idea?
The second project compiles some C++ code via Cython and fails, because it's missing a library. I use a custom build script in the pyproject.toml:
[tool.poetry.build]
script = "build.py"
generate-setup-file = false
Is there some flag in pip that I could set and retrieve in build.py to skip the compilation? Or is there a better way?
I have run
python setup.py sdist --formats=gztar,zip bdist_wheel
and then
python setup.py install
The result is that the egg files are created in the site-packages directory but not the <package-name>/<package-source files>:
$ls /usr/local/lib/python3.7/site-packages/infix*
/usr/local/lib/python3.7/site-packages/infixpy-0.0.3-py3.7.egg
/usr/local/lib/python3.7/site-packages/infixpy.egg-link
/usr/local/lib/python3.7/site-packages/infixpy-0.0.4-py3.7.egg
Notice that the directory infix was not created - and thus none of the source code was copied. What am I missing / not understanding in this local installation process?
Update When I had run
pip3 install infixpy
there was an additional directory infix and the source code was included in that directory. Running the local or devel modes of setup.py install was not causing that code to be updated and - crucially - the stacktraces from running any python code (even in a completely new ipython repl) was showing only the older / pip3 installed code. In particular the file __init__.py So my observation has been that the source file :
/usr/local/lib/python3.7/site-packages/infixpy/__init__.py
is an accurate reflection of what the python executable were using. #phd is mentioning that the source code is already included in the egg. So then I do not understand the relationship between the source code in the egg and the source code in that subdirectory - which in the lastest run of mine is completely missing.
The following commands all yield slightly different results:
pip install .: installed as uncompressed package directories and a XXX.dist-info directory
pip install infixpy: same as previous, but installed from an (remote) index (per default PyPI), not from the local directory
python setup.py install: installed as a zipped file XXX.egg
pip install --editable . or python setup.py develop: not installed, but linked as a XXX.egg-link file
So depending on the commands entered, the content of site-packages is different.
Now this is what you say you have:
$ls /usr/local/lib/python3.7/site-packages/infix*
/usr/local/lib/python3.7/site-packages/infixpy-0.0.3-py3.7.egg
/usr/local/lib/python3.7/site-packages/infixpy.egg-link
/usr/local/lib/python3.7/site-packages/infixpy-0.0.4-py3.7.egg```
This is a bit surprising, since theoretically there are 3 versions of your project that are importable (0.0.3, 0.0.4, and develop/editable). I am not sure which one is used by the Python interpreter in this case. You might want to run pip uninstall infixpy a couple of times to start fresh and alleviate these uncertainties. You can then experiment with the commands mentioned above and see how they impact the content of site-packages along with inspecting the result of pip show infixpy.
I'm running python 3.6 via anaconda 3, using Visual Studio Code.
I followed instructions like these (Interactive Brokers API install) and downloaded the package to a local directory of mine say: c:\dev\pyib, so now the code is in c:\dev\pyib\IbPy-master
I open that directory in command line and run
python setup.py install
All runs ok.
But then my program, which is in c:\dev\pyib says Module not found. (In my case ibapi). The linter is also showing red.
There is no other python installed on this pc.
Where did the package install to? and how do I check that? What will I find where the package installed itself to that shows me its there?
Or do I have to use a trial-and-error with the linter and sys.path.append()? (I tried that with the directory where the files are downloaded to - to no avail)
I'm trying to set up the PYTHONPATH using the "env" in launch.json from Visual Studio Code, as shown in this unaccepted answer.
Current sys.path:
'c:\\dev\\pyIb',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\python36.zip',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\DLLs',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-
packages',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Babel-2.5.0-py3.6.egg',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Pythonwin'
I deleted the ib directory and re-ran the install. The last line says: Writing C:\Users\user\AppData\Local\Continuum\anaconda3\Lib\site-packages\IbPy2-0.8.0-py3.6.egg-info So is the location of the egg-info the location of my undetected module? The actual folder in the site-packages is called ib.
Or could my problems be because of a difference in Lib vs. lib with the lowercase in the sys.path and the uppercase in the actual directory?
But the real question here is still: HOW DO I KNOW WHERE the package was installed what should I search for?
This answer is specific for anaconda3 Python and packages installed using python setup.py install (which is actually using distutils)
Take a look at anaconda3\Lib\site-packages you should see a directory for the package you installed.
The way to know for sure where your package is, is by doing a pip list then trying to pip uninstall and re-install again using the python setup.py install: Here are the detailed instructions:
When uninstalling, pip will tell you it cannot because it was done via distutils.
You'll get a message like this:
DEPRECATION: Uninstalling a distutils installed project (ibpy2) has been deprecated and will be removed in a future version.
This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
You'll be prompted to continue anyway. If you choose No, then you can find the directory in
C:\Users\<yourusername>\AppData\Local\Continuum\anaconda3\Lib\site-packages
Thanks to Emanuel Mtali for pointing me in the right direction
Some more information:
The problem I had was due to a stupid mistake of mine. I was running setup of a different (but related) package not used anymore. IbPy2 instead of TwsAPI. I was supposed to run the setup.py of the package installed via the latest version of the MSI from IB, and NOT the IbPy2 package. :-(
I'm trying to use easy_install on windows7x64 and getting this:
<output>
c:\Python27\Scripts>easy_install.exe django-piston
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 2] No such file or directory: 'C:\\Python27\\Lib\\site-packages\\Lib\
\site-packages\\test-easy-install-4132.pth'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
C:\Python27\Lib\site-packages\Lib\site-packages\
This directory does not currently exist. Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).
Where can I change easy_install's path to correct one "C:\Python27\Lib\site-packages"?
easy_install should install the packages in the Python installation's site-packages directory. It could have happened that you changed Python's directory accidentally.
Reinstalling easy_install should fix this.
Otherwise you could use:easy_install --install-dir C:\Python27\Lib\site-packages\Lib\site-packages\ django-piston
However, this last option has two issues:
Make sure C:\Python27\Lib\site-packages\Lib\site-packages\ is in PYTHONPATH
You will need to indicate a install-dir directory for every new package you install