I've got a couple of projects here for which I'm preparing documentation at the moment, hosted at readthedocs.org. FYI, all of them use poetry and I use custom .readthedocs.yml files with this entry:
python:
install:
- method: pip
path: .
It works fine for most projects, but it fails for two for different reasons during installation of the project via pip:
The first one uses PyGObject, which failes like this:
Package gobject-introspection-1.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `gobject-introspection-1.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gobject-introspection-1.0' found
Command '('pkg-config', '--print-errors', '--exists', 'gobject-introspection-1.0 >= 1.56.0')' returned non-zero exit status 1.
Try installing it with: 'sudo apt install libgirepository1.0-dev'
So it seems that PyGObject cannot be installed without some system packages to be installed. I could rearrange the code so that the import is not top-level. But still I need it in the dependencies. Can I tell pip install to ignore this single package somehow? Any other idea?
The second project compiles some C++ code via Cython and fails, because it's missing a library. I use a custom build script in the pyproject.toml:
[tool.poetry.build]
script = "build.py"
generate-setup-file = false
Is there some flag in pip that I could set and retrieve in build.py to skip the compilation? Or is there a better way?
I tried to install the lanelet2 library according to the github installation guide at https://github.com/fzi-forschungszentrum-informatik/Lanelet2.
When I perform catkin build I get the following error:
Errors << lanelet2_python:cmake /home/student/catkin_ws/logs/lanelet2_python/build.cmake.000.log
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package):
Could not find a package configuration file provided by "boost_python3"
(requested version 1.71.0) with any of the following names:
boost_python3Config.cmake
boost_python3-config.cmake
Add the installation prefix of "boost_python3" to CMAKE_PREFIX_PATH or set
"boost_python3_DIR" to a directory containing one of the above files. If
"boost_python3" provides a separate development package or SDK, be sure it
has been installed.
My OS is Ubuntu 20.04 with ROS noetic. The build is performed inside a venv with Python Version 3.8.10.
The command python is pointing to python3. I've also installed the following dependencies:
sudo apt-get install ros-noetic-rospack ros-noetic-catkin ros-noetic-mrt-cmake-modules
sudo apt-get install libboost-dev libeigen3-dev libgeographic-dev libpugixml-dev libpython3-dev libboost-python-dev python3-catkin-tools
Does someone have an idea how to resolve this error?
See neutrinoyu's comment at https://github.com/ethz-asl/kalibr/issues/368#issuecomment-651726289
/kalibr/Schweizer-Messer/numpy_eigen/cmake/add_python_export_library.cmake:89
change
list(APPEND BOOST_COMPONENTS python3)
to
list(APPEND BOOST_COMPONENTS python)
What I should have:
I want my Yocto Project to build a package for my Python project with all dependencies inside. The project has to run out of box on the resulting read-only sdcard image.
It simply should install all requirements in the required version to the package.
What I tried without luck:
Calling pip in do_install():
"pip/pip3 is not found", even it's in RDEPENDS.
Anyway, I really prefer this way.
With inherit pypi:
When trying with inherit pypi, it tries to get also my local sources (my pyton project) from pypi. And I have always to copy the requirements to the recipe. This is not my preferred way.
Calling pip in pkg_postinst():
It tries to install the modules on first start and fails, because the system has no internet connection and it's a read-only system. It must run out of the box without installation on first boot time. Does its stuff to late.
Where I'll get around:
There should be no need to change anything in the recipes when something changes in requirements.txt.
Background information
I'm working with Yocto Rocko in a Linux environment.
In the Hostsystem, there is no pip installed. I want to run this one installed from RDEPENDS in the target system.
Building the Package (only this recipe) with:
bitbake myproject
Building the whole sdcard image:
bitbake myProject-image-base
The recipe:
myproject.bb (relevant lines):
RDEPENDS_${PN} = "python3 python3-pip"
APP_SOURCES_DIR := "${#os.path.abspath(os.path.dirname(d.getVar('FILE', True)) + '/../../../../app-sources')}"
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI = " \
file://${APP_SOURCES_DIR}/myProject \
...
"
inherit allarch # tried also with pypi and setuptools3 for the pypi way.
do_install() { # Line 116
install -d -m 0755 ${D}/myProject
cp -R --no-dereference --preserve=mode,links -v ${APP_SOURCES_DIR}/myProject/* ${D}/myProject/
pip3 install -r ${APP_SOURCES_DIR}/myProject/requirements.txt
# Tried also python ${APP_SOURCES_DIR}/myProject/setup.py install
}
# Tried also this, but it's no option because the data MUST be included in the Package:
# pkg_postinst_${PN}() {
# #!/bin/sh -e
# pip3 install -r /myProject/requirements.txt
# }
FILES_${PN} = "/myProject/*"
Resulting Errors:
Expected to install the listed modules from requirements.txt into the myProject package, so that the python app will run directly on the resulting readonly sdcard image.
With pip, I get:
| /*/tmp/work/*/myProject/0.1.0-r0/temp/run.do_install: 116: pip3: not found
| WARNING: exit code 127 from a shell command.
| ERROR: Function failed: do_install ...
When using pypi:
404 Not Found
ERROR: myProject-0.1.0-r0 do_fetch: Fetcher failure for URL: 'https://files.pythonhosted.org/packages/source/m/myproject/myproject-0.1.0.tar.gz'. Unable to fetch URL from any source.
=> But it should not fetch myProject, since it is already local and nowhere remote.
Any ideas? What would be the best way to reach to a ready to use sdcard image without the need to change recipes when requirements.txt changes?
You should use RDEPENDS_${PN} to take care of your dependencies for your app in the recipe.
For example, assuming your python app needs aws-iot-device-sdk-python module, you should add it to RDEPENDS in the recipe. In your case, it would be like this:
RDEPENDS_${PN} = "python3 \
python3-pip \
python3-aws-iot-device-sdk-python \
"
Here's the link showing the Python modules supported by OpenEmbedded Layer.
https://layers.openembedded.org/layerindex/branch/master/layer/meta-python/
If the modules you need are not there, you will likely need to create recipes for the modules.
My newest findings:
Yocto/bitbake seems to suppress interpreting the requirements, because this breaks automatic dependency resolving what could lead to conflicts.
Reason: The required modules from setup.py would not be stored as independent packages, but as part of my package. So, bitbake does not know about this modules what could conflict with other packages that probably requires same modules in different versions.
What was in my recipe:
MY_INSTALL_ARGS = "--root=${D} \
--prefix=${prefix} \
--install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
do_install() {
PYTHONPATH=${PYTHON_SITEPACKAGES_DIR} \
${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install ${MY_INSTALL_ARGS}
}
If I execute this outside of bitbake as python3 setup.py install ${MY_INSTALL_ARGS}, all will be installed correctly, but in the recipe, no requirements are installed.
There is a parameter --no-deps, but I didn't find where it is set.
I think there could be one possibility to exploit the requirements out of setup.py:
Find out where to disable --no-deps in the openembedded/poky layer for easy_install.
Creating a separate PYTHON_SITEPACKAGES_DIR
Install this separate PYTHON_SITEPACKAGES_DIR in eg the home directory as private python modules dir.
This way, no python module would trigger a conflict.
Since I do not have the time to experiment with this, I'll define now one recipe per requirement.
You try installing pip?
Debian
apt-get install python-pip
apt-get install python3-pip
Centos
yum install python-pip
I am facing this problem while moving the python-package directory of XGBoost.
Traceback (most recent call last): File "setup.py", line 19, in LIB_PATH = libpath'find_lib_path' File "xgboost/libpath.py", line 46, in find_lib_path 'List of candidates:\n' + ('\n'.join(dll_path))) builtin.XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
Could anyone explain to me how to fix it?
thanks in advance.
You get that message when trying to install the xgboost Python package without the xgboost binaries present. The proper way to install the xgboost Python package from source is the following (assuming you have a compiler such as gcc installed):
git clone --recursive https://github.com/dmlc/xgboost.git
cd xgboost
./build.sh
cd python-package
python setup.py install
I prefer to do it inside a virtual environment. Note that the option --recursive when cloning the repo is essential since it will also clone folders from different repos such as dmlc-core which are necessary for building xgboost.
The other answers didn't work for me so I installed xgboost through Conda commands as listed here.
Just run conda install -c conda-forge xgboost
The first answer's suggestions did not work for me, and left me with the same error as the original question.
If I'm assuming correctly that your full error message is something like this:
C:\Users\Matt\xgboost\python-package>python setup.py install
Traceback (most recent call last):
File "setup.py", line 19, in <module>
LIB_PATH = [os.path.relpath(libfile, CURRENT_DIR) for libfile in libpath['find_lib_path']()]
File "xgboost/libpath.py", line 49, in find_lib_path
'List of candidates:\n' + ('\n'.join(dll_path)))
XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
List of candidates:
C:\Users\Matt\xgboost\python-package\xgboost\xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\../../lib/xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\./lib/xgboost.dll
C:\Users\Matt\AppData\Local\Programs\Python\Python35\xgboost\xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\../../windows/x64/Release/xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\./windows/x64/Release/xgboost.dll
then the solution is to
1) Get/find/download the library that setup.py is looking for. Search the xgboost folder for .dll files. See if you can find something like xgboost.dll it might be called libxgboost.dll. If you can, move to step 2. If you cannot find it download it here
2) Copy the .dll file into the xgboost/python-package/xgboost folder. If that .dll is not called xgboost.dll (i.e. if it's called libxgboost.dll) then change the name to xgboost.dll
3) Run the commands as outlined in Gustavo answer. Note these are meant to be run from Git Bash.
If you want a more soup-to-nuts tutorial, this was the best one I found.
thanks to Joe Nyland that found the a good answer for this problem in here.
As he said (and also worked for me) you need to run following commands:
$ brew install gcc#5
$ pip install xgboost
Try this it worked for me:
brew install gcc-5
brew install cmake
pip install xgboost
In my case (Ubuntu 16.04, with CUDA 9.0) the Python library build was evidently broken by adding R library package to the cmake options list:
1) python lib installation works OK:
cmake .. -DUSE_CUDA=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-${CUDA_VER}
cd /tmp/xgboost/python-package && python3 setup.py install
2) adding '-DR_LIB=ON' switch to cmake breaks subsequent python library installation attempt:
cmake .. -DUSE_CUDA=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-${CUDA_VER} -DR_LIB=ON
cd /tmp/xgboost/python-package && python3 setup.py install
> "XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root
> path?"
I had the same issues. I downloaded the file (xgboost.dll) from
(https://picnet.com.au/blog/xgboost-windows-x64-binaries-for-download/)
into the already existing xgboost folder
(C:\Users\Naganandini\AppData\Roaming\Python\Python37\site-packages\xgboost).
It's perfectly working now :)
I faced the same error while installing xgboost using anaconda prompt because the installation of xgboost was interfering with the versions of some other pre-installed libraries. Installing xgboost by creating a virtual environment solved the issue.
On macOS and Linux:
python3 -m pip install --user virtualenv #Install virtualenv module
python3 -m venv env #Create a virtual environment 'env'
source env/bin/activate #Activate virtual environment
pip install xgboost #Install xgboost
On Windows:
py -m pip install --user virtualenv #Install virtualenv module
py -m venv env #Create a virtual environment 'env'
.\env\Scripts\activate #Activate virtual environment
pip install xgboost #Install xgboost
Refer https://packaging.python.org/guides/installing-using-pip-and-virtual-environments for help regarding virtual environment.
Search for xgboost.dll file under your C:\Users<your-user-name> directory.
If you have installed xgboost, somewhere you should find a copy of the xgboost.dll file.
Once you find the xgboost.dll file, just copy it to one of the paths mentioned in the error message,
e.g. C:\Users<your-user-name>\AppData\Roaming\Python\Python37\site-packages\xgboost\ . Import should work now.
1.I started off with the conda installation from anaconda prompt:
(base) C:\Users\abhi.b>conda install -c anaconda py-xgboost
This lead to the following error :
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.|
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- py-xgboost -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
Decided not to followup this path, as exploring python downgrade was not an option for me.
2.There after i followed "The final attempt that works " on the below link :
https://medium.com/analytics-vidhya/this-is-what-you-should-do-if-you-fail-to-install-the-latest-or-an-older-version-of-xgboost-on-7942a7641eee
However i got the same error as other users faced ::
>>> import xgboost as xgb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\__init__.py", line 11, in <module>
from .core import DMatrix, Booster
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\core.py", line 161, in <module>
_LIB = _load_lib()
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\core.py", line 123, in _load_lib
lib_paths = find_lib_path()
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\libpath.py", line 45, in find_lib_path
raise XGBoostLibraryNotFound(
xgboost.libpath.XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
List of candidates:
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\../../lib/xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\./lib/xgboost.dll
C:\Users\abhi.b\Anaconda3\xgboost\xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\../../windows/x64/Release/xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\./windows/x64/Release/xgboost.dll
>>> exit()
BTW the link to download xgboost.dll as posted above -
https://picnet.com.au/blog/xgboost-windows-x64-binaries-for-download/
was not clear to me ,so i decided to not follow this thread.
3.Finally ,the solution was very simple for me,Open ANACONDA PROMPT and Type
(base) C:\Users\abhi.b>pip install xgboost
This lead to
Collecting xgboost
Downloading xgboost-1.2.1-py3-none-win_amd64.whl (86.5 MB)
|████████████████████████████████| 86.5 MB 1.7 MB/s
Requirement already satisfied: scipy in c:\users\abhi.b\anaconda3\lib\site-packages (from xgboost) (1.5.0)
Requirement already satisfied: numpy in c:\users\abhi.b\anaconda3\lib\site-packages (from xgboost) (1.18.5)
Installing collected packages: xgboost
Successfully installed xgboost-1.2.1
Then goto python prompt
(base) C:\Users\abhi.b>python
>>>import xgboost as xgb
Or you can type the same command on IDE ie Spyder etc
Voila!!
I am trying to export a GeoTiff with Blender using the Blender Python API (based on Python 3), so I've decided to install GDAL on Ubuntu (14.04). What I would like is to get the module as a standalone folder that I could put in the modules directory of Blender (/home/user/.config/blender/2.73/scripts/modules).
The thing is I've run through several different problems trying to install GDAL. I've tried to install from source (for GDAL 2.0.0) here : Official PyPi Gdal
I ran sudo apt-get install libgdal-dev gdal-bin (I list it here because it may be important)
When I am in the extracted GDAL folder, using python setup.py build & python setup.py install, the library installs to /usr/local/lib/python2.7/dist-packages/osgeo. However, when I run python from command line, running from osgeo import osr returns ImportError: No module named _gdal
Following GDAL via pip , I used pip (pip install GDAL) to install the library, and the folder it went to was /usr/lib/python3/dist-packages/osgeo (using pip show ...). Again, running python3 and trying to import results in the same error. Of course, when I copy-paste each folder in the blender module directory, I get the same error in the Blender Python console.
So I decided to compile the sources using ./configure --with-python & make & make install in the source folder. I then copied the folder GDAL-x.x.x/build/lib.linux-x86_64-3.4/osgeo to the blender modules directory and got this time the error when importing : ImportError: /home/yvesu/.config/blender/2.73/scripts/modules/osgeo/_gdal.so: undefined symbol: _Py_ZeroStruct.
Trying to compile with python3 using python3 setup.py build returns the error error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
EDIT 1:
I think I've found the solution : I went to the directory swig/python (not found in a GDAL-1.11.0 folder but gdal-1.11.0 fodler, can't remember where I downloaded it from), ran python3 setup.py build & python3 setup.py install and could finally find the folder in /usr/local/lib/python3.4/dist-packages/GDAL-1.11.0-py3.4-linux-x86_64.egg/osgeo. When I put this osgeo folder oni the Blender modules directory, I was able to import osgeo in Blender. I will report if anything went wrong.
I think I've listed all my attempts at installing GDAL on Ubuntu. Can anyone point me in the right direction? Do you think it is even possible to install it as a standalone module, or do I need linked libraries through LD_LIBRARY_PATH?
Here is the solution I've found :
Download Gdal sources (v2.0.0 is the current stable release) from ftp://ftp.remotesensing.org/gdal/2.0.0/ or http://download.osgeo.org/gdal/2.0.0/ and untar
Go to the directory gdal2.0.0/swig/python
Run python3 setup.py build & python3 setup.py install
Finally find the module folder in, on Ubuntu : /usr/local/lib/python3.4/dist-packages/GDAL-2.0.0-py3.4-linux-x86_64.egg/osgeo
I can now use it in Blender (copying in the modules directory)