I'm trying to follow the instructions here to install ParallelFDTD's python bindings on a windows machine. I've barely worked with C++ before, and certainly never with Boost! Mostly have experience with Python.
So far, I have done the following:
installed the Windows SDK
installed the CUDA toolkit
I have successfully created a conda environment:
conda create -n PFDTD -c conda-forge boost py-boost cmake numpy scipy
conda activate PFDTD
Running conda list shows that boost 1.73 is installed, along with py-boost 1.73 and libboost 1.73 as well as boost-cpp 1.68.
So far so good.
When I try to install and build the library as per the instructions with the following, I get an error
pip install git+https://github.com/AaltoRSE/ParallelFDTD.git
produces:
Building wheels for collected packages: pyParallelFDTD
Running command python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-38
creating build\lib.win-amd64-cpython-38\pyParallelFDTD
copying dist\libPyFDTD\__init__.py -> build\lib.win-amd64-cpython-38\pyParallelFDTD
running build_ext
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22000.
-- The CXX compiler identification is MSVC 19.33.31630.0
-- The CUDA compiler identification is NVIDIA 11.8.89
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CONDA_PREFIX C:\ProgramData\Anaconda3\envs\opti-acoustics
-- Found Python: C:/ProgramData/Anaconda3/envs/opti-acoustics/python.exe (found version "3.8.13") found components: Interpreter
-- Found PythonLibs: C:/Python310/libs/python310.lib (found version "3.10.0")
-- CONDA_PREFIX C:\ProgramData\Anaconda3\envs\opti-acoustics
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
CMake Error at C:/ProgramData/Anaconda3/envs/opti-acoustics/Library/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Boost (missing: Boost_INCLUDE_DIR system thread date_time
unit_test_framework python38 chrono numpy38) (Required is at least version
"1.41")
Call Stack (most recent call first):
C:/ProgramData/Anaconda3/envs/opti-acoustics/Library/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
C:/ProgramData/Anaconda3/envs/opti-acoustics/Library/share/cmake-3.24/Modules/FindBoost.cmake:2376 (find_package_handle_standard_args)
CMakeLists.txt:86 (find_package)
-- Configuring incomplete, errors occurred!
See also "C:/Users/../AppData/Local/Temp/pip-req-build-3mu877y3/build/temp.win-amd64-cpython-38/Release/CMakeFiles/CMakeOutput.log".
See also "C:/Users/../AppData/Local/Temp/pip-req-build-3mu877y3/build/temp.win-amd64-cpython-38/Release/CMakeFiles/CMakeError.log".
The key part of the error message seems to be:
Could NOT find Boost (missing: Boost_INCLUDE_DIR system thread date_time
unit_test_framework python38 chrono numpy38) (Required is at least version
"1.41")
The instructions suggest that the installation with conda should be sufficient, but it seems some environment variables are missing. In ParallelFDTD's CMakeLists.txt, you can see how it tries to handle boost installation in a conda env here.
It appears that conda install -c conda-forge boost py-boost installed successfully, for instance there are a lot of .hpp files and a /python folder as well as a bunch of other directories in C:\ProgramData\Anaconda3\envs\acoustics_env\library\include\boost. There are also a lot of boost_xxx.dll files in C:\ProgramData\Anaconda3\envs\acoustics_env\library\bin.
So far, I've forked the ParallelFDTD library so that I can make changes to the CMakeLists.txt, including pointing the BOOST_ROOT, Boost_INCLUDE_DIR and/or Boost_LIBRARY_DIR explicitly to folders in the anaconda env if need be, but I haven't been able to figure out the correct paths to use... I can also try installing boost manually, but just not sure what the best approach is.
Desperate for help! I'm sure it's a simple step I am missing...
Found lots of other threads that were somewhat related, but none of them seemed to be dealing with this kind of context of working in a conda environment with boost/py-boost.
Related
I'm trying to install the pyarrow==0.17.1 package on my MacBook pro machine (Apple M1 chip) and I'm getting the following error
-- Found PkgConfig: /opt/homebrew/bin/pkg-config (found version "0.29.2")
-- Found Arrow: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/include (found version "7.0.0")
-- Arrow version: 7.0.0 (HOME: /opt/homebrew/Cellar/apache-arrow/7.0.0_3)
-- Arrow SO and ABI version: 700
-- Arrow full SO version: 700.0.0
-- Found the Arrow core shared library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow.dylib
-- Found the Arrow core import library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow.dylib
-- Found the Arrow core static library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow.a
-- Found ArrowPython: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/include (found version "7.0.0")
-- Found the Arrow Python by HOME: /opt/homebrew/Cellar/apache-arrow/7.0.0_3
-- Found the Arrow Python shared library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow_python.dylib
-- Found the Arrow Python import library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow_python.dylib
-- Found the Arrow Python static library: /opt/homebrew/Cellar/apache-arrow/7.0.0_3/lib/libarrow_python.a
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/6m/ychzdp6s7l390zfm77myvxzr0000gn/T/pip-install-lv5qrivl/pyarrow_12e0171ed6fa433da71d5392c3e10c3a/build/temp.macosx-12.0-arm64-3.7
-- Finished cmake for pyarrow
-- Running cmake --build for pyarrow
cmake --build . --config release --
[ 6%] Compiling Cython CXX source for lib...
[ 6%] Built target lib_pyx
[ 13%] Building CXX object CMakeFiles/lib.dir/lib.cpp.o
/private/var/folders/6m/ychzdp6s7l390zfm77myvxzr0000gn/T/pip-install-lv5qrivl/pyarrow_12e0171ed6fa433da71d5392c3e10c3a/build/temp.macosx-12.0-arm64-3.7/lib.cpp:774:10: fatal error: 'arrow/python/config.h' file not found
#include "arrow/python/config.h"
^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/lib.dir/lib.cpp.o] Error 1
make[1]: *** [CMakeFiles/lib.dir/all] Error 2
make: *** [all] Error 2
error: command 'cmake' failed with exit status 2
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
I couldn't find a proven solution for this problem, the only reference I did review is the following - https://github.com/apache/arrow/issues/2281, as mentioned, no proven solution there.
Versions I'm using are:
python 3.7.12
pip 22.0.4
On my machine, I have the following installed - apache-arrow, apache-arrow-glib, cmake.
I've also tried to install the package with the --no-use-pep517 flag as the following- pip install --no-use-pep517 pyarrow==0.17.1 - no success there.
I took a peek inside arrow/python/ and config.h is obviously missing, I also look inside arrow/ which does contains a config.h, however, it's not the config.h that was required.
Does anyone have an idea how to solve this issue?
pyarrow==0.17.1 predates M1 chips (May 2020 vs November 2020).
Any particular reason you wouldn't want to use pyarrow==7.0.0? Your brew installed arrow seems to be 7.0.0 too.
Link for the issue raised in github: github issue link for reference
Application Version: CURA 3.6 version
Platform:
Processor: Intel(R) Core(TM) i5-8300H CPU # 2.30 GHz
RAM: 16.0 GB
System type: 64-bit Operating system, x64-based processor
Steps to Reproduce
I am trying to run source from https://github.com/Ultimaker/Cura/wiki/Running-Cura-from-Source-on-Windows#microsoft-visual-studio-2019.
And attempting to run step 10.
10:
Open CMake and point its source directory to your cura-build repository and the build directory to the build subdirectory.
Actual Results
D:\work\cura-build-master\cura-build-master\build>cmake -DCMAKE_INSTALL_PREFIX=../install_dir -DCMAKE_BUILD_TYPE=Release -G "NMake Makefiles" ..
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/MicrosoftVisualStudio14.0/VC/bin/amd64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/MicrosoftVisualStudio14.0/VC/bin/amd64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at C:/Program Files/CMake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter)
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
C:/Program Files/CMake/share/cmake-3.18/Modules/FindPython/Support.cmake:2966 (find_package_handle_standard_args)
C:/Program Files/CMake/share/cmake-3.18/Modules/FindPython3.cmake:389 (include)
CMakeLists.txt:94 (find_package)
-- Configuring incomplete, errors occurred!
See also "D:/work/cura-build-master/cura-build-master/build/CMakeFiles/CMakeOutput.log".
Expected results
CMake should generate all files in build directory.
From the above error it is evident that the CMake is not able to find python executable.
I have no python installed on system. The only python available is from the CURA (which is installed when the cura-build-environment is build). How to link or make the CMake find the python executable which is installed?
For my case it is installed/found the executable in this folder:
\cura-build-environment-master\install_dir\bin
I have checked and found that we can set the python executable:
for windows:
set(Python3_EXECUTABLE D:/cura-build-environment-master/install_dir/bin/python.exe)
I am using buildroot tagged 2017.11.2 (building for aarch64). I enable build of python3 library for opencv3 in the buildroot configuration:
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_OPENCV3=y
BR2_PACKAGE_OPENCV3_LIB_PYTHON=y
I can see during the build that cmake says:
-- Host: Linux 4.13.0-36-generic x86_64
-- Target: Linux aarch64
-- C++ Compiler: /home/buildroot/output/host/bin/aarch64-linux-gnu-g++ (ver 6.4.1)
But later I see
[100%] Linking CXX shared module ../../lib/python3/cv2.cpython-36m-x86_64-linux-gnu.so
[100%] Built target opencv_python3
I would expect cv2.cpython-36m-x86_64-linux-gnu.so to be named cv2.cpython-36m-aarch64-linux-gnu.so. What can I do to fix this?
It looks like the required distutils environment variables are not set. Can you try setting OPENCV3_CONF_ENV to $(PKG_PYTHON_DISTUTILS_ENV)? If that works, please prepare a patch and send it to the Buildroot mailing list.
I'm trying to compile dlib on my computer (OS X 10.10) but am running into an error I'm not sure how to fix. When I run CMake I get this output
Configuring cmake ...
-- The C compiler identification is AppleClang 7.0.0.7000176
-- The CXX compiler identification is AppleClang 7.0.0.7000176
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find Boost
-- Boost version: 1.59.0
-- Found the following Boost libraries:
-- python
CMake Error at /usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find PythonLibs: Found unsuitable version "2.7.10", but required
is at least "3.4" (found PYTHON_LIBRARY-NOTFOUND)
Call Stack (most recent call first):
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:386 (_FPHSA_FAILURE_MESSAGE)
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/FindPythonLibs.cmake:205 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
/Users/gabriel/Devspace/dlib/dlib/add_python_module:64 (FIND_PACKAGE)
CMakeLists.txt:6 (include)
-- Configuring incomplete, errors occurred!
See also "/Users/gabriel/Devspace/dlib/tools/python/build/CMakeFiles/CMakeOutput.log".
error: cmake configuration failed!
First of all I'm wondering how CMake could not find Boost but then in the next line give the version of boost I currently have installed on my computer? Second, I'm wondering how I go about fixing the pythonlibs error? I've tried Googling around but mainly have just found other people with similar errors but not what it means or how to fix it. Any help would be greatly appreciated. Thanks
Is that your boost installed with python 3?If so, I suggest you to set your default python version 3.x, or reinstall boost with the current python version like this:
To compile Boost.Python yourself download boost from boost.org and then go into the boost root folder and run these commands:
./bootstrap.sh --with-libraries=python
./b2
sudo ./b2 install
I've solved the problem Could NOT find Boost with the above solution.
I'm trying to cross compile opencv for my beaglebone black. All seemed well until I added python support to the cmake flags, since im going to be needing the python wrappers for my project.
I have numpy and python-dev installed, and I'm currently running python 2.7.3.
I installed both with:
sudo apt-get install numpy
sudo apt-get install python-dev
This is what I'm getting:
Scanning dependencies of target opencv_python
[ 92%] Building CXX object modules/python/CMakeFiles/opencv_python.dir/src2/cv2.cpp.o
**Linking CXX shared library** ../../lib/cv2.so
/usr/lib/gcc/arm-linux-gnueabi/4.6/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpython2.7.so when searching for -lpython2.7
/usr/lib/gcc/arm-linux-gnueabi/4.6/../../../../arm-linux-gnueabi/bin/ld: skipping incompatible /usr/lib/libpython2.7.a when searching for -lpython2.7
/usr/lib/gcc/arm-linux-gnueabi/4.6/../../../../arm-linux-gnueabi/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
make[2]: *** [lib/cv2.so] Error 1
make[1]: *** [modules/python/CMakeFiles/opencv_python.dir/all] Error 2
make: *** [all] Error 2
I'm using the available toolchain provided in the linux dist of opencv for arm devices. I really didn't change anything else aside from that.
Is there something wrong with my python dependencies?
cmake flags used:
cmake -DSOFTFP=ON -DENABLE_NEON=ON –D BUILD_ZLIB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -DCMAKE_TOOLCHAIN_FILE=../opencv-2.4.5/platforms/linux/arm-gnueabi.toolchain.cmake ../
I appreciate any help
As your CMake reports, it finds only x86 variant of Python. So you'll need some embedded Linux distro like Buildroot or OpenEmbedded to avoid dealing with dependency hell, i.e. you'll need to cross-compile Python, numpy and all its dependencies and embedded Linux distro would take this job from you.
But there are more issues. One of the biggest problems lies in numpy dependencies like LAPACK, BLAS etc. These libraries are not very good suited for cross compilation.
OpenEmbedded seems to have recipes for OpenCV with Python support. In Buildroot it is still a work in progress.