We Keep Coding

pip install -e . vs setup.py

I have been locally editing (inside a conda env) the package GSTools cloned from the github repo https://github.com/GeoStat-Framework/GSTools, to adapt it to my own purposes. The package is c++ wrapped in python (cython).
I've thus far used pip install -e . in the main package dir for my local changes. But I want to now use their OpenMP support by setting the env variable export GSTOOLS_BUILD_PARALLEL=1 . Then doing pip install -e . I get among other things in the terminal ...
Installing collected packages: gstools
Running setup.py develop for gstools
Successfully installed gstools-1.3.6.dev37
The issue is nothing actually changed because, setup.py (shown below) is supposed to print "OpenMP=True" if the env variable is set to GSTOOLS_BUILD_PARALLEL=1 in the linux terminal , and print something else if its not set to 1.
here is setup.py.
# -*- coding: utf-8 -*-
"""GSTools: A geostatistical toolbox."""
import os
​
import numpy as np
from Cython.Build import cythonize
from extension_helpers import add_openmp_flags_if_available
from setuptools import Extension, setup
​
# cython extensions
CY_MODULES = [
Extension(
name=f"gstools.{ext}",
sources=[os.path.join("src", "gstools", *ext.split(".")) + ".pyx"],
include_dirs=[np.get_include()],
define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")],
)
for ext in ["field.summator", "variogram.estimator", "krige.krigesum"]
]
# you can set GSTOOLS_BUILD_PARALLEL=0 or GSTOOLS_BUILD_PARALLEL=1
if int(os.getenv("GSTOOLS_BUILD_PARALLEL", "0")):
added = [add_openmp_flags_if_available(mod) for mod in CY_MODULES]
print(f"## GSTools setup: OpenMP used: {any(added)}")
else:
print("## GSTools setup: OpenMP not wanted by the user.")
​
# setup - do not include package data to ignore .pyx files in wheels
setup(ext_modules=cythonize(CY_MODULES), include_package_data=False)
I tried instead just python setup.py install but that gives
UNKNOWN 0.0.0 is already the active version in easy-install.pth
Installed /global/u1/b/benabou/.conda/envs/healpy_conda_gstools_dev/lib/python3.8/site-packages/UNKNOWN-0.0.0-py3.8-linux-x86_64.egg
Processing dependencies for UNKNOWN==0.0.0
Finished processing dependencies for UNKNOWN==0.0.0
and import gstools
no longer works correctly.
So how can I install my edited version of the package with OpenMP support?

developer of GSTools here.
I guess you don't see the printed message, because pip is suppressing output for the setup. So you could try making pip verbose with:
GSTOOLS_BUILD_PARALLEL=1 pip install -v -e .
BTW, we are always interested in enhancements. So maybe you are willing the share your edits on GSTools? :-)
Cheers,
Sebastian

ImportError: dynamic module does not define module export function (PyInit_cv_bridge_boost)

When I built my program with catkin build, I got the following error:
File "/opt/ros/melodic/lib/python2.7/dist-packages/cv_bridge/core.py", line 91, in encoding_to_cvtype2
from cv_bridge.boost.cv_bridge_boost import getCvType
ImportError: dynamic module does not define module export function (PyInit_cv_bridge_boost)
After searching for the cause, I found that cv_bridge is built with python2 by default.
My environment is as follows.
Virtualenv (python3.6)
ROS melodic
Jetson AGX xavier
I wanted to use cv_bridge with python3, so I used the link below to build cv_bridge with the following steps.
Unable to use cv_bridge with ROS Kinetic and Python3
sudo apt-get install python-catkin-tools python3-dev python3-catkin-pkg-modules python3-numpy python3-yaml ros-kinetic-cv-bridge
# Create catkin workspace
mkdir catkin_ws
cd catkin_ws
source ~/virtualenv/jetson/bin/activate
catkin init
# Instruct catkin to set cmake variables
catkin config -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.6m -DPYTHON_LIBRARY=/usr/lib/aarch64-linux-gnu/libpython3.6m.so
# Instruct catkin to install built packages into install place. It is $CATKIN_WORKSPACE/install folder
catkin config --install
# Clone cv_bridge src
git clone https://github.com/ros-perception/vision_opencv.git src/vision_opencv
# Find version of cv_bridge in your repository
apt-cache show ros-kinetic-cv-bridge | grep Version
cd src/vision_opencv/
git checkout 1.13.0(maybe, I forgot)
cd ../../
# Build
catkin build cv_bridge
# Extend environment with new package
source install/setup.bash --extend
After performing this procedure, from cv_bridge.boost.cv_bridge_boost import getCvType no longer causes an error.
After that, I put other nodes I created in catkin_ws and built it with catkin build. However, when I run it with roslauch, I get the following error:
RLException: [sample.launch] is neither a launch file in package [sample_proc] nor is [sample_proc] a launch file name
Running source devel / setup.bash does not eliminate this error.
The ROS_PACKAGE_PATH when building cv_bridge with python3 is as follows.
(jetson) nvidia#nvidia:~/catkin_ws$ echo $ROS_PACKAGE_PATH
/home/nvidia/catkin_ws/install/share:/opt/ros/melodic/share
By default, ROS_PACKAGE_PATH was as follows:
(jetson) nvidia#nvidia:~/catkin_ws$ echo $ROS_PACKAGE_PATH
/opt/ros/melodic/share:/home/nvidia/catkin_ws/src/vision_opencv/cv_bridge:/home/nvidia/catkin_ws/src/ddynamic_reconfigure:/home/nvidia/catkin_ws/src/vision_opencv/image_geometry:/home/nvidia/catkin_ws/src/vision_opencv/opencv_tests:/home/nvidia/catkin_ws/src/pose_msgs:/home/nvidia/catkin_ws/src/output:/home/nvidia/catkin_ws/src/pose_check_proc:/home/nvidia/catkin_ws/src/post_proc:/home/nvidia/catkin_ws/src/realsense-ros/realsense2_camera:/home/nvidia/catkin_ws/src/realsense-ros/realsense2_description:/home/nvidia/catkin_ws/src/set_datetime_proc:/home/nvidia/catkin_ws/src/vision_opencv/vision_opencv
Do I need to do anything different if I use virtualenv?
Thanks!

How to fix 'No module named cv2' in AWS EC2 Linux, imports from Command Line

I am deploying a web application onto an AWS EC2 instance, and I'm getting an error. The logs indicate that I do not have cv2 installed.
ModuleNotFoundError: No module named 'cv2'
However, if I ssh into my instance, and run python from the shell I can import no problem.
https://drive.google.com/drive/folders/1-w3BN9pMAhkiDM40fODCPdjvU1Nx71UT?usp=sharing
I have already installed opencv onto the Linux server and checked that it is available for import.
From my application.py file
import cv2
File "/opt/python/current/app/localize.py", line 9, in
but from the command line:
>>> import cv2
>>> cv2.__version__
'4.1.0'
I expected the import to work since it works from the command line.

Check if your python package is available for the root/admin user but not accessible for the user trying to run the code?
If you can import that module in your EC2, then it is installed, but more importantly for which user it is installed, and for which version.
First try :
chmod 755 on all directories in python path for your default python and see if it works.(This will provide permissions for all import libraries in Python)
If your script is running Python3.7 and Default is Python2.7 then you might have to do --
sudo pip3 install opencv-python
the way to check the version defaults is:
which python ---Will provide default python path and version
which pip ---- Will Provide default PIP details

As #rayryeng suggested, I'm running Python 3.x from Elastic Beanstalk, and Python 2.x from command line. I fixed it by installing the correct version of cv2 for Python 3 and including the following before my import:
import sys
sys.path.append('/usr/local/lib64/python3.6/site-packages')

Try this:
cd
wget https://github.com/opencv/opencv/archive/3.2.0.zip
virtualenv project
source project/bin/activate
pip install numpy
mkdir local
unzip opencv-3.2.0.zip
cd opencv-3.2.0
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_SHARED_LIBS=NO -D WITH_FFMPEG=ON -D BUILD_opencv_python2=ON -D CMAKE_INSTALL_PREFIX=~/local ~/opencv-3.2.0
make
make install
cp ~/local/lib/python2.7/site-packages/cv2.so ~/project/lib64/python2.7/site-packages/
Read more from here: Source

Can't install BigARTM python library on linux

I'm trying to install BigARTM on my mint following this instruction, all steps seems to be passed successful, for example last one:
*#*​-PC ~ $ cd ~/bigartm/python
​*#*​-PC ~/bigartm/python $ sudo python setup.py install
returns: ...Finished processing dependencies for bigartm==0.8.0
So import artm gives me import error (this is how it might be imported)
What might be wrong?

Installing GDAL Python binding in Ubuntu to use as standalone module

I am trying to export a GeoTiff with Blender using the Blender Python API (based on Python 3), so I've decided to install GDAL on Ubuntu (14.04). What I would like is to get the module as a standalone folder that I could put in the modules directory of Blender (/home/user/.config/blender/2.73/scripts/modules).
The thing is I've run through several different problems trying to install GDAL. I've tried to install from source (for GDAL 2.0.0) here : Official PyPi Gdal
I ran sudo apt-get install libgdal-dev gdal-bin (I list it here because it may be important)
When I am in the extracted GDAL folder, using python setup.py build & python setup.py install, the library installs to /usr/local/lib/python2.7/dist-packages/osgeo. However, when I run python from command line, running from osgeo import osr returns ImportError: No module named _gdal
Following GDAL via pip , I used pip (pip install GDAL) to install the library, and the folder it went to was /usr/lib/python3/dist-packages/osgeo (using pip show ...). Again, running python3 and trying to import results in the same error. Of course, when I copy-paste each folder in the blender module directory, I get the same error in the Blender Python console.
So I decided to compile the sources using ./configure --with-python & make & make install in the source folder. I then copied the folder GDAL-x.x.x/build/lib.linux-x86_64-3.4/osgeo to the blender modules directory and got this time the error when importing : ImportError: /home/yvesu/.config/blender/2.73/scripts/modules/osgeo/_gdal.so: undefined symbol: _Py_ZeroStruct.
Trying to compile with python3 using python3 setup.py build returns the error error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
EDIT 1:
I think I've found the solution : I went to the directory swig/python (not found in a GDAL-1.11.0 folder but gdal-1.11.0 fodler, can't remember where I downloaded it from), ran python3 setup.py build & python3 setup.py install and could finally find the folder in /usr/local/lib/python3.4/dist-packages/GDAL-1.11.0-py3.4-linux-x86_64.egg/osgeo. When I put this osgeo folder oni the Blender modules directory, I was able to import osgeo in Blender. I will report if anything went wrong.
I think I've listed all my attempts at installing GDAL on Ubuntu. Can anyone point me in the right direction? Do you think it is even possible to install it as a standalone module, or do I need linked libraries through LD_LIBRARY_PATH?

Here is the solution I've found :
Download Gdal sources (v2.0.0 is the current stable release) from ftp://ftp.remotesensing.org/gdal/2.0.0/ or http://download.osgeo.org/gdal/2.0.0/ and untar
Go to the directory gdal2.0.0/swig/python
Run python3 setup.py build & python3 setup.py install
Finally find the module folder in, on Ubuntu : /usr/local/lib/python3.4/dist-packages/GDAL-2.0.0-py3.4-linux-x86_64.egg/osgeo
I can now use it in Blender (copying in the modules directory)