We Keep Coding

Why does onnxruntime fail to create CUDAExecutionProvider in Linux(Ubuntu 20)?

import onnxruntime as rt
ort_session = rt.InferenceSession(
"my_model.onnx",
providers=["CUDAExecutionProvider"],
)
onnxruntime (onnxruntime-gpu 1.13.1) works (in Jupyter VsCode env - Python 3.8.15) well when providers is ["CPUExecutionProvider"]. But for ["CUDAExecutionProvider"] it sometimes(not always) throws an error as:
[W:onnxruntime:Default, onnxruntime_pybind_state.cc:578 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/reference/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.
I tried following the provided link in the error, and tried different setups in the conda environment to test the code with various version combinations.

Replacing:
import onnxruntime as rt
with
import torch
import onnxruntime as rt
somehow perfectly solved my problem.

download_and_prepare() rock_paper_scissors comes with error

Hei,
I try to use dataset "rock_paper_scissors" from tensorflow_datasets.But it does not work when download and prepares the dataset, even after I have installed tfds-nightly.
The main part of the codes are written like this:
import tensorflow_datasets as tfds
hands_builder = tfds.builder("rock_paper_scissors")
hands_builder.download_and_prepare()
**Errors:
TypeError: Expected binary or unicode string, got WindowsGPath('C:\\Users\\Wilso\\tensorflow_datasets\\downloads\\download.tensorflow.org_rpswkA0RDlYQq0ExTYSutVLV0RD6uF6ByMRei_4e51yGU8.zip')
versions relative to the codes are:
tensorflow-gpu: 2.1.0
tensorflow_datasets: 3.0.0
tfds-nightly : 4.6.0
operating system: Windows-10

AttributeError: module 'tensorflow.python.keras' has no attribute 'applications'

Trying to convert a keras model (Thumbs.h5) into an onnx model on Google Colab, however I am getting an "AttributeError: module 'tensorflow.python.keras' has no attribute 'applications'" error when I run the code.
My code:
from tensorflow.python.keras import backend as K
from tensorflow.python.keras.models import load_model
import onnx
import keras2onnx
onnx_model_name = 'fish-resnet50.onnx'
model = load_model('model-resnet50-final.h5')
onnx_model = keras2onnx.convert_keras(model, model.name)
onnx.save_model(onnx_model, onnx_model_name)
What I've tried:
Updating keras with !pip install keras --upgrade (already updated)
Running it locally with a jupyter notebook on my M1 Mac (V12.4) to get the same error
Pointers or solutions greatly appreciated.

As per the documentation
keras2onnx has been tested on Python 3.5 - 3.8, with tensorflow
1.x/2.0 - 2.2
So install compatible version of tensorflow.
Also keras-onnx is not under active development so use tf2onnx as per the documentation

How to import newly compiled python module?

I have compiled lightgbm with GPU support for python from sources following this guide http://lightgbm.readthedocs.io/en/latest/GPU-Windows.html
Test usage from console was succesful:
C:\github_repos\LightGBM\examples\binary_classification>"../../lightgbm.exe" config=train.conf data=binary.train valid=binary.test objective=binary device=gpu
[LightGBM] [Warning] objective is set=binary, objective=binary will be ignored. Current value: objective=binary
[LightGBM] [Warning] data is set=binary.train, data=binary.train will be ignored. Current value: data=binary.train
[LightGBM] [Warning] valid is set=binary.test, valid_data=binary.test will be ignored. Current value: valid=binary.test
[LightGBM] [Info] Finished loading parameters
[LightGBM] [Info] Loading weights...
Then I tried to import in Python with no luck. It import anaconda version without GPU support:
from sklearn.datasets import load_iris
iris = load_iris()
import lightgbm as lgb
lgtrain = lgb.Dataset(iris.data, iris.target)
lgb_clf = lgb.train(
{
'objective' : 'regression',
'metric' : 'rmse',
'num_leaves' : 350,
#'max_depth': 14,
'learning_rate' : 0.017,
'feature_fraction' : 0.5,
'bagging_fraction' : .8,
'verbosity' : -1 ,
'device' : 'gpu'
},
lgtrain,
num_boost_round=3500,
verbose_eval=100
)
LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'
I believe I have to specify the location but how?

I think this might not be specific to lightGBM, but rather a problem with Anaconda's virtual environment. When working within the Anaconda virtual env, your system paths are modified to point to Anaconda installation directories.
As you point out, this leads to Anaconda loading its own version, rather than the external version you configured, compiled and tested.
There are several ways to force Anaconda to find your package, see this related discussion.
The suggestions that involve running ln -s are only for Linux and Mac, but you can do something similar in Windows.
You could start by uninstalling the Anaconda version of lightGBM, then create a copy of the custom-compiled version within the Anaconda path. You can discover this using
import sys
sys.path

Remove previously installed Python package with the following command:
pip uninstall lightgbm
or
conda uninstall lightgbm
After doing that navigate to the Python package directory and install it with the library file which you've compiled:
cd LightGBM/python-package
python setup.py install --precompile

deepwater with tensorflow on RHEL6

I am trying to get Python + deepwater + tensorflow to run on RHEL 6.7. Using conda, I have installed python 3.6.0, tensorflow 1.1.0 and also gcc 4.8.5. TF is working fine.
I have installed the following libraries using pip install: h2o-3.11.0.3904-py2.py3-none-any.whl and h2o-3.11.0-py2.py3-none-any.whl.
I tried to run the following example from the h2o tutorial
import h2o
from h2o.estimators.deepwater import H2ODeepWaterEstimator
h2o.init()
train = h2o.import_file("https://h2o-public-test-data.s3.amazonaws.com/bigdata/laptop/mnist/train.csv.gz")
features = list(range(0,784))
target = 784
train[target] = train[target].asfactor()
model = H2ODeepWaterEstimator(epochs=100, activation="Rectifier", hidden=[200,200], ignore_const_cols=False,
mini_batch_size=256, input_dropout_ratio=0.1, hidden_dropout_ratios=[0.5,0.5], stopping_rounds=3,
stopping_tolerance=0.05, stopping_metric="misclassification", score_interval=2, score_duty_cycle=0.5,
score_training_samples=1000, score_validation_samples=1000, nfolds=5, gpu=False, seed=1234, backend="tensorflow")
model.train(x=features, y=target, training_frame=train)
The following exception is thrown
Exception: Unable to initialize the native Deep Learning backend: Cannot find TensorFlow native library for OS: linux, architecture: x86_64. See https://github.com/tensorflow/tensorflow/tree/master/java/README.md for possible solutions (such as building the library from source).
Is there anything else that I am missing? Would I need to build the bits from scratch for this platform?