We Keep Coding

How to find the right libnvinfer version for Cuda

I am building a Docker image, for deep learning:
cuda:11.2.0-cudnn8-devel-ubuntu20.04
PYTHON_VERSION=3.7.9
For this task I need 3 dependencies to install, but I can't find the right version. The error I get, when building the Docker image:
E: Version '8.1.1.33-1+cuda11.2' for 'libnvinfer8' was not found E:
Version '8.1.1.33-1+cuda11.2' for 'libnvinfer-dev' was not found E:
Version '8.1.1.33-1+cuda11.2' for 'libnvinfer-plugin8' was not found
I was experimenting with other versions as well, but I had no success, so the question is: where/how can I find the right versions which works with cuda 11.2 and ubuntu 20.04. Is there a rule of thumb?

To check which version of TensorRT your tensorflow installation expects:
>>> import tensorflow
>>> tensorflow.__version__
'2.8.0'
>>> from tensorflow.python.compiler.tensorrt import trt_convert as trt
>>> trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version()
(7, 2, 2)
>>> trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version()
2022-03-24 08:59:15.415733: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
The last command shows that indeed libnvinfer.so is missing on your system (you can also check this fact using ldconfig -p | grep libnv).
To install it (adapted from Tensorflow's gpu.Dockerfile), take the TensorRT version in your output from above, double check it's available for your CUDA version on the nvidia repository, and install:
export LIBNVINFER=7.2.2 LIBNVINFER_MAJOR_VERSION=7 CUDA_VERSION=11.0
apt-get install -y libnvinfer${LIBNVINFER_MAJOR_VERSION}=${LIBNVINFER}-1+${CUDA_VERSION} libnvinfer-plugin${LIBNVINFER_MAJOR_VERSION}=${LIBNVINFER}-1+${CUDA_VERSION}
The last Python command above should now start working. If you get a similar error again, double check that the so file locations (check with ldconfig -p | grep libnv) are included in LD_LIBRARY_PATH.
Also double check your CUDA version. In my case, I was running the docker.io/nvidia/cuda:11.5.1-cudnn8-runtime-ubuntu20.04 image, which already contains the math libraries and specifically libnvrtc.so.11.2 (so for a newer CUDA version than TensorRT supports on the nvidia repository). This becomes evident after running the apt-get command above, which gave this output:
The following packages have unmet dependencies:
libnvinfer7 : Depends: cuda-nvrtc-11-0 but it is not going to be installed

When doubt:
https://www.tensorflow.org/install/source#gpu
This was actually the answer and it works like a charm.. follow the official site for Tested build configurations
tensorflow-2.5.0
cuda:11.2.0
PYTHON_VERSION=3.8.11
libcudnn8=8.1.1.33-1+cuda11.2
libcudnn8-dev=8.1.1.33-1+cuda11.2

pycharm error with pyenv virtualenv: The SDK seems invalid

I am experiencing a problem with pycharm when trying to select as interpreter a python bin created through a pyenv virtualenv.
Cannot set up a python SDK
at Python 3.10 (remote_maax) ($HOME/.pyenv/versions/remote_maax/bin/python).
The SDK seems invalid.
I would like to provide more details but pycharm does not provide any additional output.
Even weirder, I am not using python 3.10, the virtualenv has been created with python 3.8.5.
The virtualenv also appears to work when activated manually from the shell with pyenv activate. I can pip install packages and use them correcly.
I am on a new install of pop os 20.10 (ubuntu), kernel 5.8.0-7642-generic, pyenv 1.2.22, pycharm community 2020.3.3 installed via flatpak.
Does anyone know what can cause this?
Edit: found some more details by launching pycharm from the shell
com.jetbrains.python.sdk.InvalidSdkException: Failed to determine Python's sys.path value:
STDOUT:
STDERR: /home/gio/.pyenv/versions/remote_maax/bin/python3.8: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/gio/.pyenv/versions/remote_maax/bin/python3.8)
at com.jetbrains.python.sdk.PythonSdkType.getSysPathsFromScript(PythonSdkType.java:457)
at com.jetbrains.python.sdk.PythonSdkType.getSysPath(PythonSdkType.java:439)
at com.jetbrains.python.sdk.PythonSdkUpdater.evaluateSysPath(PythonSdkUpdater.java:576)
at com.jetbrains.python.sdk.PythonSdkUpdater.getLocalSdkPaths(PythonSdkUpdater.java:436)
at com.jetbrains.python.sdk.PythonSdkUpdater.updateLocalSdkPaths(PythonSdkUpdater.java:400)
at com.jetbrains.python.sdk.PythonSdkUpdater.updateLocalSdkVersionAndPaths(PythonSdkUpdater.java:370)
at com.jetbrains.python.sdk.PythonSdkUpdater.lambda$updateVersionAndPathsSynchronouslyAndScheduleRemaining$0(PythonSdkUpdater.java:267)
at com.intellij.openapi.progress.impl.CoreProgressManager$1.run(CoreProgressManager.java:275)
at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:962)
at com.intellij.openapi.progress.impl.CoreProgressManager$4.run(CoreProgressManager.java:520)
at com.intellij.openapi.progress.impl.ProgressRunner.lambda$new$0(ProgressRunner.java:79)
at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:235)
at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:178)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:658)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:610)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:65)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:165)
at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:235)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
at java.base/java.lang.Thread.run(Thread.java:834)
Unfortunately, I do have libc 2.32 installed, so now I'm even more confused.
glibc-source/groovy,groovy,now 2.32-0ubuntu3 all [installed]
GNU C Library: sources
libc6/groovy,now 2.32-0ubuntu3 amd64 [installed]
GNU C Library: Shared libraries

There seems to a problem with the flatpack version of Pycharm. I got exactly the same problem (PopOS, Pycharm Professional 2020.3). In the end I replaced the flatpack version with the standalone one (I downloaded Pycharm from Jetbrains site) and work just fine in the exactly the same environment.

Runtime error using Python Library Keops using CUDA in Ubuntu18.04

I am trying to run samples from the Python library: GeomLoss, which depends on CUDA, Pytorch and Keops in Ubuntu 18.04.3. I downloaded Python3.7 using Anaconda, and I am using CUDA 10.1. The gcc version is 7.4.0.
When I run samples from GeomLoss, the error message said:
RuntimeError: [KeOps] This KeOps shared object has been compiled
without cuda support: try to set tagHostDevice to 0 or recompile the
formula with a working version of cuda.
I cannot change tagHostDevice to 0 since this will disable GPU calculation according to their documentation. I checked CUDA and Pytorch installation and they was no error.
But when I tried to run the installation checking code from KeOps:
import torch
import pykeops.torch as pktorch
x = torch.arange(1, 10, dtype=torch.float32).view(-1, 3)
y = torch.arange(3, 9, dtype=torch.float32).view(-1, 3)
my_conv = pktorch.Genred('SqNorm2(x-y)', ['x = Vi(3)', 'y = Vj(3)'])
print(my_conv(x, y))
I received error message:
error -- unsupported GNU version! gcc versions later than 6 are not supported! ^~~~~ CMake Error at
keopslibKeOpstorch91c92bd508_generated_link_autodiff.cu.o.Release.cmake:219
I checked CUDA documentation, for Ubuntu 18.04.3, the native linux distribution support should be gcc-7.3.0. for x86_64. I used gcc --version to check default gcc in system and it is using gcc-7.4.0. I am not sure if this is the problem with using KeOps with CUDA and GPU. Also, I believe KeOps will not support gcc versions before 7. So I am really confused about what should I do to fix the problem right now.
I am wondering if anyone has experienced similar problems with GeomLoss and KeOps or other libraries. I am indeed grateful for any suggestions. Thanks!

I did the following steps and it worked for me:
First, by checking the dependencies in this link I noticed that nvcc compiler is not installed. By going to Nvidia Toolkit Installation Guide I did the following steps:
wget https://developer.download.nvidia.com/compute/cuda/11.2.2/local_installers/cuda_11.2.2_460.32.03_linux.run
sudo sh cuda_11.2.2_460.32.03_linux.run
Then I realized that nvcc command not working, so I did add them to the path using:
nano ~/.bashrc
# Add the following two lines:
export PATH="$PATH:/usr/local/cuda/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
In general, it is important to see the logs of PyKeops to check the process. So, you can always change the verbose and debug mode and see the details to check out what failed:
# Testing PyKeops installation
import pykeops
# Changing verbose and mode
pykeops.verbose = True
pykeops.build_type = 'Debug'
# Clean up the already compiled files
pykeops.clean_pykeops()
# Test Numpy integration
pykeops.test_numpy_bindings()

ImportError: Could not find 'cudart64_100.dll

I'm trying to install tensorflow-gpu==2.0.0-beta1 on my Windows 10 machine and got this error:
ImportError: Could not find 'cudart64_100.dll'. TensorFlow requires
that this DLL be installed in a directory that is named in your %PATH%
environment variable. Download and install CUDA 10.0 from this URL:
https://developer.nvidia.com/cuda-90-download-archive
I made all things from:
official documentation: https://www.tensorflow.org/install/gpu
from here: https://medium.com/#teavanist/install-tensorflow-gpu-on-windows-10-5a23c46bdbc7
Checked PATH variable: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin
also have CUDA_PATH : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0 in variables
file C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\cudart64_100.dll exists
Did system restart
But error still occurs
How can I fix this?

The simplest way to fix is to install the latest ‘NVIDIA GPU Computing Toolkit’, because if it's not there, you'll be missing the 'cudart64_100.dll' library.
The only issue is that the latest copy of CUDA has this particular library upgraded to 'cudart64_101.dll', while the latest TensorFlow still requires the older 'cudart64_100.dll'.
Anyways, one way to deal with this issue is to install the latest CUDA + CUDA from September 2018 and then copy 'cudart64_100.dll' library from old install to the new one.
Or just visit my site where I linked the 'cudart64_100.dll' library downloaded from the CUDA Toolkit 10.0 (Sept 2018), to make it easier to copy it into the latest CUDA directory.
Here are some screenshots to illustrate the process: https://www.joe0.com/2019/10/19/how-resolve-tensorflow-2-0-error-could-not-load-dynamic-library-cudart64_100-dll-dlerror-cudart64_100-dll-not-found/

I have faced similar issue. I have added the directory of the cudart64_100.dll file to the PATH variable but still it prompts the error "cudart64_100.dll" not found. In the end I finally manage to make it work by adding the following codes. Hope it helps.
import ctypes
hllDll = ctypes.WinDLL("C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v10.0\\bin\\cudart64_100.dll")

I had similar error:
cudart64_101.dll not found
It is because the latest version of CUDA requires older CUDA-version.dll files to work properly. The solution would be try installing the previous version of CUDA.
Once you have downloaded CUDA 10.1 run the .exe file which will first extract the necessary files in C:\Users\your_user_name\AppData\Local\Temp\CUDA.
Once the extraction is completed do not proceed with the installation navigate to the directory C:\Users\your_user_name\AppData\Local\Temp\CUDA\cudart\bin and here you will find the missing DLL file cudart64_101.dll and cudart32_101.dll copy both the files to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin and then cancel the installation.
Follow the same steps for any CUDA version will work for sure. Hope this helps thank you!!!

I also was dealing with the current situation that tensorflow expects cudart64_101.dll and NVIDIA offers Version 10.2 as main version (including cudart64_102.dll).
I simply installed both versions. I have both versions in the windows path. Beside disc space, I did not have problems so far and GPU is used in tensorflow.

*EDIT
Every Tensorflow version requires a specific version of CUDA.
The easiest way is to open https://www.tensorflow.org/install/gpu and read which version did you need.
ex. CUDA® Toolkit —TensorFlow supports CUDA® 11.2 (TensorFlow >= 2.5.0)
If you want to install Tensor Flow v 2.5.0 you must have exactly CUDA v 11.2 installed.
TensorFlow 2.2 try to import cudart64_101.dll, cusparse64_10.dll and cublas64_10.dll but they are part from CUDA 10.1 If you have OTHER cuda version will get ImportError: Could not find 'cudart64_100.dll as this files are only available in cuda 10.1.
If you want to use older tensorflow version you have to use it with appropriate cuda version

Just rename
\program files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudart64_102.dll
to cudart100.dll

you can find the cudart64_100.dll file in this website link.
and extract it, add cudart64_100.dll to your C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
after you run your python script you will see:
Successfully opened dynamic library cudart64_100.dll

NVidia maintains a download archive of older CUDA driver releases. I would recommend downloading the installer from here instead of a third-party archive:
https://developer.nvidia.com/cuda-toolkit-archive
That page has a link to a page for downloading version 10.0:
https://developer.nvidia.com/cuda-10.0-download-archive
For the CUDA-NN libraries, the archive may be found here:
https://developer.nvidia.com/rdp/cudnn-archive
You need to be a member of the NVidia developer program to access this archive, but it is free to sign-up.
There should be no problem installing multiple versions at once, since the CUDA installer uses a separate installation directory for each version and the library files have different names. You will, however, want to select a custom installation to avoid replacing your display drivers, since you're installing older versions.
The CUDA-NN libraries (at least for Windows) are distributed as zip files. Copy the contents where you would like them (I installed each as a subdirectory under the corresponding CUDA driver installation directory).
Finally, add all of the various CUDA and CUDA-NN directories to your PATH, if they aren't already there so Python and Tensorflow can find them.

The actual fix is to download and install CUDA Toolkit
But if you're in hurry, you can also disable gpu, before importing tensorflow.
import os
# BAD IDEA ::::: Disabling gpu . The actual fix is to download and install cuda toolkit.
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import tensorflow as tf

Renaming the libraries might find a way to solve this issue, but that is not advised, as the root cause for this issue is due to version mismatch of CDNN and cuDNN. It's advised to install the right CDNN and cuDNN versions.
I too have faced similar issues and realized that the issue was with CUDA and CUDNN version mismatch.
Can refer here for the proper CDNN and cuDNN versions. From the reference below for TensorFlow 2.4.0 it is recommended to use CUDA 11.0 and cuDNN 8.0.
Or you can refer here to download cuDNN for suitable CUDA.

Tensorflow GPU setup: error with CUDA on PyCharm

I have TF 0.8 installed on Python3, MacOSX El Capitan.
When running a simple test code for TF I get this message:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10):
Library not loaded: #rpath/libcudart.7.5.dylib
Referenced from: /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow.so
Reason: image not found
My .bash_profile is as follows:
export PATH=/usr/local/bin:$PATH
export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.5/lib:/usr/local/cuda/lib
in /Developer/NVIDIA/CUDA-7.5/lib I have a file called libcudart.7.5.dylib
in /usr/local/cuda/lib I have an alias called libcudart.7.5.dylib
I have tried several permutations of .bash_profile without success. ANy idea what may be wrong?
Note that I can successfully use my GPU with Theano so there's no reason to believe the GPU/cuDNN/CUDA install may be faulty.

If you're getting this error, make sure you installed CUDA, cuDNN correctly as described in the Tensorflow install instructions. Note the TF, CUDA, cuDNN version you're installing and what Python version you're using.
Filenames, paths etc frequently vary, so small tweaks in filenames and paths may be needed in your case if errors are cropping up. Sometimes it's hard for others to help you because your system may have a very specific path setup/version that cannot be understood by someone in a forum.
If you're getting exactly the error I'm describing in the OP, take a step back and check:
is this happening in PyCharm?
is this happening in iPython within PyCharm?
is it happening in both?
is this happening in iPython in Terminal?
In my case it was happening only in PyCharm. In iPython outside of PyCharm (that is using the Mac 'Terminal' software) everything worked fine. But when doing iPython in PyCharm, or when running the test file through PyCharm, I would get the error. This means it has something to do with PyCharm, not the Tensorflow install.
Make sure your DYLD_LIBRARY_PATH is correctly pointing to the libcudart.7.5.dylib file. Navigate there with Finder, do a Spotlight search search and find the file or its alias. Then place that path in your .bash_profile. In my case, this is working:
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib
If your problem is PyCharm, a specific configuration is needed. Go to the upper right corner of the GUI and click the gray down arrow.
Choose "Edit Configuration". You will see an Environment option where you need to click on the ... box and enter the DYLD_LIBRARY_PATH that applies to your case.
Note that there's an Environment option for the specific file you're working on (it will be highlighted in the left panel) and for Defaults (put DYLD_... there as well if you want future files you create to have this). Note that you need to save this config or else when you close PyCharm it won't stick.

As an extension to pepe answer, which is the correct one, I don't mind if the following is integrated to the original answer.
I would like to add that if you wish to make this change permanent in pyCharm (affects only the current project) and not having it to do for every net file, is possible, from the interface shown above by pepe, by going under the "Default" to set the DYLD_LIBRARY_PATH.
Keep in mind, that this change by itself it doesn't alter the run configuration of the current script, which eventually still need to be manually changed ( or deleted and regenerated from the new defaults)

Could you try TF 0.9, which adds the MacOX GPU support?
https://github.com/tensorflow/tensorflow/blob/r0.9/RELEASE.md

It appears that you are not finding CUDA on your system. This could be for a number of reasons including installing CUDA for one version of python while running a different version of python that isn't aware of the other versions installed files.
Please take a look at my answer here.
https://stackoverflow.com/a/41073045/1831325

In my case, tensorflow version is 1.1, the dlopen error happens in both
ipython and pycharm
Environment:
Cuda version:8.0.62
cudnn version:6
error is a little different in pycharm and ipython. I cannot remeber too much detail, but ipython says there is no libcudnn.5.dylib, but pycharm just says there is
import error, image not found
Solution:
Download cudnn version 5, from
https://developer.nvidia.com/rdp/cudnn-download
Unzip the cudnn. Copy lib/ to /usr/local/cuda/lib. Copy include/ to /usr/local/cuda/include
unzip cuda.zip
cd cuda
sudo cp -r lib /usr/local/cuda/lib
sudo cp include/cudnn.h /usr/local/cuda/include
Add the lib directory path to your DYLD_LIBRARY_PATH. like this in my ~/.bash_profile:
export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-8.0/lib:/usr/local/cuda/lib${DYLD_LIBRARY_PATH:+:${DYLD_LIBRARY_PATH}}
In the tensorflow official installation guide, it says need cudnn 5.1, so ,this is all about careless。
https://www.tensorflow.org/install/install_mac
Requirements to run TensorFlow with GPU support.
If you are installing TensorFlow with GPU support using one of the mechanisms described in this guide, then the following NVIDIA software must be installed on your system:
...
The NVIDIA drivers associated with CUDA Toolkit 8.0.
cuDNN v5.1. For details, see NVIDIA's documentation.
...

I think the problem is in the SIP (System Integrity Protection). Restricted processes run with cleared environment variables and you got this error.
You need to go to Recovery Mode, start Terminal and input
$ csrutil disable
, and reboot