We Keep Coding

how to set up cudf for cuda 9.0

I have Cuda 9.0 version and
I have tried
pip install cudf==0.6.1
ERROR: Could not find a version that satisfies the requirement cudf==0.6.1 (from versions: none)
ERROR: No matching distribution found for cudf==0.6.1

Did you check the documentation?
https://github.com/rapidsai/cudf/blob/master/README.md
You should use conda to install it.

cudf 0.6.1 provides 2 wheels, Python 3.6 and Python 3.7, both for 64-bit Linux. No source code.
Source code and installation instructions are at https://github.com/rapidsai/cudf#installation. You can try to install using conda or compile from sources.

Tensorflow-gpu only trains custom object detection models using version 1.14

I am ecstatic that I have found a setup where I can train and test my custom object detection models but I am confused why the latest tensorflow-gpu version doesn't work on my PC when another setup works on my laptop:
The Setup I am using(PC):
OS: Windows 10
Python: 3.6.4
TensorFlow: 1.14.0(gpu)
CUDA: 10
cudnn: 9.0?(can't remember)
repo + location: https://github.com/tensorflow/models/research/object_detection
The Setup I am using(laptop):
OS: OSX
Python: 3.7.5
TensorFlow: 2.1.0-rc0(standard)
repo + location: https://github.com/tensorflow/models/research/object_detection
how(when?) can I train custom models using tensorflow-gpu v2.0.0 on my PC?
thx

If you need to run tensorflow-gpu on Windows, Microsoft Visual Studio is a pre-requisite. This requirement is also mentioned here. You can download the community version of Visual Studio 2019 here.
Though this requirement is not directly mentioned in tensorflow-gpu installation docs, it is a part of CUDA installation guide for Windows. Also you need to ensure you have a NVIDIA GPU card with CUDA Compute Capability 3.5 or higher. You can check if your GPU meets the Compute Capability requirement here.
Here are a few resources to help with tf-gpu installation:
https://towardsdatascience.com/installing-tensorflow-with-cuda-cudnn-and-gpu-support-on-windows-10-60693e46e781
https://www.youtube.com/watch?v=KZFn0dvPZUQ

Tensorflow GPU / CUDA installation on Ubuntu

I have set up a Ubuntu 18.04 and tried to make Tensorflow 2.2 GPU work (I have an Nvidia/CUDA graphic card) with Python.
Even after reading the documentation https://www.tensorflow.org/install/gpu#linux_setup, it failed (see below for details about how it failed).
Question: would you have a canonical "todo" list (starting point: freshly installed Ubuntu server) on how to install tensorflow-gpu and make it work, with a few steps?
Notes:
I have read many similar forum posts, and I think that having a canonical "todo" (from a fresh Ubuntu install to having tensorflow-gpu working) would be interesting, with a few steps/bash commands
the documentation I used involved
export LD_LIBRARY_PATH...
# Add NVIDIA package repository
sudo apt-key adv --fetch-keys http://developer.download...
...
# Install CUDA and tools. Include optional NCCL 2.x
sudo apt install cuda9.0 cuda...
Even after a lot of trial and errors (I don't copy/paste all the different errors here, would be too long), then at the end:
import tensorflow
always failed. Some reasons included `ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory. I have already read the relevant question here, or this very long (!) Github issue.
After some trial and error, import tensorflow works, but it doesn't use the GPU (see also Tensorflow not running on GPU).

Well, I was facing the same problem. The first thing to do is to look up, which Tensorflow version is required. In your case Tensorflow 2.2. requires CUDA 10.1. The correct cuDNN version is also important. In your case it would be cuDNN 7.4. An additional point is the installed python version. I would recommend Python 3.5-3.8. If one those mismatch, a fully compatibility is almost impossible.
So if you want a check list, here you go:
Install CUDA 10.1 by installing nvidia-cuda-toolkit.
Install the cuDNN version compatible with CUDA 10.1.
Export CUDA environment variables.
If Bazel is not installed, you will be asked on that.
Install TensorFlow 2.2 using pip. I would highly recommend the usage of a virtual environment.
You can find the compatibility check list of Tensorflow and CUDA here
You can find the CUDA Toolkit here
Finally get cuDNN in the correct version here
That's all.

I faced the problem as well when using the Google Cloud Platform for two projects involving deep learning. They provide servers with nothing but a freshly installed Ubuntu OS. Regarding my experience, I recommend doing the following steps:
Look up the cuda and cuDNN version supported by the current Tensorflow release on the Tensorflow page.
Install the targeted cuda version from the deb package retrieved from Nvidias cuda page and be careful that more recent cuda versions might not work! This will automatically install the corresponding Nvidia drivers.
Install the targeted cuDNN version from this page and again be careful that a more recent cuDNN version might not work.
Install tensorflow-gpu using pip.
This should work. Your problem is probably that you are using a more recent cuda version than targeted by the current Tensorflow release.

To install tensorflow-gpu, the guidelines which are provided on official website are very tedious for beginers, instead we can do these simple steps:
Note : NVIDIA driver must be installed before this(you can verify this using command nvidia-smi).
Install Anaconda https://www.anaconda.com/distribution/?
Create an virtual environment using command "conda create -n envname"
Then activate env using command "conda activate envname"
Finally install tensorflow using command "conda install tensorflow-gpu"
With the given code
import tensorflow as tf
if tf.test.gpu_device_name():
print('Default GPU Device{}'.format(tf.test.gpu_device_name()))
else:
print("not using gpu")
You can find the tutorial on link given below
https://www.pugetsystems.com/labs/hpc/Install-TensorFlow-with-GPU-Support-the-Easy-Way-on-Ubuntu-18-04-without-installing-CUDA-1170/?

I would suggest to first check the availability of GPU using nvidia-smi command.
I had faced the same issue, i was able to resolve it by using docker container, you can install docker using Install Docker Engine on Ubuntu or use the Digital Ocean guide (i used this one) How To Install and Use Docker on Ubuntu 18.04
After that it is simple just run the following command based on the requirements
NV_GPU='0' nvidia-docker run --runtime=nvidia -it -v /path/to/folder:/path/to/folder/for/docker/container nvcr.io/nvidia/tensorflow:17.11
NV_GPU='0' nvidia-docker run --runtime=nvidia -it -v /storage/research/:/storage/research/ nvcr.io/nvidia/tensorflow:20.12-tf2-py3
Here '0' represents the GPU number, if you want to use more than one GPU just use '0,1,2' and so on ....
Hope this solves the issue.

ImportError: Could not find 'cudnn64_7.dll' , while importing tensorflow

This is an issue that many of us must have come across. While installing tensorflow, this is one of the error messages that pops up for most of the users. I could not install Tensorflow 1.10.0 due to the following error that I posted a few days back at
ImportError: Could not find 'cudnn64_7.dll'
I am using Windows 10 and was trying to implement
import tensorflow as tf
through Conda environment.
What can I do to resolve this issue?

1) Go to the cuDNN Archive
2) Click on Download cuDNN v7.6.1 (June 24, 2019), for CUDA 10.0
(you need CUDA 10 installed. NOT 10.1. If you installed the wrong version, uninstall
it and install the 10 which works with tensorflow-gpu)
3) Click on the link for your operating system.
4) Unzip it. It should unzip to a folder called CUDA.
5) Go into the CUDA folder and copy the contents
6) Open the installed CUDA 10 location. For windows 10 it is "Download cuDNN v7.6.1 (June 24, 2019), for CUDA 10.0"
7) Paste the contents from your clipboard to the folder.
8) have a coffee. You are done!

Jeremy Demers' answer worked for me, and I was able to repeat his process. However, I used cuDNN 10.1 instead of version 10 and installed tensorflow version 2.4.0-dev20200705 first via pip install tensorflow-gpu, and then `pip install tensorflow-nightly to get the latest build. Hardware: 2060 Super, 8GB.
Edit:
The recommended way to get tensorflow nightly via pip is:
pip install tf-nightly

Here is what I did.
Step 1) Installed 'NVIDIA GEFORCE EXPERIENCE' in my computer to check my Driver version.
Step 2) The driver version was an old one. Update was available. So I updated my Graphic driver.
My GPU properties now are:-
NVIDIA GEFORCE EXPERIENCE Version 3.14.1.48
GeForce 940MX
Driver Version 398.82
Intel(R) Core(TM) i5-7200U CPU #2.50GHz
7.9 GB RAM
Now, through conda environment ( I created an environment named 'tensorflow' ), when I executed the statement
(tensorflow) C:\Users\Arnab Sinha>pip install --ignore-installed --upgrade tensorflow-gpu
I encountered the following message :-
pandas 0.23.4 requires python-dateutil>=2.5.0, which is not installed.
pandas 0.23.4 requires pytz>=2011k, which is not installed.
I then installed the required packages by executing the following commands one after the other
pip install python-dateutil
and
pip install pytz
after which I ran the command in Python 3.6.6
import tensorflow as tf
and then
print(tf.__version__)
which gave the output
1.10.0
Here is how I installed Tensorflow 1.10.0 into my computer. The Anaconda Navigator however does not have the update of Tensorflow 1.10.0. Please inform me if you have found the update for it.

SciPy can't install old version Windows 10

I'm trying to install an older version of SciPy (either 0.16.1 or 0.17.0), but when I try to use pip I get the result:
pip install scipy==0.16.1
I get the following message
Failed building wheel for scipy
So I found the .whl file here https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy but the problem is that the version here is 1.0.1, and I can't seem to find any older ones, even in the SciPy website (there are for linux and mac)
.
.
PS.
I need to use either of this versions because I'm trying to run Theano on a GPU, and the tutorial provided here http://deeplearning.net/software/theano/install_windows.html says this version is a requirement if there is any tutorial that is better and more up-to-date I would be appreciated