We Keep Coding

Make TensorFlow use the GPU on an ARM Mac

I have installed TensorFlow on an M1 (ARM) Mac according to these instructions. Everything works fine.
However, model training is happening on the CPU. How do I switch training to the GPU?
In: tensorflow.config.list_physical_devices()
Out: [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
In the documentation of Apple's TensorFlow distribution I found the following slightly confusing paragraph:
It is not necessary to make any changes to your existing TensorFlow scripts to use ML Compute as a backend for TensorFlow and TensorFlow Addons. There is an optional mlcompute.set_mlc_device(device_name='any') API for ML Compute device selection. The default value for device_name is 'any', which means ML Compute will select the best available device on your system, including multiple GPUs on multi-GPU configurations. Other available options are CPU and GPU. Please note that in eager mode, ML Compute will use the CPU. For example, to choose the CPU device, you may do the following:
# Import mlcompute module to use the optional set_mlc_device API for device selection with ML Compute.
from tensorflow.python.compiler.mlcompute import mlcompute
# Select CPU device.
mlcompute.set_mlc_device(device_name='cpu') # Available options are 'cpu', 'gpu', and 'any'.
So I try to run:
from tensorflow.python.compiler.mlcompute import mlcompute
mlcompute.set_mlc_device(device_name='gpu')
and get:
WARNING:tensorflow: Eager mode uses the CPU. Switching to the CPU.
At this point I am stuck. How can I train keras models on the GPU to my MacBook Air?
TensorFlow version: 2.4.0-rc0

Update
The tensorflow_macos tf 2.4 repository has been archived by the owner. For tf 2.5, refer to here.
It's probably not useful to disable the eager execution fully but to tf. functions. Try this and check your GPU usages, the warning message can be misleading.
import tensorflow as tf
tf.config.run_functions_eagerly(False)
The current release of Mac-optimized TensorFlow has several issues that yet not fixed (TensorFlow 2.4rc0). Eventually, the eager mode is the default behavior in TensorFlow 2.x, and that is also unchanged in the TensorFlow-MacOS. But unlike the official, this optimized version uses CPU forcibly for eager mode. As they stated here.
... in eager mode, ML Compute will use the CPU.
That's why even we set explicitly the device_name='gpu', it switches back to CPU as the eager mode is still on.
from tensorflow.python.compiler.mlcompute import mlcompute
mlcompute.set_mlc_device(device_name='gpu')
WARNING:tensorflow: Eager mode uses the CPU. Switching to the CPU.
Disabling the eager mode may work for the program to utilize the GPU, but it's not a general behavior and can lead to such puzzling performance on both CPU/GPU. For now, the most appropriate approach can be to choose device_name='any', by that the ML Compute will query the available devices on the system and selects the best device(s) for training the network.

Try with turning off the eager execution...
via following
import tensorflow as tf
tf.compat.v1.disable_eager_execution()
Let me know if it works.

Is there an easy to switch out of GPU in python

After attempting to utilize my GPU to run a machine learning model, I keep on getting the error "cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version." At this point, I would just like to use my CPU, however, I'm not sure how since every time I try to train my algorithm, it gives the error even though I never imported the code
import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
I only ran this code once and now my model can't train Is there an easy way to get rid of this and just go back to train it just with CPU?

This should set it so your tensorflow won't see a cuda GPU when it looks and will default to the CPU.
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

It is recommended that you set the CUDA_VISIBLE_DEVICES environment variable to expose different GPUs before starting the processes. Otherwise, TensorFlow will attempt to allocate almost the entire memory on all of the available GPUs, which prevents other processes from using those GPUs (even if the current process isn't using them).
If you set the environment variable CUDA_VISIBLE_DEVICES=-1 you will use the CPU only. If you don't set that environment variable you will allocate memory to all GPUs but by default only use GPU 0. You can also set it to the specific GPU you want to use. CUDA_VISIBLE_DEVICES=0 will only use GPU 0.
This environment variable is created by the user, it won't exist until you create it. You need to set the variable before tensorflow is imported (usually that is before you start your script).

Using a GPU with Tensorflow Eager Execution on Google Colaboratory

I'm trying to offload computing to a GPU in Tensorflow eager execution on Google Colaboratory, but due to the way Colab handles GPUs I'm having trouble.
Normally, when using a GPU you change the runtime to a GPU accelerated one and Tensorflow automatically uses the available GPU for your calculations. If you were to try and set the GPU manually, you'd find you can't because there isn't one in the list of connected devices.
Eager execution, however, doesn't automatically use a GPU if one is available, and because you can't set one manually it doesn't seem like one can be used.
Please see the attached notebook: https://drive.google.com/file/d/1NeJQoO5Yi5V-m7Hmm85wmm1Cl5SrY33m/view?usp=sharing
Trying to specify a GPU to use throws the following error: RuntimeError: Error copying tensor to device: /job:localhost/replica:0/task:0/device:GPU:0. /job:localhost/replica:0/task:0/device:GPU:0 unknown device.

Eager execution does in fact let you specify GPUs manually (for example, search for "GPU" in https://www.tensorflow.org/programmers_guide/eager).
The particular error message you've included indicates that TensorFlow can't find a GPU, so make sure that you've installed the GPU-compatible version of TensorFlow 1.7. After you've done that, you'll need to make one more change to your code. In particular, copy the tensors x and y to GPU before multiplying them, like so:
with tf.device('/gpu:0'):
x = x.gpu()
y = y.gpu()
%timeit x * y

How to change the processing unit during run time (from GPU to CPU)?

In the context of deep neural networks training, the training works faster when it uses the GPU as the processing unit.
This is done by configuring CudNN optimizations and changing the processing unit in the environment variables with the following line (Python 2.7 and Keras on Windows):
os.environ["THEANO_FLAGS"] = "floatX=float32,device=gpu,optimizer_including=cudnn,gpuarray.preallocate=0.8,dnn.conv.algo_bwd_filter=deterministic,dnn.conv.algo_bwd_data=deterministic,dnn.include_path=e:/toolkits.win/cuda-8.0.61/include,dnn.library_path=e:/toolkits.win/cuda-8.0.61/lib/x64"
The output is then:
Using gpu device 0: TITAN Xp (CNMeM is disabled, cuDNN 5110)
The problem is that the GPU memory is limited compared to the RAM (12GB and 128GB respectively), and the training is only one phase of the whole flow. Therefore I want to change back to CPU once the training is completed.
I've tried the following line, but it has no effect:
os.environ["THEANO_FLAGS"] = "floatX=float32,device=cpu"
My questions are:
Is it possible to change from GPU to CPU and vice-versa during runtime? (technically)
If yes, how can I do it programmatically in Python? (2.7, Windows, and Keras with Theano backend).

Yes this is possible at least for the tensorflow backend. You just have to also import tensorflow and put your code into the following with:
with tf.device('/cpu:0'):
your code
with tf.device('/gpu:0'):
your code
I am unsure if this also works for theano backend. However, switching from one backend to the other one is just setting a flag beforehand so this should not provide too much trouble.

tensorflow: works well in one-GPU computer but Out of Memory in two-GPU computer

Before I installed the second graphics card, I had successfully trained a model which is VGG-11 architecture using tensorflow and a single GPU that has 6GB memory. But I got OOM error when I installed the second graphics card and ran the same code(allow_growth=True and no tf.device() used).
My understanding is that my second card(GPU:0) has 8GB memory and TF would use my "device:gpu:0" as default to do computation when I did not use tf.device() to specify any device. And the memory should be enough because 8GB > 6GB.
Then I tried to use CUDA_VISIBLE_DEVICES=0 to block one and ran the same code. TF worked successfully.
What is the problem?