We Keep Coding

How to implement LSTM in tensorflow v1 from pandas dataframe

I've tried following tutorials on implementing this but I keep getting dimension errors on the LSTM layer.
ValueError: Input 0 of layer LSTM is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 2]
import random
import numpy as np
import tensorflow as tf
from tensorflow import feature_column as fc
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, DenseFeatures, Reshape
from sklearn.model_selection import train_test_split
def df_to_dataset(features, target, batch_size=32):
return tf.data.Dataset.from_tensor_slices((dict(features), target)).batch(batch_size)
# Reset randomization seeds
np.random.seed(0)
tf.random.set_random_seed(0)
random.seed(0)
# Assume 'frame' to be a dataframe with 3 columns: 'optimal_long_log_return', 'optimal_short_log_return' (independent variables) and 'equilibrium_log_return' (dependent variable)
X = frame[['optimal_long_log_return', 'optimal_short_log_return']][:-1]
Y = frame['equilibrium_log_return'].shift(-1)[:-1]
X_train, _X, y_train, _y = train_test_split(X, Y, test_size=0.5, shuffle=False, random_state=1)
X_validation, X_test, y_validation, y_test = train_test_split(_X, _y, test_size=0.5, shuffle=False, random_state=1)
train = df_to_dataset(X_train, y_train)
validation = df_to_dataset(X_validation, y_validation)
test = df_to_dataset(X_test, y_test)
feature_columns = [fc.numeric_column('optimal_long_log_return'), fc.numeric_column('optimal_short_log_return')]
model = Sequential()
model.add(DenseFeatures(feature_columns, name='Metadata'))
model.add(LSTM(256, name='LSTM'))
model.add(Dense(1, name='Output'))
model.compile(loss='logcosh', metrics=['mean_absolute_percentage_error'], optimizer='Adam')
model.fit(train, epochs=10, validation_data=validation, verbose=1)
loss, accuracy = model.evaluate(test, verbose=0)
print(f'Target Error: {accuracy}%')
After seeing this issue elsewhere I've tried setting input_shape=(None, *X_train.shape), input_shape=X_train.shape, neither works. I also tried inserting a Reshape layer model.add(Reshape(X_train.shape)) before the LSTM layer and it fixed the issue but I got another issue in its place:
InvalidArgumentError: Input to reshape is a tensor with 64 values, but the requested shape has 8000
...and I'm not even sure adding the Reshape layer is doing what I think it is doing. After all, why would reshaping the data to its own shape fix things? Something is happening with my data that I just don't understand.
Also, I'm using this for time series analysis (stock returns), so I would think that the LSTM model should be stateful and temporal. Would I need to move the timestamp index into its own column in the pandas database before converting to a tensor?
Unfortunately I'm obligated to use tensorflow v1.15 as this is being developed on the QuantConnect platform and they presumably won't be updating the library any time soon.
EDIT: I've made a bit of progress by using TimeseriesGenerator, but now I'm getting the following error (which returns no results on Google):
KeyError: 'No key found for either mapped or original key. Mapped Key: []; Original Key: []'
Code below (I'm sure I'm using the input_shape arguments incorrectly):
train = TimeseriesGenerator(X_train, y_train, 1, batch_size=batch_size)
validation = TimeseriesGenerator(X_validation, y_validation, 1, batch_size=batch_size)
test = TimeseriesGenerator(X_test, y_test, 1, batch_size=batch_size)
model = Sequential(name='Expected Equilibrium Log Return')
model.add(LSTM(256, name='LSTM', stateful=True, batch_input_shape=(1, batch_size, X_train.shape[1]), input_shape=(1, X_train.shape[1])))
model.add(Dense(1, name='Output'))
model.compile(loss='logcosh', metrics=['mean_absolute_percentage_error'], optimizer='Adam', sample_weight_mode='temporal')
print(model.summary())
model.fit_generator(train, epochs=10, validation_data=validation, verbose=1)
loss, accuracy = model.evaluate_generator(test, verbose=0)
print(f'Model Accuracy: {accuracy}')

Turns out this specific issue relates to a patch that Quantconnect made to pandas dataframes which interfered with the older version of tensorflow/keras.

Error when checking target: expected dense_3 to have shape (4,) but got array with shape (10,)

I am Beginner in Python
I using Keras and Tensorflow to Create ML Model . When I have 4 classes it work well but add more Classes it got this error. Add have added 10 classes.
I use python 3.7 , Keras 2.3.1, Tensorflow 1.14.
I'm following tutorial in this link
here.
I have lost 3 days to solve this problem but I could not
Thanks
Classification.py File
import os as os
from keras.models import load_model
from keras.utils import np_utils
import model as md
import preparation as prep
import visualization as vis
from keras.utils import to_categorical
from keras import metrics
# ----------- dataset settings -----------
# number of instances per class used for train and test in total:
# should be smaller or equal than generated subset
INSTANCES_PER_CLASS = 5000
NUM_CLASS_LIMIT = 345 # limit of classes
# path of the dataset seperated in train and test
DATA_PATH = os.path.join(os.getcwd(), "../Draw/dataset/train_test_20k/")
# path for all created files
MODEL_PATH = os.path.join(os.getcwd(), "models/" + str(NUM_CLASS_LIMIT) + "/" + str(INSTANCES_PER_CLASS) + "/")
# ----------- model settings -----------
MODEL_NAME = 'model.h5' # name for the freezed model
# input size
IMG_WIDTH = 28
IMG_HEIGHT = 28
IMG_SIZE = IMG_WIDTH * IMG_HEIGHT
IMG_DIM = 1
# training settings
EPOCHS = 10
BATCH_SIZE = 256
if __name__ == "__main__":
# create new directories if required
if not os.path.isdir(MODEL_PATH):
os.makedirs(MODEL_PATH)
# get the dataset
num_classes, x_train, x_test, y_train, y_test, classes_dict = prep.collect_data(NUM_CLASS_LIMIT)
print("trainingsset instances {}".format(x_train.shape))
print("trainingsset labels {}".format(y_train.shape))
# plot first test images
#vis.plot_first_n_images(x_test, y_test, classes_dict, 100)
# class representation as "binary" vector
y_train = np_utils.to_categorical(y_train, num_classes=num_classes)
y_test = np_utils.to_categorical(y_test, num_classes=num_classes)
# create or load keras model
if not os.path.isfile(MODEL_PATH + MODEL_NAME):
print("create model...")
model = md.build_model(input_shape=x_train.shape[1:], num_classes=num_classes)
else:
print("load existing model...")
model = load_model(MODEL_PATH + MODEL_NAME)
# score trained model using validation set
scores = model.evaluate(x_test, y_test, verbose=1)
print('test loss:', scores[0])
print('test accuracy:', scores[1])
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['acc'])
# print model information if desired
print(model.summary())
# model training from scratch or retrain by existing model
hist = model.fit(x_train, y_train, batch_size=BATCH_SIZE,
epochs=EPOCHS,
validation_data=[x_test, y_test],
shuffle=True)
#from keras.utils import plot_model
#plot_model(model, to_file=MODEL_PATH + 'model.png')
# evaluation process
print("evaluate model...")
# summarize history during training phase
# plot training and validation set accuracy
vis.plot_training_history_accuracy(hist)
# test set evaluation
scores = model.evaluate(x_test, y_test, verbose=1)
print(scores)
print('test loss:', scores[0])
print('test accuracy:', scores[1])
# create and plot confusion matrix
#y_pred = model.predict(x_test)
#vis.plot_confusion_matrix(y_pred, y_test, classes=list(classes_dict.values()))
# freeze the model (architecture and weights)
model.save(os.path.join(MODEL_PATH, MODEL_NAME))
print('saved trained model at {}'.format(os.path.join(MODEL_PATH, MODEL_NAME)))
The Model Code model.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from keras.models import Model
from keras.layers import Input, Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
def build_model(input_shape, num_classes):
"""
Builds the model architecture based on MNIST CNN
https://www.tensorflow.org/tutorials/estimators/cnn
Args:
input_spape: Input shape of the model
num_classes: Number of classes
Returns:
keras.models.Model: The created model
"""
inputs = Input(shape=input_shape)
x = Conv2D(32, (5,5), activation='relu')(inputs)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Conv2D(128, (3, 3), activation='relu')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Dropout(0.2)(x)
x = Flatten()(x)
x = Dense(512, activation='relu')(x)
x = Dense(256, activation='relu')(x)
predictions = Dense(num_classes, activation='softmax')(x)
print("SHOW DESEN", num_classes)
return Model(inputs=inputs, outputs=predictions)

I believe the error is coming from this step:
hist = model.fit(x_train, y_train, batch_size=BATCH_SIZE,
epochs=EPOCHS,
validation_data=[x_test, y_test],
shuffle=True)
The reference to "target" means the "y_train" array. You might want to confirm that the y_train that you are trying to fit matches the actual model.summary(), by checking the y_train.shape just before the fit. It's possibly that the old model that was designed for 4 classes is being used, even though the new 10 class y_train is being passed to the model.fit().
Diagnostic code example (model.fit is commented out):
# This outputs the model design. The final layer should be (None,num_classes)
print(model.summary())
# print shape of y_train/target data. The second dimension should match the number of classes from above (?, num_classes)
print(y_train.shape)
#model.fit( x_train, y_train, epochs=1)
I hope this helps.

How to train a CNN model on 2 classes of 100 samples each and then test it on 200 new samples?

I've got 2 classes for my training set: Birds(100 samples) and no_birds(100) samples. And, the test set is unlabelled consisting of 200 test samples (mixed with birds and no_birds). For every sample in the test set I intend to classify it as bird or no_bird using CNN with Keras.
import numpy as np
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Activation
from keras.layers.core import Dense, Flatten
from keras.optimizers import Adam
from keras.metrics import categorical_crossentropy
from keras.preprocessing.image import ImageDataGenerator
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import *
from sklearn.metrics import confusion_matrix
import itertools
import matplotlib.pyplot as plt
train_path = 'dataset/train_set'
test_path = 'dataset/test_set'
train_batches = ImageDataGenerator().flow_from_directory(train_path, target_size=(224,224), classes=['bird', 'no_bird'], batch_size=10) # bird directory consisting of 100
test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=['unknown'], batch_size=10)
model = Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224,224,3)),
Flatten(),
Dense(2, activation='softmax'),
])
model.compile(Adam(lr=.0001), loss='categorical_crossentropy', metrics=['accuracy'])
model.fit_generator(train_batches, steps_per_epoch=20, validation_data=test_batches, validation_steps=20, epochs=10, verbose=2)
Error I'm getting at the last step is this:
ValueError: Error when checking target: expected dense_1 to have shape (2,) but got array with shape (1,)
Now, I know it could be probably because of test_set having only 1 directory, since it's unlabelled. Correct me if I'm wrong. What should I do to make this work?

It seems your test set is unlabelled. Remove validation arguments from model.fit. It should be:
model.fit_generator(train_batches, steps_per_epoch=20, epochs=10, verbose=2)
You can't validate without labels.

the line test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=['unknown'], batch_size=10) is wrong
you should do test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=['bird', 'no_bird'], batch_size=10) still. That way you can score your predictions
folowup information:
when you look at https://keras.io/models/sequential/, it says
validation_data: tuple (x_val, y_val) or tuple (x_val, y_val, val_sample_weights) on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. This will override validation_split.
Your test data must be the same shape as your train data. You'll have to organize the test data directory so it's structured the same as the training data

Why my training speed in Keras with multi_gpu_model is worse than single gpu?

My Keras version is 2.0.9, and using tensorflow backend.
I tried to implement multi_gpu_model in keras. However, training with 4 gpus was even worse than 1 gpu in practice. I got 25sec for 1 gpu, and 50sec for 4 gpus. Could you give me the reason why this happens?
/blog for multi_gpu_model
https://www.pyimagesearch.com/2017/10/30/how-to-multi-gpu-training-with-keras-python-and-deep-learning/
I used this commend for 1 gpu
CUDA_VISIBLE_DEVICES=0 python gpu_test.py
and for 4 gpus,
python gpu_test.py
-Here is source code for training.
from keras.datasets import mnist
from keras.layers import Input, Dense, merge
from keras.layers.core import Lambda
from keras.models import Model
from keras.utils import to_categorical
from keras.utils.training_utils import multi_gpu_model
import time
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
inputs = Input(shape=(784,))
x = Dense(4096, activation='relu')(inputs)
x = Dense(2048, activation='relu')(x)
x = Dense(512, activation='relu')(x)
x = Dense(64, activation='relu')(x)
predictions = Dense(10, activation='softmax')(x)
model = Model(inputs=inputs, outputs=predictions)
'''
m_model = multi_gpu_model(model, 4)
m_model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
m_model.summary()
a=time.time()
m_model.fit(x_train, y_train, batch_size=128, epochs=5)
print time.time() - a
a=time.time()
m_model.predict(x=x_test, batch_size=128)
print time.time() - a
'''
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
a=time.time()
model.fit(x_train, y_train, batch_size=128, epochs=5)
print time.time() - a
a=time.time()
model.predict(x=x_test, batch_size=128)
print time.time() - a
And this is gpu state with running 4 gpus.

I can give you what I think is the answer, but I don't have it fully working myself. I was tipped off onto this by a bug report, but in the source code for multi_gpu_model it says:
# Instantiate the base model (or "template" model).
# We recommend doing this with under a CPU device scope,
# so that the model's weights are hosted on CPU memory.
# Otherwise they may end up hosted on a GPU, which would
# complicate weight sharing.
with tf.device('/cpu:0'):
model = Xception(weights=None,
input_shape=(height, width, 3),
classes=num_classes)
I think this is the problem. I'm still working on making it work myself, though.

Keras - Save image embedding of the mnist data set

I've written the following simple MLP network for the MNIST db.
from __future__ import print_function
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import callbacks
batch_size = 100
num_classes = 10
epochs = 20
tb = callbacks.TensorBoard(log_dir='/Users/shlomi.shwartz/tensorflow/notebooks/logs/minist', histogram_freq=10, batch_size=32,
write_graph=True, write_grads=True, write_images=True,
embeddings_freq=10, embeddings_layer_names=None,
embeddings_metadata=None)
early_stop = callbacks.EarlyStopping(monitor='val_loss', min_delta=0,
patience=3, verbose=1, mode='auto')
# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model = Sequential()
model.add(Dense(200, activation='relu', input_shape=(784,)))
model.add(Dropout(0.2))
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(60, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(30, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
history = model.fit(x_train, y_train,
callbacks=[tb,early_stop],
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
The model ran fine, and I could see the scalars info on TensorBoard. However when I've changed embeddings_freq=10 to try and visualize the images (Like seen here) I got the following error:
Traceback (most recent call last):
File "/Users/shlomi.shwartz/IdeaProjects/TF/src/minist.py", line 65, in <module>
validation_data=(x_test, y_test))
File "/Users/shlomi.shwartz/tensorflow/lib/python3.6/site-packages/keras/models.py", line 870, in fit
initial_epoch=initial_epoch)
File "/Users/shlomi.shwartz/tensorflow/lib/python3.6/site-packages/keras/engine/training.py", line 1507, in fit
initial_epoch=initial_epoch)
File "/Users/shlomi.shwartz/tensorflow/lib/python3.6/site-packages/keras/engine/training.py", line 1117, in _fit_loop
callbacks.set_model(callback_model)
File "/Users/shlomi.shwartz/tensorflow/lib/python3.6/site-packages/keras/callbacks.py", line 52, in set_model
callback.set_model(model)
File "/Users/shlomi.shwartz/tensorflow/lib/python3.6/site-packages/keras/callbacks.py", line 719, in set_model
self.saver = tf.train.Saver(list(embeddings.values()))
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1139, in __init__
self.build()
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1161, in build
raise ValueError("No variables to save")
ValueError: No variables to save
Q: What am I missing? is that the right way of doing it in Keras?
Update: I understand there is some prerequisite in order to use embedding projection, however I haven't found a good tutorial for doing so in Keras, any help would be appreciated.

What is called "embedding" here in callbacks.TensorBoard is, in a broad sense, any layer weight. According to Keras documentation:
embeddings_layer_names: a list of names of layers to keep eye on. If None or empty list all the embedding layer will be watched.
So by default, it's going to monitor the Embedding layers, but you don't really need a Embedding layer to use this visualization tool.
In your provided MLP example, what's missing is the embeddings_layer_names argument. You have to figure out which layers you're going to visualize. Suppose you want to visualize the weights (or, kernel in Keras) of all Dense layers, you can specify embeddings_layer_names like this:
model = Sequential()
model.add(Dense(200, activation='relu', input_shape=(784,)))
model.add(Dropout(0.2))
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(60, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(30, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
embedding_layer_names = set(layer.name
for layer in model.layers
if layer.name.startswith('dense_'))
tb = callbacks.TensorBoard(log_dir='temp', histogram_freq=10, batch_size=32,
write_graph=True, write_grads=True, write_images=True,
embeddings_freq=10, embeddings_metadata=None,
embeddings_layer_names=embedding_layer_names)
model.compile(...)
model.fit(...)
Then, you can see something like this in TensorBoard:
You can see the relevant lines in Keras source if you want to figure out what's happening regarding embeddings_layer_names.
Edit:
So here's a dirty solution for visualizing layer outputs. Since the original TensorBoard callback does not support this, implementing a new callback seems inevitable.
Since it will take up a lot of page space to re-write the entire TensorBoard callback here, I'll just extend the original TensorBoard, and write out the parts that are different (which is already quite lengthy). But to avoid duplicated computations and model saving, re-writing the TensorBoard callback will be a better and cleaner way.
import tensorflow as tf
from tensorflow.contrib.tensorboard.plugins import projector
from keras import backend as K
from keras.models import Model
from keras.callbacks import TensorBoard
class TensorResponseBoard(TensorBoard):
def __init__(self, val_size, img_path, img_size, **kwargs):
super(TensorResponseBoard, self).__init__(**kwargs)
self.val_size = val_size
self.img_path = img_path
self.img_size = img_size
def set_model(self, model):
super(TensorResponseBoard, self).set_model(model)
if self.embeddings_freq and self.embeddings_layer_names:
embeddings = {}
for layer_name in self.embeddings_layer_names:
# initialize tensors which will later be used in `on_epoch_end()` to
# store the response values by feeding the val data through the model
layer = self.model.get_layer(layer_name)
output_dim = layer.output.shape[-1]
response_tensor = tf.Variable(tf.zeros([self.val_size, output_dim]),
name=layer_name + '_response')
embeddings[layer_name] = response_tensor
self.embeddings = embeddings
self.saver = tf.train.Saver(list(self.embeddings.values()))
response_outputs = [self.model.get_layer(layer_name).output
for layer_name in self.embeddings_layer_names]
self.response_model = Model(self.model.inputs, response_outputs)
config = projector.ProjectorConfig()
embeddings_metadata = {layer_name: self.embeddings_metadata
for layer_name in embeddings.keys()}
for layer_name, response_tensor in self.embeddings.items():
embedding = config.embeddings.add()
embedding.tensor_name = response_tensor.name
# for coloring points by labels
embedding.metadata_path = embeddings_metadata[layer_name]
# for attaching images to the points
embedding.sprite.image_path = self.img_path
embedding.sprite.single_image_dim.extend(self.img_size)
projector.visualize_embeddings(self.writer, config)
def on_epoch_end(self, epoch, logs=None):
super(TensorResponseBoard, self).on_epoch_end(epoch, logs)
if self.embeddings_freq and self.embeddings_ckpt_path:
if epoch % self.embeddings_freq == 0:
# feeding the validation data through the model
val_data = self.validation_data[0]
response_values = self.response_model.predict(val_data)
if len(self.embeddings_layer_names) == 1:
response_values = [response_values]
# record the response at each layers we're monitoring
response_tensors = []
for layer_name in self.embeddings_layer_names:
response_tensors.append(self.embeddings[layer_name])
K.batch_set_value(list(zip(response_tensors, response_values)))
# finally, save all tensors holding the layer responses
self.saver.save(self.sess, self.embeddings_ckpt_path, epoch)
To use it:
tb = TensorResponseBoard(log_dir=log_dir, histogram_freq=10, batch_size=10,
write_graph=True, write_grads=True, write_images=True,
embeddings_freq=10,
embeddings_layer_names=['dense_1'],
embeddings_metadata='metadata.tsv',
val_size=len(x_test), img_path='images.jpg', img_size=[28, 28])
Before launching TensorBoard, you'll need to save the labels and images to log_dir for visualization:
from PIL import Image
img_array = x_test.reshape(100, 100, 28, 28)
img_array_flat = np.concatenate([np.concatenate([x for x in row], axis=1) for row in img_array])
img = Image.fromarray(np.uint8(255 * (1. - img_array_flat)))
img.save(os.path.join(log_dir, 'images.jpg'))
np.savetxt(os.path.join(log_dir, 'metadata.tsv'), np.where(y_test)[1], fmt='%d')
Here's the result:

You need at least one Embedding Layer in Keras. On stats was a good explanation about them. It is not directly for Keras, but the concepts are roughly the same. What is an embedding layer in a neural network

So, I conclude that what you actually want (it's not completely clear from your post) is to visualize the predictions of your model, in a manner similar to this Tensorboard demo.
To start with, reproducing this stuff is non-trivial even in Tensorflow, let alone Keras. The said demo makes very brief and passing references to things like metadata & sprite images that are necessary in order to obtain such visualizations.
Bottom line: although non-trivial, it is indeed possible to do it with Keras. You don't need the Keras callbacks; all you need is your model predictions, the necessary metadata & sprite image, and some pure TensorFlow code. So,
Step 1 - get your model predictions for the test set:
emb = model.predict(x_test) # 'emb' for embedding
Step 2a - build a metadata file with the real labels of the test set:
import numpy as np
LOG_DIR = '/home/herc/SO/tmp' # FULL PATH HERE!!!
metadata_file = os.path.join(LOG_DIR, 'metadata.tsv')
with open(metadata_file, 'w') as f:
for i in range(len(y_test)):
c = np.nonzero(y_test[i])[0][0]
f.write('{}\n'.format(c))
Step 2b - get the sprite image mnist_10k_sprite.png as provided by the TensorFlow guys here, and place it in your LOG_DIR
Step 3 - write some Tensorflow code:
import tensorflow as tf
from tensorflow.contrib.tensorboard.plugins import projector
embedding_var = tf.Variable(emb, name='final_layer_embedding')
sess = tf.Session()
sess.run(embedding_var.initializer)
summary_writer = tf.summary.FileWriter(LOG_DIR)
config = projector.ProjectorConfig()
embedding = config.embeddings.add()
embedding.tensor_name = embedding_var.name
# Specify the metadata file:
embedding.metadata_path = os.path.join(LOG_DIR, 'metadata.tsv')
# Specify the sprite image:
embedding.sprite.image_path = os.path.join(LOG_DIR, 'mnist_10k_sprite.png')
embedding.sprite.single_image_dim.extend([28, 28]) # image size = 28x28
projector.visualize_embeddings(summary_writer, config)
saver = tf.train.Saver([embedding_var])
saver.save(sess, os.path.join(LOG_DIR, 'model2.ckpt'), 1)
Then, running Tensorboard in your LOG_DIR, and selecting color by label, here is what you get:
Modifying this in order to get predictions for other layers is straightforward, although in this case the Keras Functional API may be a better choice.