I trained my model and saved it:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array
new_model=tf.keras.models.load_model('the_model.h5')
new_model.summary()
img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)
img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)
#prints out (1,227,227,3) the expected shapes
so the architecture of my model is the following one, i'm using pre-trained resnet50
backbone = ResNet50(input_shape=(227,227,3),weights='imagenet', include_top=False)
model = Sequential()
model.add(backbone)
model.add(GlobalAveragePooling2D())
model.add(Dropout(0.5))
model.add(Dense(64,activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1,activation='sigmoid'))
i tried visualize outputs of hidden layers, however with keras or keract i can't get the outputs
with keras :
layer_outputs=[]
for layer in new_model.layers:
if layer.name=='resnet50':
temp = [l.output for l in layer.layers]
layer_outputs=temp
else:
layer_outputs.append(layer.output)
activation_model = Model(inputs=new_model.input, outputs=layer_outputs)
the error caused by the last line :
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []
i feel like my model inputs are matching with layer_outputs so i don't really understand the error, indeed when i'm checking :
print(new_model.layers[0].input)
#prints out :Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)
print(layer_outputs[0])
#prints out : Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)
when using keract :
a = keract.get_activations(new_model, img) # with just one sample.
keract.display_activations(a, directory='f', save=True)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,227,227,3]
Any idea of how i can fix it, or another viable solution to get outputs from hidden layers using pre-trained model?
Thanks,
Okey, i found a convenient solution to my problem.
Indeed i think this problem occurs because my sequential model is itself made up of another model (resnet).
Since i didn't add many layers on top of the pre-trained resnet model, i just decided to visualize the feature maps from the resnet model
img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)
img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)
loaded=tf.keras.models.load_model('age_gender_train.h5')
layer_outputs=[ layer.output for layer in loaded.layers[0].layers]
res = loaded.layers[0]
activation_model = Model(inputs=res.input, outputs=layer_outputs)
activations=activation_model.predict(img)
img = np.squeeze(img,axis=0)
Then you can easily display features maps using activations variable.
Note that since you have the outputs of the resnet model, it might be possible to get the feature maps from the layers on top by repeating the process. Using the output of resnet as input and removing the resnet model from the layer_outputs.(i did not try this, could not work)
Hope it could help someone
Related
I am building an LSTM-based Deep Q-learning Network with Python, Keras, and Tensorflow and have run into the following problem. After I have created the network with a given batch_input_shape and try to fit the network to data of that shape, I receive the following error:
WARNING:tensorflow:Model was constructed with shape (64, 1, 10) for
input Tensor("lstm_34_input:0", shape=(64, 1, 10), dtype=float32), but
it was called on an input with incompatible shape (32, 1, 10).
I have created the following toy example to simply demonstrate the code which causes the problem.
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.python.keras.layers import Flatten
from tensorflow.keras.layers import Dense, Dropout, LSTM
from tensorflow.keras.optimizers import Adam
# Hyperparameters
action_space = range(0, 10)
input_length = 10
batch_size = 64
timesteps = 1
learning_rate = 0.0001
# Create random input variables
state = np.random.randint(0, 100, size=(batch_size, timesteps, input_length))
target = np.random.randint(0, 100, size=(batch_size, len(action_space)))
# Build the model
model = Sequential()
model.add(LSTM(10, batch_input_shape=(batch_size, timesteps, input_length), return_sequences=True, activation="tanh",
recurrent_dropout=0, stateful=False))
model.add(LSTM(10, activation="tanh", return_sequences=True, recurrent_dropout=0, stateful=False))
model.add(Flatten())
model.add(Dense(len(action_space), activation="relu"))
model.compile(loss="mean_squared_error", optimizer=Adam(lr=learning_rate))
# Fit the model
model.fit(state, target, epochs=1, verbose=1)
This creates the error seen above.
My understanding is that the input layer should expect to receive a batch of shape (64, 1, 10) and we pass it this shape. However, the input layer appears to receive a shape of (32, 1, 10). We can verify that state.shape is (64, 1, 10) as expected so at some stage there is a reshaping of this input, or perhaps the error refers to an input to the hidden or output layer?
Any help would be greatly appreciated.
Update
I am using Tensorflow GPU version 2.3.0
and Keras version 2.3.1
For anyone trying to find a way to implement batch_input_shape in order to use a stateful LSTM, the answer comes from Simon's comment above:
add batch_size=batch_size to model.fit
You can also add it to model.predict() for a DQN.
I load a pretrained model. However, I want to add Spectral Normalization to this model.
I can do it in this way if the model is fully sequential:
import tensorflow as tf
base_model = tf.keras.applications.VGG16(input_shape=(720, 1280, 3), weights=None, include_top=False)
# Add spectral normalization
base_model_new = tf.keras.models.Sequential()
for layer in base_model.layers:
if isinstance(layer, tf.keras.layers.Conv2D):
spec_norm_layer = SpectralNormalization(layer)
base_model_new.add(spec_norm_layer)
else:
base_model_new.add(layer)
input = tf.keras.Input(shape=(720, 1280, 3), name='image')
x = base_model_new(input)
model = tf.keras.models.Model(inputs=[input], outputs=x, name='test')
However, this method does not work when for example using a ResNet where there are Add or Concatenate layers which expect a list of inputs. How to solve this for ResNets?
I have the following code which works on pre-trained VGG model but fails on ResNet and Inception model.
vgg_model = keras.applications.vgg16.VGG16(weights='imagenet')
type(vgg_model)
vgg_model.summary()
model = Sequential()
for layer in vgg_model.layers:
model.add(layer)
Now, changing the model to ResNet as follows:
resnet_model=keras.applications.resnet50.ResNet50(weights='imagenet')
type(resnet_model)
resnet_model.summary()
model = Sequential()
for layer in resnet_model.layers:
model.add(layer)
gives the following error:
ValueError: Input 0 is incompatible with layer res2a_branch1: expected axis -1 of input shape to have value 64 but got shape (None, 56, 56, 256)
The problem is due to the fact that unlike VGG, Resnet does not have a sequential architecture (e.g. some layers are connected to more than one layers, there are skip connections, etc.). Therefore you cannot iterate over the layers in the model one after another and connect each layer to the previous one (i.e. sequentially). You can plot the architecture of the model using plot_model() to have a better understanding of this point.
I have taken a standard ResNet50 model:
model = keras.applications.resnet50.ResNet50(include_top=False,
weights='imagenet',
classes=10,
input_shape=(224, 224, 3))
And added several dense layers of my own:
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(2, activation='softmax'))
model = Model(input=model.input, output=top_model(model.output))
This way it works great, however, when I want to delete last Dense and Dropout layers with model.pop() keras wont work well:
model.layers[-1].layers
[<keras.layers.core.Flatten at 0x16b5c00b8>,
<keras.layers.core.Dense at 0x16b5c0320>,
<keras.layers.core.Dropout at 0x16b5c02e8>,
<keras.layers.core.Dense at 0x16b5c0d68>]
model.layers[-1].pop()
model.layers[-1].pop()
model.layers[-1].layers
[<keras.layers.core.Flatten at 0x1ae6e5940>,
<keras.layers.core.Dense at 0x1ae6e9e10>]
model.layers[-1].outputs = [model.layers[-1].layers[-1].output]
model.outputs = model.layers[-1].outputs
model.layers[-1].layers[-1].outbound_nodes = []
Then I just compile the model and when trying to predict, I get an error:
You must feed a value for placeholder tensor 'flatten_7_input_12' with dtype float and shape [?,1,1,2048]
model.pop() takes care of all the underlying settings, including setting the model.output to the output of the new last layer. Therefore, you don't need to handle anything regarding the outputs.
Please also note that you are assigning to the model variable; thus, model.outputs is already referring to the extended model's output.
Here is a sample code that works fine on keras 2.0.6 with TensorFlow backend (1.4.0):
import keras
from keras.models import Sequential, Model
from keras.layers import *
import numpy as np
model = keras.applications.resnet50.ResNet50(include_top=False,
weights='imagenet',
classes=10,
input_shape=(224, 224, 3))
top_model = Sequential()
top_model.add(keras.layers.Flatten(input_shape=model.output_shape[1:]))
top_model.add(keras.layers.Dense(256, activation='relu'))
top_model.add(keras.layers.Dropout(0.5))
top_model.add(keras.layers.Dense(2, activation='softmax'))
model_extended = Model(input=model.input, output=top_model(model.output))
model_extended.layers[-1].pop()
model_extended.layers[-1].pop()
model_extended.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model_extended.predict(np.zeros((1, 224, 224, 3)))
I want to use a pretrained imagenet VGG16 model in keras and add my own small convnet on top. I am only interested in the features, not the predictions
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.applications.vgg16 import VGG16
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input
import numpy as np
import os
from keras.models import Model
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
load images from directory (the dir contains 4 images)
IF = '/home/ubu/files/png/'
files = os.listdir(IF)
imgs = [img_to_array(load_img(IF + p, target_size=[224,224])) for p in files]
im = np.array(imgs)
load the base model, preprocess input and get the features
base_model = VGG16(weights='imagenet', include_top=False)
x = preprocess_input(aa)
features = base_model.predict(x)
this works, and I get the features for my images on the pretrained VGG.
I now want to finetune the model and add some convolutional layers.
I read https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html and https://keras.io/applications/ but cannot quite bring them together.
adding my model on top:
x = base_model.output
x = Convolution2D(32, 3, 3)(x)
x = Activation('relu')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Convolution2D(32, 3, 3)(x)
x = Activation('relu')(x)
feat = MaxPooling2D(pool_size=(2, 2))(x)
building the complete model
model_complete = Model(input=base_model.input, output=feat)
stop base layers from being learned
for layer in base_model.layers:
layer.trainable = False
new model
model_complete.compile(optimizer='rmsprop',
loss='binary_crossentropy')
now fit the new model, the model is 4 images and [1,0,1,0] are the class labels.
But this is obviously wrong:
model_complete.fit_generator((x, [1,0,1,0]), samples_per_epoch=100, nb_epoch=2)
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
How is this done?
How would I do it if I only wanted to replace the last convolutional block (conv block5 in VGG16) instead of adding something?
How would I only train the bottleneck features?
The features output features has shape (4, 512, 7, 7). There are four images, but what is in the other dimensions? How would I reduce that to a (1,x) array?
Fitting model
The problem with your generator code is that the fit_generator method expects a generator function to generate the data for fitting which you don't provide.
You can either define a generator as done in the tutorial that you have linked to or create the data and labels yourself and fit your model yourself:
model_complete.fit(images, labels, batch_size=100, nb_epoch=2)
where images are your generated training images and labels are the corresponding labels.
Removing last layer
Assuming you have a model variable and the "pop" method described below, you can do model = pop(model) to remove the last layer.
Training only specific layers
As you have done in your code, you can do:
for layer in base_model.layers:
layer.trainable = False
Then you can "unfreeze" and layer that you want by changing their trainable property to True.
Changing dimensions
To change the output to a 1D array you can use the Flatten layer
The pop method
def pop(model):
'''Removes a layer instance on top of the layer stack.
This code is thanks to #joelthchao https://github.com/fchollet/keras/issues/2371#issuecomment-211734276
'''
if not model.outputs:
raise Exception('Sequential model cannot be popped: model is empty.')
else:
model.layers.pop()
if not model.layers:
model.outputs = []
model.inbound_nodes = []
model.outbound_nodes = []
else:
model.layers[-1].outbound_nodes = []
model.outputs = [model.layers[-1].output]
model.built = False
return model
Use model.fit(X, y) to train on your dataset as explained here: https://keras.io/models/model/#fit
Additionally you should add a Flatten layer and a dense layer with an ouput shape of 1 to get the correct result shape.