We Keep Coding

Load weight of a model trained as one functional restnet into restnet model but built using multiple layer

I have used a code that trained a resnet model as none functional layer;
base_model = tf.keras.applications.ResNet50(include_top=False, weights=None, input_shape=(224, 224, 3))
base_model.trainable = True
inputs = Input((224, 224, 3))
h = base_model(inputs, training=True)
model = Model(inputs, projection_3)
when you call summary:
Layer (type) Output Shape Param #
=================================================================
input_image (InputLayer) [(None, 256, 256, 3)] 0
resnet50 (Functional) (None, 8, 8, 2048) 23587712
=================================================================
Now, I need to load the weight into resnet built of many layer
Resmodel = tf.keras.applications.ResNet50(input_tensor=inputs, weights=None, include_top=False)
However, when loading the weight, I got:
model.load_weights(filename)
ValueError: Layer count mismatch when loading weights from file. Model expected 106 layers, found 4 saved layers.
Its the same model, only one functional (whole model as one layer) and the other split into many layers. How do I transfer the weights between them.

try saving the model again
model_n = model.layers[1]
model_n.save("new_model.h5")

Unexpected output shape from a keras dense layer

I try to create a minimal non-convolutional NN image binary classifier with one hidden layer only (as a practice before more complicated models):
def make_model(input_shape):
inputs = keras.Input(shape=input_shape)
x = layers.Dense(128, activation="ReLU")(inputs)
outputs = layers.Dense(1, activation="sigmoid")(x)
return keras.Model(inputs, outputs)
model = make_model(input_shape=(256, 256, 3))
Its model.summary() shows
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 256, 256, 3)] 0
dense (Dense) (None, 256, 256, 128) 512
dense_1 (Dense) (None, 256, 256, 1) 129
=================================================================
Total params: 641
Trainable params: 641
Non-trainable params: 0
Since the dense_1 layer has one neuron only, what I expect from this layer is an output shape of (None, 1) (i,e, a single number indicating the predicted binary label) but instead the model gives (None, 256, 256, 1).
What's wrong with my model setting and how can I get it right?

You have to flatten your preposterously large tensor if you want to use the output shape (None, 1):
import tensorflow as tf
def make_model(input_shape):
inputs = tf.keras.layers.Input(shape=input_shape)
x = tf.keras.layers.Dense(128, activation="relu")(inputs)
x = tf.keras.layers.Flatten()(x)
outputs = tf.keras.layers.Dense(1, activation="sigmoid")(x)
return tf.keras.Model(inputs, outputs)
model = make_model(input_shape=(256, 256, 3))
print(model.summary())

A mistake is in your function make_model.
def make_model(input_shape):
inputs = keras.Input(shape=input_shape)
x = layers.Dense(128, activation="ReLU")(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
return keras.Model(inputs, outputs)
You probably wanted the second line to be
x = layers.Dense(128, activation="ReLU")(inputs)
and not
x = layers.Dense(128, activation="ReLU")(x)
and unfortunately, x exists in scope, so it didn't throw an error.

TensorFlow input shape error at Dense output layer is contradictory to what model.summary() says

I am playing around with an NLP problem (sentence classification) and decided to use HuggingFace's TFBertModel along with Conv1D, Flatten, and Dense layers. I am using the functional API and my model compiles. However, during model.fit(), I get a shape error at the output Dense layer.
Model definition:
# Build model with a max length of 50 words in a sentence
max_len = 50
def build_model():
bert_encoder = TFBertModel.from_pretrained(model_name)
input_word_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_word_ids")
input_mask = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_mask")
input_type_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="input_type_ids")
# Create a conv1d model. The model may not really be useful or make sense, but that's OK (for now).
embedding = bert_encoder([input_word_ids, input_mask, input_type_ids])[0]
conv_layer = tf.keras.layers.Conv1D(32, 3, activation='relu')(embedding)
dense_layer = tf.keras.layers.Dense(24, activation='relu')(conv_layer)
flatten_layer = tf.keras.layers.Flatten()(dense_layer)
output_layer = tf.keras.layers.Dense(3, activation='softmax')(flatten_layer)
model = tf.keras.Model(inputs=[input_word_ids, input_mask, input_type_ids], outputs=output_layer)
model.compile(tf.keras.optimizers.Adam(lr=1e-5), loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
return model
# View model architecture
model = build_model()
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_word_ids (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
input_mask (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
input_type_ids (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
tf_bert_model (TFBertModel) ((None, 50, 768), (N 177853440 input_word_ids[0][0]
input_mask[0][0]
input_type_ids[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 48, 32) 73760 tf_bert_model[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 48, 24) 792 conv1d[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 1152) 0 dense[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 3) 3459 flatten[0][0]
==================================================================================================
Total params: 177,931,451
Trainable params: 177,931,451
Non-trainable params: 0
__________________________________________________________________________________________________
# Fit model on input data
model.fit(train_input, train['label'].values, epochs = 3, verbose = 1, batch_size = 16,
validation_split = 0.2)
And this is the error message:
ValueError: Input 0 of layer dense_1 is incompatible with the layer: expected axis -1 of input shape to have value 1152 but received
input with shape [16, 6168]
I am unable to understand how the input shape to layer dense_1 (the output dense layer) can be 6168? As per the model summary, it should always be 1152.

The shape of your input is likely not as you expect. Check the shape of train_input.

How do I connect two keras models into one model?

Let's say I have a ResNet50 model and I wish to connect the output layer of this model to the input layer of a VGG model.
This is the ResNet model and the output tensor of ResNet50:
img_shape = (164, 164, 3)
resnet50_model = ResNet50(include_top=False, input_shape=img_shape, weights = None)
print(resnet50_model.output.shape)
I get the output:
TensorShape([Dimension(None), Dimension(6), Dimension(6), Dimension(2048)])
Now I want a new layer where I reshape this output tensor to (64,64,18)
Then I have a VGG16 model:
VGG_model = VGG_model = VGG16(include_top=False, weights=None)
I want the output of the ResNet50 to reshape into the desired tensor and fed in as an input to the VGG model. So essentially I want to concatenate two models. Can someone help me do that?
Thank you!

There are multiple ways you can do this. Here is one way of using Sequential model API to do it.
import tensorflow as tf
from tensorflow.keras.applications import ResNet50, VGG16
model = tf.keras.Sequential()
img_shape = (164, 164, 3)
model.add(ResNet50(include_top=False, input_shape=img_shape, weights = None))
model.add(tf.keras.layers.Reshape(target_shape=(64,64,18)))
model.add(tf.keras.layers.Conv2D(3,kernel_size=(3,3),name='Conv2d'))
VGG_model = VGG16(include_top=False, weights=None)
model.add(VGG_model)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
Model summary is as follows
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
resnet50 (Model) (None, 6, 6, 2048) 23587712
_________________________________________________________________
reshape (Reshape) (None, 64, 64, 18) 0
_________________________________________________________________
Conv2d (Conv2D) (None, 62, 62, 3) 489
_________________________________________________________________
vgg16 (Model) multiple 14714688
=================================================================
Total params: 38,302,889
Trainable params: 38,249,769
Non-trainable params: 53,120
_________________________________________________________________
Full code is here.

How can I insert a scalar value and a binary value to a layer (last layer) in keras?

I am trying to modify the network which is implemented here. This network uses chest x ray images as input and classifies it into 14 categories (13 types of diseases and no finding). The network does not take the patient age and gender as an input. So I want to provide the network with that information too. In short At the last 3 layers of the network is like the following:
bn (BatchNormalization) (None, 7, 7, 1024) 4096 conv5_block16_concat[0][0]
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 1024) 0 bn[0][0]
__________________________________________________________________________________________________
predictions (Dense) (None, 14) 14350 avg_pool[0][0]
So what I have done so far is the following:
I simply pop the last dense layer using model_vgg16.layers.pop().
Then as expexted the network turns into:
bn (BatchNormalization) (None, 7, 7, 1024) 4096 conv5_block16_concat[0][0]
__________________________________________________________________________________________________
avg_pool (GlobalAveragePooling2 (None, 1024) 0 bn[0][0]
I know that I can add a layer using:
new_layer = Dense(14, activation='softmax', name='my_dense')
inp = model.input
out = new_layer(model.layers[-1].output)
model2 = Model(inp, out)
But I do not know how to add a layer that takes the inputs from previous layer together with 1 scalar value (age [0:100]), and one binary value gender [0:1].
So How can I add a last layer that takes inputs from previous layer together with 1 scalar value and 1 binary value?
Edit: The base model I am using is DenseNet121. The some final layers looks like this:
EDIT
The way I load the model is the following:
cp = ConfigParser()
cp.read(config_file)
# default config
output_dir = cp["DEFAULT"].get("output_dir")
base_model_name = cp["DEFAULT"].get("base_model_name")
class_names = cp["DEFAULT"].get("class_names").split(",")
image_source_dir = cp["DEFAULT"].get("image_source_dir")
image_dimension = cp["TRAIN"].getint("image_dimension")
output_weights_name = cp["TRAIN"].get("output_weights_name")
weights_path = os.path.join(output_dir, output_weights_name)
best_weights_path = os.path.join(output_dir, f"best_{output_weights_name}")
model_weights_path = best_weights_path
model_factory = ModelFactory()
model = model_factory.get_model(
class_names,
model_name=base_model_name,
use_base_weights=False,
weights_path=model_weights_path)
Now the model is in variable model.
Then as suggested I do
x = model.output
flat1 = Flatten()(x)
and get this error:
ValueError: Input 0 is incompatible with layer flatten_27: expected min_ndim=3, found ndim=2
When I repeat the same thing after removing the last layer using model.layers.pop()
I still get the same error? Even though I have spent couple of hours cannot overcome that problem. So how can this be done?

try this
from keras.applications.densenet import DenseNet121
from keras.layers import Dense, GlobalAveragePooling2D,concatenate
input_image = Input(shape=(224, 224, 3))
# normalize age
input_age_and_gender = Input(shape=(2,))
base_model = DenseNet121(input_tensor=input_image, weights='imagenet', include_top=False)
x = base_model.output
encoded_image = GlobalAveragePooling2D()(x)
out = concatenate([encoded_image,input_age_and_gender])
output = Dense(14, activation='softmax')(out)
model = Model([input_image,input_age_and_gender],output)

You can have a multi input model.
So instead of just using this:
img_input = Input(shape=input_shape)
base_model = base_model_class(
include_top=False,
input_tensor=img_input,
input_shape=input_shape,
weights=base_weights,
pooling="avg")
x = base_model.output
predictions = Dense(len(class_names), activation="sigmoid", name="predictions")(x)
model = Model(inputs=img_input, outputs=predictions)
I am not sure what your base_model looks like there. BUT for the sake of it check the following, where the first input is imaginary and the shape of the second input should be the shape of your age_gender_df.values:
input1 = Input(shape=(64,64,1))
conv11 = Conv2D(32, kernel_size=4, activation='relu')(input1)
pool11 = MaxPooling2D(pool_size=(2, 2))(conv11)
conv12 = Conv2D(16, kernel_size=4, activation='relu')(pool11)
pool12 = MaxPooling2D(pool_size=(2, 2))(conv12)
flat1 = Flatten()(pool12)
# INSTEAD OF THE ABOVE INPUT I WROTE YOU CAN USE YOUR BASE MODEL
input2 = Input(shape=(2,2)) # HERE THIS SHOULD BE THE SHAPE OF YOUR AGE/GENDER DF
layer = Dense(10, activation='relu')(input2)
flat2 = Flatten()(layer)
merge = concatenate([flat1, flat2])
# interpretation model
hidden1 = Dense(10, activation='relu')(merge)
hidden2 = Dense(10, activation='relu')(hidden1)
output = Dense(14, activation='linear')(hidden2)
model = Model(inputs=[input1, input2], outputs=output)
Summary
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_30 (InputLayer) (None, 64, 64, 1) 0
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 61, 61, 32) 544 input_30[0][0]
__________________________________________________________________________________________________
max_pooling2d_23 (MaxPooling2D) (None, 30, 30, 32) 0 conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 27, 27, 16) 8208 max_pooling2d_23[0][0]
__________________________________________________________________________________________________
input_31 (InputLayer) (None, 2, 2) 0
__________________________________________________________________________________________________
max_pooling2d_24 (MaxPooling2D) (None, 13, 13, 16) 0 conv2d_24[0][0]
__________________________________________________________________________________________________
dense_38 (Dense) (None, 2, 10) 30 input_31[0][0]
__________________________________________________________________________________________________
flatten_23 (Flatten) (None, 2704) 0 max_pooling2d_24[0][0]
__________________________________________________________________________________________________
flatten_24 (Flatten) (None, 20) 0 dense_38[0][0]
__________________________________________________________________________________________________
concatenate_9 (Concatenate) (None, 2724) 0 flatten_23[0][0]
flatten_24[0][0]
__________________________________________________________________________________________________
dense_39 (Dense) (None, 10) 27250 concatenate_9[0][0]
__________________________________________________________________________________________________
dense_40 (Dense) (None, 10) 110 dense_39[0][0]
__________________________________________________________________________________________________
dense_41 (Dense) (None, 14) 154 dense_40[0][0]
==================================================================================================
Total params: 36,296
Trainable params: 36,296
Non-trainable params: 0
Visualisation:
EDIT:
In your case I suppose the model should look like the following:
img_input = Input(shape=input_shape)
base_model = base_model_class(
include_top=False,
input_tensor=img_input,
input_shape=input_shape,
weights=base_weights,
pooling="avg")
x = base_model.output
flat1 = Flatten()(x)
input2 = Input(shape=(2,2)) # HERE THIS SHOULD BE THE SHAPE OF YOUR AGE/GENDER DF
layer = Dense(10, activation='relu')(input2)
flat2 = Flatten()(layer)
merge = concatenate([flat1, flat2])
# interpretation model
hidden1 = Dense(10, activation='relu')(merge)
hidden2 = Dense(10, activation='relu')(hidden1)
output = Dense(14, activation='linear')(hidden2)
model = Model(inputs=[img_input, input2], outputs=output)

Issue is solved by first removing the last layer (prediction layer) by
model_original.layers.pop()
Then defining another model which is the replica of the original model except the last layer
model2 = keras.Model(model_original.input, model_original.layers[-1].output)
after that the input, which includes the age is defined
age = layers.Input(shape=(1,))
Next, age input and the last layer of the previously defined network is concatenated using
x = model2.output
concatenated = layers.concatenate([x, age])
In th final step, a prediction layer is added after the concatenation to complete the network
output = Dense(14, activation='linear')(concatenated)
model3 = keras.Model(inputs=[model_original.input, age], outputs=output)
So the final layers of the design looks like that: