I am working on a biomedical image segmentation project. I am using U-net model for the job. The problem is, when i train the model, the validation loss doesn't seem to be practical.
I used the dice_coef_loss as loss function, as well as, in the metric. The result of the training is the below graph. The graph clearly shows that validation loss is not following my loss function, cause the two graphs are distinguishable. Though, the train loss does follow the train dice_coef_loss values.
(The first image from the left is training and validation loss, third one is tarining and validation dice_coef_loss as metric)
The history graph of training
(Sorry i am not yet eligible to embed an image, please check the link)
Here is my model
def unet(input_size=(256,256,1)):
inputs = Input(input_size)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2)
conv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(pool3)
conv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2))(conv4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)
up6 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(conv5), conv4], axis=3)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(up6)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv6)
up7 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(conv6), conv3], axis=3)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(up7)
conv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv7)
up8 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(conv7), conv2], axis=3)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(up8)
conv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv8)
up9 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv8), conv1], axis=3)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(up9)
conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv9)
conv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv9)
return Model(inputs=[inputs], outputs=[conv10])
Here is the loss function
import keras.backend as K
smooth=100
def dice_coef(y_true, y_pred):
y_truef=K.flatten(y_true)
y_predf=K.flatten(y_pred)
And=K.sum(y_truef* y_predf)
return((2* And + smooth) / (K.sum(y_truef) + K.sum(y_predf) + smooth))
def dice_coef_loss(y_true, y_pred):
return -dice_coef(y_true, y_pred)
Compilation
model.compile(optimizer=Adam(lr=1e-5), loss=dice_coef_loss, metrics=["binary_accuracy", dice_coef_loss, iou])
Note
I have tried:
learning rate tuning
tuning batch size
data augmentation
changing the loss function
If any one want to look at the code here is the link to kaggle kernel
Addition Note
To make my self clear :
The 1st and second graph was generated from a same function using same dataset (either from train or validation time), so I want the train_loss curve from 1st plot equal to train_dice_coef_loss from 3rd plot. And val_loss curve from 1st plot equal to val_dice_coef_loss from 3rd plot.
But unfortunately val_loss curve doesn't match val_dice_coef_loss curve.
P.S. I am new here. Any suggestion to improve my question is appreciated. Thank you.
As, ZabirAlNazi suggested it was a problem of the used library. Changing the imports from keras to tensorflow.keras solved that issue.
from tensorflow.keras.models import Model, load_model, save_model
from tensorflow.keras.layers import Input, Activation, BatchNormalization, Dropout, Lambda, Dense, Conv2D
Related
i used unet model with 512x512x1 input shape
but i have a problem. I am getting a ValueError : Dimensions must be equal. I know that something goes wrong with input shapes. i need solve this problem.maybe there is a problem with the loss of the model
X Train Shape : (512, 512, 1)
Y Train Shape : (512, 512, 1)
Model:
from keras.models import Model
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import concatenate, Conv2D, MaxPooling2D, Conv2DTranspose
from keras.layers import Input, merge, UpSampling2D,BatchNormalization
from keras.callbacks import ModelCheckpoint
from keras.optimizers import Adam
from keras.preprocessing.image import ImageDataGenerator
from keras import backend as K
import tensorflow as tf
K.set_image_data_format('channels_last')
def dice_coef(y_true, y_pred):
smooth = 0.005
y_true_f = K.flatten(y_true)
y_pred_f = K.flatten(y_pred)
intersection = K.sum(y_true_f * y_pred_f)
return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)
def dice_coef_loss(y_true, y_pred):
return 1-dice_coef(y_true, y_pred)
def unet_model():
inputs = Input((512 , 512, 1))
conv1 = Conv2D(64, (3, 3), activation='relu', padding='same') (inputs)
batch1 = BatchNormalization(axis=1)(conv1)
conv1 = Conv2D(64, (3, 3), activation='relu', padding='same') (batch1)
batch1 = BatchNormalization(axis=1)(conv1)
pool1 = MaxPooling2D((2, 2)) (batch1)
conv2 = Conv2D(128, (3, 3), activation='relu', padding='same') (pool1)
batch2 = BatchNormalization(axis=1)(conv2)
conv2 = Conv2D(128, (3, 3), activation='relu', padding='same') (batch2)
batch2 = BatchNormalization(axis=1)(conv2)
pool2 = MaxPooling2D((2, 2)) (batch2)
conv3 = Conv2D(256, (3, 3), activation='relu', padding='same') (pool2)
batch3 = BatchNormalization(axis=1)(conv3)
conv3 = Conv2D(256, (3, 3), activation='relu', padding='same') (batch3)
batch3 = BatchNormalization(axis=1)(conv3)
pool3 = MaxPooling2D((2, 2)) (batch3)
conv4 = Conv2D(512, (3, 3), activation='relu', padding='same') (pool3)
batch4 = BatchNormalization(axis=1)(conv4)
conv4 = Conv2D(512, (3, 3), activation='relu', padding='same') (batch4)
batch4 = BatchNormalization(axis=1)(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2)) (batch4)
conv5 = Conv2D(1024, (3, 3), activation='relu', padding='same') (pool4)
batch5 = BatchNormalization(axis=1)(conv5)
conv5 = Conv2D(1024, (3, 3), activation='relu', padding='same') (batch5)
batch5 = BatchNormalization(axis=1)(conv5)
up6 = Conv2DTranspose(512, (2, 2), strides=(2, 2), padding='same') (batch5)
up6 = concatenate([up6, conv4], axis=1)
conv6 = Conv2D(512, (3, 3), activation='relu', padding='same') (up6)
batch6 = BatchNormalization(axis=1)(conv6)
conv6 = Conv2D(512, (3, 3), activation='relu', padding='same') (batch6)
batch6 = BatchNormalization(axis=1)(conv6)
up7 = Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same') (batch6)
up7 = concatenate([up7, conv3], axis=1)
conv7 = Conv2D(256, (3, 3), activation='relu', padding='same') (up7)
batch7 = BatchNormalization(axis=1)(conv7)
conv7 = Conv2D(256, (3, 3), activation='relu', padding='same') (batch7)
batch7 = BatchNormalization(axis=1)(conv7)
up8 = Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same') (batch7)
up8 = concatenate([up8, conv2], axis=1)
conv8 = Conv2D(128, (3, 3), activation='relu', padding='same') (up8)
batch8 = BatchNormalization(axis=1)(conv8)
conv8 = Conv2D(128, (3, 3), activation='relu', padding='same') (batch8)
batch8 = BatchNormalization(axis=1)(conv8)
up9 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (batch8)
up9 = concatenate([up9, conv1], axis=1)
conv9 = Conv2D(64, (3, 3), activation='relu', padding='same') (up9)
batch9 = BatchNormalization(axis=1)(conv9)
conv9 = Conv2D(64, (3, 3), activation='relu', padding='same') (batch9)
batch9 = BatchNormalization(axis=1)(conv9)
conv10 = Conv2D(1, (1, 1), activation='sigmoid')(batch9)
model = Model(inputs=[inputs], outputs=[conv10])
model.compile(optimizer=Adam(lr=1e-4), loss=dice_coef_loss, metrics=[dice_coef])
return model
model = unet_model()
If you check the model summary (model.summary()) you will see that the last layer output shape is:
Layer (type) Output Shape Param # Connected to
===============================================================================
...
...
conv2d_18 (Conv2D) (None, 2560, 512, 1) 65 batch_normalization_17[0][0]
The output shape is (None,2560,512,1) which is not match with the desired output shape that is (None, 512, 512, 1)
This mismatch is because of your concatenate layers. You have set axis=1 parameter in the concatenate layers which concatenate second axis, but you need to concatenate layers based on last axis which is axis=-1 and the default.
So, Remove axis parameter from your concatenate layers to let it remain the default or set axis=-1 in all your concatenate layers.
For example:
#up6 = concatenate([up6, conv4], axis=1) # remove axis
up6 = concatenate([up6, conv4])
Then, you will get last layer shape the same as your input and your y_train:
Layer (type) Output Shape Param # Connected to
===============================================================================
...
...
conv2d_18 (Conv2D) (None, 512, 512, 1) 65 batch_normalization_17[0][0]
I have an autoencoder with two outputs and the first output should be used as an input to the next part of autoencoder after some changes. I put my code here and I'm a little confused that this code is logically true or not. in the following code I have one output in decoder part named act11 the output of sigmoid activation and the second output is in the w extraction part with name pred_W. I feed bncv11 to the GaussianNoise instead of act11. I want to know it is correct or not? based on back propagation rules and the structure of the network is it possible to do this? can I just use the output of activation in this code act11=Activation('sigmoid',name='imageprim')(bncv11) for the output of model?
all of my questions is about this part of the code:
decoded = Conv2D(1, (5, 5), padding='same', name='decoder_output',dilation_rate=(2,2))(BNd)
bncv11=BatchNormalization()(decoded)
act11=Activation('sigmoid',name='imageprim')(bncv11)
decoded_noise = GaussianNoise(0.5)(bncv11)
#----------------------w extraction------------------------------------
convw1 = Conv2D(64, (3,3), activation='relu', padding='same', name='conl1w',dilation_rate=(2,2))(decoded_noise)
watermark_extraction=Model(inputs=[image,wtm],outputs=[act11,pred_w])
I want to know the above code based on deep learning is correct or not?
I just use act11 in the activation and for output but use bncv11 for feeding during learning.
wtm=Input((28,28,1))
image = Input((28, 28, 1))
conv1 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl1e',dilation_rate=(2,2))(image)
conv2 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl2e',dilation_rate=(2,2))(conv1)
conv3 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl3e',dilation_rate=(2,2))(conv2)
BN=BatchNormalization()(conv3)
encoded = Conv2D(1, (5, 5), activation='relu', padding='same',name='encoded_I',dilation_rate=(2,2))(BN)
add_const = Kr.layers.Lambda(lambda x: x[0] + x[1])
encoded_merged = add_const([encoded,wtm])
#-----------------------decoder------------------------------------------------
deconv1 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl1d',dilation_rate=(2,2))(encoded_merged)
deconv2 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl2d',dilation_rate=(2,2))(deconv1)
deconv3 = Conv2D(64, (5, 5), activation='relu',padding='same', name='convl3d',dilation_rate=(2,2))(deconv2)
deconv4 = Conv2D(64, (5, 5), activation='relu',padding='same', name='convl4d',dilation_rate=(2,2))(deconv3)
BNd=BatchNormalization()(deconv3)
decoded = Conv2D(1, (5, 5), padding='same', name='decoder_output',dilation_rate=(2,2))(BNd)
bncv11=BatchNormalization()(decoded)
act11=Activation('sigmoid',name='imageprim')(bncv11)
decoded_noise = GaussianNoise(0.5)(bncv11)
#----------------------w extraction------------------------------------
convw1 = Conv2D(64, (3,3), activation='relu', padding='same', name='conl1w',dilation_rate=(2,2))(decoded_noise)
convw2 = Conv2D(64, (3, 3), activation='relu', padding='same', name='convl2w',dilation_rate=(2,2))(convw1)
convw3 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl3w',dilation_rate=(2,2))(convw2)
convw4 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl4w',dilation_rate=(2,2))(convw3)
convw5 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl5w',dilation_rate=(2,2))(convw4)
convw6 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl6w',dilation_rate=(2,2))(convw5)
pred_w = Conv2D(1, (1, 1), activation='sigmoid', padding='same', name='reconstructed_W',dilation_rate=(2,2))(convw6)
watermark_extraction=Model(inputs=[image,wtm],outputs=[act11,pred_w])
I am having trouble getting this model to compile.
I am trying to implement a VGG16 but I will be using a custom loss function. The target variable has a shape of (?, 14, 14, 9, 6) where we only use binary crossentropy on Y_train[:,:,:,:,0] then Y_train[:,:,:,:,1] as a switch to turn off the loss effectively making this a mini-batch -- the others will be used on a separate branch of the neural net. This is a binary classification problem on this branch so I only want to have output of shape (?, 14, 14, 9, 1).
I have listed my error below. Can you please explain firstly what is going wrong and secondly how to mitigate this issue?
Model code
img_input = Input(shape = (224,224,3))
x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(img_input)
x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
# # Block 2
x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1')(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
# Block 3
x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1')(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2')(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x)
# # Block 4
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3')(x)
x = MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
# # Block 5
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3')(x)
x = Conv2D(512, (3, 3), padding='same', activation='relu', kernel_initializer='normal', name='rpn_conv1')(x)
x_class = Conv2D(9, (1, 1), activation='sigmoid', kernel_initializer='uniform', name='rpn_out_class')(x)
x_class = Reshape((14,14,9,1))(x_class)
model = Model(inputs=img_input, outputs=x_class)
model.compile(loss=rpn_loss_cls(), optimizer='adam')
Loss function code:
def rpn_loss_cls(lambda_rpn_class=1.0, epsilon = 1e-4):
def rpn_loss_cls_fixed_num(y_true, y_pred):
return lambda_rpn_class * K.sum(y_true[:,:,:,:,0]
* K.binary_crossentropy(y_pred[:,:,:,:,:], y_true[:,:,:,:,1]))
/ K.sum(epsilon + y_true[:,:,:,:,0])
return rpn_loss_cls_fixed_num
Error:
ValueError: logits and labels must have the same shape ((?, ?, ?, ?) vs (?, 14, 14, 9, 1))
Note: I have read multiple question on this site having the same error, but none of the solutions allowed my model to compile.
Potential solution:
I continued messing with this and found that by adding
y_true = K.expand_dims(y_true, axis=-1)
I was able to compile the model. Still dubious that this is going to work correctly.
Keras model set y_true shape equivalent to input shape. Therefore, when your loss function gets shape mismatch error. So you need to align dimensions by using expand_dims. This, however, needs to be done considering your model architecture, data and loss function. Code below will compile.
def rpn_loss_cls(lambda_rpn_class=1.0, epsilon = 1e-4):
def rpn_loss_cls_fixed_num(y_true, y_pred):
y_true = tf.keras.backend.expand_dims(y_true, -1)
return lambda_rpn_class * K.sum(y_true[:,:,:,:,0]
* K.binary_crossentropy(y_pred[:,:,:,:,:], y_true[:,:,:,:,1]))
/ K.sum(epsilon + y_true[:,:,:,:,0])
return rpn_loss_cls_fixed_num
I am new to CNN and I am not able to identify how to solve this problem.
In this code I am training a set of images to obtain mask from convolutional network.the images are grayscale with shape (200,200). I am not able to identify where I am making a mistake.Also everytime I run my code there is error at different inputs.Any help would be appreciated.
following is the log generated:
Creating training images...
Saving to .npy files done.
Creating test images...
Saving to .npy files done.
------------------------------
Loading and preprocessing train data...
------------------------------
------------------------------
Creating and compiling model...
------------------------------
C:/Users/Asus/Desktop/training.py:101: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(25, (3, 3), activation="relu", padding="same", data_format="channels_last")`
conv2 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="th")(inputs)
C:/Users/Asus/Desktop/training.py:102: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(25, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv2 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="th")(conv2)
C:/Users/Asus/Desktop/training.py:103: UserWarning: Update your `MaxPooling2D` call to the Keras 2 API: `MaxPooling2D(pool_size=(2, 2), data_format="channels_last")`
pool2 = MaxPooling2D(pool_size=(2, 2), dim_ordering="tf")(conv2)
C:/Users/Asus/Desktop/training.py:105: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(50, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv3 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="th")(pool2)
C:/Users/Asus/Desktop/training.py:106: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(50, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv3 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="th")(conv3)
C:/Users/Asus/Desktop/training.py:107: UserWarning: Update your `MaxPooling2D` call to the Keras 2 API: `MaxPooling2D(pool_size=(2, 2), data_format="channels_last")`
pool3 = MaxPooling2D(pool_size=(2, 2),dim_ordering="tf")(conv3)
C:/Users/Asus/Desktop/training.py:109: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(100, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv4 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="th")(pool3)
C:/Users/Asus/Desktop/training.py:110: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(100, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv4 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="th")(conv4)
C:/Users/Asus/Desktop/training.py:111: UserWarning: Update your `MaxPooling2D` call to the Keras 2 API: `MaxPooling2D(pool_size=(2, 2), data_format="channels_last")`
pool4 = MaxPooling2D(pool_size=(2, 2), dim_ordering="tf")(conv4)
C:/Users/Asus/Desktop/training.py:113: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(200, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv5 = Conv2D(200, (3, 3), activation='relu', padding='same',dim_ordering="th")(pool4)
C:/Users/Asus/Desktop/training.py:114: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(200, (3, 3), activation="relu", padding="same", data_format="channels_first")`
conv5 = Conv2D(200, (3, 3), activation='relu', padding='same',dim_ordering="th")(conv5)
C:/Users/Asus/Desktop/training.py:116: UserWarning: Update your `Conv2DTranspose` call to the Keras 2 API: `Conv2DTranspose(200, (2, 2), strides=(2, 2), padding="same", data_format="channels_first")`
up6 = concatenate([Conv2DTranspose(200, (2, 2), strides=(2, 2), padding='same',dim_ordering="th")(conv5), conv4], axis=3)
Traceback (most recent call last):
File "<ipython-input-25-4b34507d9da0>", line 1, in <module>
runfile('C:/Users/Asus/Desktop/training.py', wdir='C:/Users/Asus/Desktop')
File "C:\Users\Asus\AppData\Local\Continuum\anaconda3\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\Asus\AppData\Local\Continuum\anaconda3\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Asus/Desktop/training.py", line 205, in <module>
train_and_predict()
File "C:/Users/Asus/Desktop/training.py", line 163, in train_and_predict
model = get_unet()
File "C:/Users/Asus/Desktop/training.py", line 116, in get_unet
up6 = concatenate([Conv2DTranspose(200, (2, 2), strides=(2, 2), padding='same',dim_ordering="th")(conv5), conv4], axis=3)
File "C:\Users\Asus\AppData\Local\Continuum\anaconda3\envs\tensorflow\lib\site-packages\keras\layers\merge.py", line 641, in concatenate
return Concatenate(axis=axis, **kwargs)(inputs)
File "C:\Users\Asus\AppData\Local\Continuum\anaconda3\envs\tensorflow\lib\site-packages\keras\engine\topology.py", line 594, in __call__
self.build(input_shapes)
File "C:\Users\Asus\AppData\Local\Continuum\anaconda3\envs\tensorflow\lib\site-packages\keras\layers\merge.py", line 354, in build
'Got inputs shapes: %s' % (input_shape))
ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 200, 50, 50), (None, 100, 50, 25)]
Here is my code:
#load dataset
import h5py
h5f = h5py.File('liver_augmented_dataset.h5', 'r')
X = h5f['ct_scans'][:]
Y = h5f['seg_mask'][:]
h5f.close()
X_ax = X[1310:2500]
Y_ax = Y[1310:2500]
X_t=X[2501:2619]
Y_t=Y[2501:2619]
image_rows = 200
image_cols = 200
def get_unet():
inputs = Input(shape=(img_rows, img_cols,1))
# conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
# conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)
# pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="tf")(inputs)
conv2 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2), dim_ordering="tf")(conv2)
conv3 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="tf")(pool2)
conv3 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2),dim_ordering="tf")(conv3)
conv4 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="tf")(pool3)
conv4 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv4)
pool4 = MaxPooling2D(pool_size=(2, 2), dim_ordering="tf")(conv4)
conv5 = Conv2D(200, (3, 3), activation='relu', padding='same',dim_ordering="tf")(pool4)
conv5 = Conv2D(200, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv5)
up6 = concatenate([Conv2DTranspose(200, (2, 2), strides=(2, 2), padding='same',dim_ordering="tf")(conv5), conv4], axis=3)
conv6 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="tf")(up6)
conv6 = Conv2D(100, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv6)
up7 = concatenate([Conv2DTranspose(100, (2, 2), strides=(2, 2), padding='same',dim_ordering="tf")(conv6), conv3], axis=3)
conv7 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="tf")(up7)
conv7 = Conv2D(50, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv7)
up8 = concatenate([Conv2DTranspose(50, (2, 2), strides=(2, 2), padding='same',dim_ordering="tf")(conv7), conv2], axis=3)
conv8 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="tf")(up8)
conv8 = Conv2D(25, (3, 3), activation='relu', padding='same',dim_ordering="tf")(conv8)
#
# up9 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(conv8), conv1], axis=3)
# conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(up9)
# conv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv9)
conv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv8)
model = Model(inputs=[inputs], outputs=[conv10])
model.compile(optimizer=Adam(lr=1e-5), loss=dice_coef_loss, metrics=[dice_coef])
return model
I was able to compile the model successfully.
I wasn't able to recreate the Concatenate error mentioned in the log.
The other you should check is the input you provide to model should be reshaped in 4 dimension as in the question you mentioned reshape error for (1190, 200, 200) ,you should convert it to (1190, 200, 200, 1) the '1' is for number of bands.
so basically you should add an extra dimension to your grey scale image and convert it to (img_rows,img_cols,bands)
I came across the same situation with grey images, a reshape on the images will solve it by adding the extra dimension for grey scale channel.
train_images_reshape = train_images.reshape(no_images_train, h,w,1)
test_images_reshape = test_images.reshape(no_images_test, h,w,1)
keras will need an extra dimension to specify the channel
format is (no_of_images, height, width, n_channels)
n_channels=1 for grey scaled images
=3 for RGB
I am using the functional Model API and I would like to create the following custom loss function in Keras:
LOSS EQUATION IMAGE
I am implementing an unsupervised CNN and the values Ix, Iy, It and epsilon are constants, with u and v being the parameters I am trying to learn.
So far my attempts have failed with my model returning a ValueError saying "None values not supported" when I try to train it. After searching google I found that this is due to my loss not being differentiable.
Could somebody please show me how to correctly create this loss function in a manner where it is differentiable by Keras. My code is below and any help would be appreciated.
def charbonnier(I_x, I_y, I_t, U, V, e):
def charb(y_true, y_pred):
loss = K.sqrt(K.pow((multiply([U, I_x]) + multiply([V, I_y]) + I_t), 2) + e)
return K.mean(loss, axis=-1)
return charb
# Return the tf session used by the backend (optional)
tf_session = K.get_session()
def cnn(frame1, frame2):
# Concatenate the two images into one tensor.
input_ = np.dstack((frame1, frame2))
input_ = np.expand_dims(input_, axis=0)
print(input_.shape)
# Compute derivatives
I_x, I_y, I_t = compute_derivatives(input_)
#input_ = K.variable(input_)
inp_shape = (1, frame1.shape[0], frame1.shape[1], 2)
inp = Input(batch_shape=inp_shape)
conv1 = Conv2D(256, (7, 7), activation='relu', padding='same')(inp)
conv2 = Conv2D(256, (5, 5), activation='relu', padding='same')(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv2)
conv3 = Conv2D(512, (5, 5), activation='relu', padding='same')(pool1)
conv4 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv3)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv4)
conv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool2)
conv6 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv6)
conv7 = Conv2D(1024, (3, 3), activation='relu', padding='same')(pool3)
conv8 = Conv2D(512, (1, 1), activation='relu', padding='same')(conv7)
pool4 = MaxPooling2D(pool_size=(2, 2))(conv8)
conv9 = Conv2D(256, (5, 5), activation='relu', padding='same')(pool4)
upsample1 = UpSampling2D(size=(2, 2))(conv9)
conv10 = Conv2D(128, (5, 5), activation='relu', padding='same')(upsample1)
upsample2 = UpSampling2D(size=(2, 2))(conv10)
conv11 = Conv2D(64, (5, 5), activation='relu', padding='same')(upsample2)
upsample3 = UpSampling2D(size=(2, 2))(conv11)
conv12 = Conv2D(2, (15, 13), padding='same')(upsample3)
leaky_relu = LeakyReLU(alpha=0.3)(conv12)
upsample4 = UpSampling2D(size=(2, 2), name='flow')(leaky_relu)
model = Model(inputs=inp, outputs=upsample4)
U = K.variable(model.output[0, :, :, 0])
V = K.variable(model.output[0, :, :, 1])
e = K.constant(0.1)
c_loss = charbonnier(I_x, I_y, I_t, U, V, e)
model.compile(loss=c_loss, optimizer='adam')
model.fit(input_, input_, batch_size=1, epochs=10)
model.evaluate()
Solved.
Variables U and V should be created as follows:
U = model.output[0, :, :, 0]
V = model.output[0, :, :, 1]