Is it possible x*a from input size(None, None, 3) in Keras? For example, the input x and constant a: x(batch,None,None,1024)*a(batch,1).
I use input size (64, 64, 3) in train but test data should use variable input sizes. The test size can not be adjusted for fair image processing.
I try Lambda function(lambda x : x * a)(seq). Then, I got no problem in code. And then, I start model.fit function, I got error:
------------->>tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [7,4,4,1024] vs. [7,1].
.
conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)
conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)
conv_c = Conv2D(num_classes, 1, activation='softmax')(conv5)
conv_c1 = GlobalAveragePooling2D(name="class_output")(conv_c)
conv_c1_1 = conv_c1[:, 0:1]
conv_c1_2 = conv_c1[:, 1:2]
conv_c1_3 = conv_c1[:, 2:3]
conv5_b = Lambda(lambda x: x * conv_c1_1)(conv5) #conv5:Tensor(shape=(?, 4, 4, 1024))
conv5_h = Lambda(lambda x: x * conv_c1_2)(conv5) #conv_c1_1: Tensor(shape=(?, 1))
conv5_r = Lambda(lambda x: x * conv_c1_3)(conv5)
Related
I was writing a Convolution Neural Network code and this happened, the error continued even after changing the convolutions, I kept getting same errors
inp = Input(shape = (depth, height, width))
conv_1 = Convolution2D(conv_depth_1, kernel_size, kernel_size, padding = 'same', activation = 'relu')(inp)
conv_2 = Convolution2D(conv_depth_1, kernel_size, kernel_size, padding = 'same', activation = 'relu')(conv_1)
pool_1 = MaxPooling2D(pool_size = (pool_size, pool_size))(conv_2)
drop_1 = Dropout(drop_prob_1)(pool_1)
conv_3 = Convolution2D(conv_depth_2, kernel_size, kernel_size, padding = 'same', activation = 'relu')(drop_1)
conv_4 = Convolution2D(conv_depth_2, kernel_size, kernel_size, padding = 'same', activation = 'relu')(conv_3)
pool_2 = MaxPooling2D(pool_size = (pool_size, pool_size))(conv_4)
drop_2 = Dropout(drop_prob_1)(pool_2)
flat = Flatten()(drop_2)
hidden = Dense(hidden_size, activation = 'relu')(flat)
drop_3 = Dropout(drop_prob_2)(hidden)
out = Dense(num_classes, activation = 'softmax')(drop_3)
I kept getting this errors
ValueError: Exception encountered when calling layer "max_pooling2d_15" (type MaxPooling2D).
Negative dimension size caused by subtracting 2 from 1 for '{{node max_pooling2d_15/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NHWC", explicit_paddings=[], ksize=[1, 2, 2, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [?,1,1,64].
Call arguments received by layer "max_pooling2d_15" (type MaxPooling2D):
• inputs=tf.Tensor(shape=(None, 1, 1, 64), dtype=float32)
I'm trying to implement an external lambda function from a Keras model. But the function is called twice or the '### x' and '### x_reshape'are printed twice. How to eliminate this?
def RA_reshape(x):
print('\n### x ', x)
x_reshape = K.reshape(x, [1, x.shape[1].value, x.shape[2].value, x.shape[3].value])
print('\n### x_reshape ', x_reshape)
return x_reshape
def unet(pretrained_weights = None,input_size = None):
inputs = Input(input_size)
conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
RA_0 = Lambda(lambda y: RA_reshape(x=y) )
conv1 = RA_0(conv1)
Could you try to call it like that :
def unet(pretrained_weights = None,input_size = None):
inputs = Input(input_size)
conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
RA_0 = Lambda(RA_reshape)(conv1)
Tell me if that working !
This is Keras working internally and building "the graph", it does call functions twice and there is nothing much we can do.
Notice that since it's a graph, it will not print the values as you want! It will print only the symbolic tensors.
For printing the data you need to create a little model only to predict the data you want to see. Or you need to enable eager mode and not use "predict", but manually go calling each layer with a tensor.
My neural network receives a (1000, 1000, 5) shape array which undergoes convolution in one branch (5 stacked raster images) and a (12) shape array (just 12 numbers) which go through a couple of dense layers in a second branch.
The outputs are concatenated into a (31, 31, 65) shape tensor which then goes deconvolution into a final (1000, 1000) shape array.
My Issue:
I made my own simple loss function (mean error), because the output represents temperature in an area.
My issue is currently that my loss goes down significantly over 200 epochs (both loss and val_loss, from a small decimal to about -3) and the accuracy hovers around 0.002 the entire time .
I have changed the learning rate as low as 1e-5. I have given more samples to the training set (there aren't many samples to begin with unfortunately), increased (for fear of overfitting) and decreased (for lack of data) the batch size. All the input data is normalized to 0:1, which makes losses of anything beyond -1 unreasonable.
I am not sure whether I should use a different optimizer for this task, or different activation, or just remove a layer or two. But mostly I'd love to understand what is happening to make the model so unreliable.
I really tried to refrain from having to post the entire thing on here but I am officially out of ideas.
MLP Branch
dim = 12
inputs = Input(shape = (dim, ))
x = inputs
x = Dense(dim * 4, activation = 'relu')(x)
x = Dense(dim * 16, activation = 'relu')(x)
x = Dense(961, activation = 'relu')(x) # 961 nodes
x = Reshape((31, 31, 1))(x) # (31, 31, 1) array
model1 = Model(inputs, x)
Convolutional Branch
inputShape = (1000, 1000, 5)
chanDim = -1
inputs = Input(shape = inputShape)
x = inputs
# layer 1: conv, f = 8, pool = 2
x = Conv2D(8, (3, 3), padding = 'same', activation = 'relu')(x)
x = BatchNormalization(axis = chanDim)(x)
x = MaxPooling2D(pool_size = (2, 2))(x)
# layer 2: conv, f = 16, pool = 2
x = Conv2D(16, (3, 3), padding = 'same', activation = 'relu')(x)
x = BatchNormalization(axis = chanDim)(x)
x = MaxPooling2D(pool_size = (2, 2))(x)
# layer 3: conv, f = 32, pool = 2
x = Conv2D(32, (3, 3), padding = 'same', activation = 'relu')(x)
x = BatchNormalization(axis = chanDim)(x)
x = MaxPooling2D(pool_size = (2, 2))(x)
# layer 4: conv = 64, pool = 4
x = Conv2D(64, (3, 3), padding = 'same', activation = 'relu')(x)
x = BatchNormalization(axis = chanDim)(x)
x = MaxPooling2D(pool_size = (4, 4))(x)
model2 = Model(inputs, x)
Deconvolution
combinedInput = Concatenate()([model1.output, model2.output])
x = combinedInput # (31, 31, 65)
x = Conv2DTranspose(43, (3, 3), strides = (4, 4), padding = 'same', activation = 'relu')(x) # (124, 124, 43)
x = Conv2DTranspose(22, (3, 3), strides = (2, 2), padding = 'same', activation = 'relu')(x) # (248, 248, 22)
x = Lambda(lambda y: spatial_2d_padding(y))(x) # (250, 250, 22)
x = Conv2DTranspose(10, (3, 3), strides = (2, 2), padding = 'same', activation = 'relu')(x) # (500, 500, 10)
x = Conv2DTranspose(1, (3, 3), strides = (2, 2), padding = 'same', activation = 'linear')(x) # (1000, 1000, 1)
x = Lambda(lambda y: squeeze(y, axis = 3))(x) # (1000, 1000)
Compiling
def custom_loss(y_actual, y_predicted):
custom_loss_value = mean(y_actual - y_predicted)
return custom_loss_value
model = Model(inputs = [mlp.input, cnn.input], outputs = x)
model.compile(loss = custom_loss, optimizer = Adam(lr = 0.000001), metrics = ['mae'])
# train with epochs = 200, batch_size = 12
The Issue
As I explained above, my loss never stabilizes and the accuracy hovers roughly around the same number over the epochs.
I'd love to know possible reasons and possible solutions.
Edits:
Since writing this question I have attempted:
Transfering layers from the convolution branch to the deconvolution branch.
Adding BatchNormalization() after every Conv2DTranspose() layer.
I'm relatively new to keras/CNNs. I've built a model that concatenates the output from 3 sequential CNNs with some other metadata into a final Dense network. The outputs from the 3 individual layers are sensible and perform ok, but the final output when concatenated with the meta data is showing worse performance and doesn't seem to learn - even though some of this meta data should be very useful for prediction. The labels are one-hot coded classification data (4 different labels). I'm a bit confused why the final concatenated model is performing so poorly compared to the individual pieces, would appreciate any insight into what I might be doing wrong here. Thanks!
# create first conv layers
first_input = Input(shape=input_shape, dtype='int32', name='first_input')
x = Embedding(input_dim=num_features,output_dim=embedding_dim,input_length=input_shape[0])(first_input)
#x = Dropout(rate = dropout_rate)(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = MaxPooling1D(pool_size=pool_size)(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = GlobalAveragePooling1D()(x)
aux_predictions = Dense(op_units, activation=op_activation)(x)
# now create a convolutional model for second
second_input = Input(shape=input_shape, dtype='int32', name='second_input')
x = Embedding(input_dim=num_features,output_dim=embedding_dim,input_length=input_shape[0])(second_input)
#x = Dropout(rate = dropout_rate)(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = MaxPooling1D(pool_size=pool_size)(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = GlobalAveragePooling1D()(x)
aux_predictions2 = Dense(op_units, activation=op_activation)(x)
# now create a convolutional model for second
third_input = Input(shape=input_shape, dtype='int32', name='third_input')
x = Embedding(input_dim=num_features,output_dim=embedding_dim,input_length=input_shape[0])(third_input)
#x = Dropout(rate = dropout_rate)(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = MaxPooling1D(pool_size=pool_size)(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = Conv1D(filters=filters * 2,
kernel_size=kernel_size,
strides = 1,
activation='relu',
bias_initializer='random_uniform',
padding='same')(x)
x = GlobalAveragePooling1D()(x)
aux_predictions3 = Dense(op_units, activation=op_activation)(x)
# Now combine three CNN layers with metadata
auxiliary_input = Input(shape=metadata_dim, name='aux_input')
x = keras.layers.concatenate([aux_predictions, aux_predictions2, aux_predictions3, auxiliary_input ])
#x = Dropout(rate = dropout_rate)(x)
x = Dense(64, activation='relu')(x)
x = Dense(64, activation='relu')(x)
x = Dense(64, activation='relu')(x)
model_output = Dense(op_units, activation=op_activation, name='model_output')(x)
opt = SGD(lr=0.01)
fullmetamodel = Model(inputs=[first_input, second_input, third_input, auxiliary_input], outputs=[aux_predictions, aux_predictions2, aux_predictions3, model_output])
fullmetamodel.compile(
metrics=['categorical_accuracy'], loss='categorical_crossentropy',
loss_weights=[0.2, 0.2, 0.2, 1.], optimizer = opt)
callbacks = [keras.callbacks.EarlyStopping(monitor='val_loss', patience=2), TQDMNotebookCallback(leave_inner=False, leave_outer = True)]
fullmetamodel.fit(x=[first_x_train, second_x_train, third_x_train, training_meta], y=[training_labels,training_labels,training_labels, training_labels],
batch_size=32, epochs=40, validation_data=([first_x_val, second_x_val, third_x_val, val_meta], [val_labels, val_labels, val_labels, val_labels])
, verbose = 0, callbacks = callbacks) # starts training
# Output, three conv layers working ok, concatenated model performing poorly
Training
50% 20/40 [2:49:34<2:49:23, 508.20s/it]
Epoch 20
[loss: 8.002, dense_118_loss: 0.749, dense_119_loss: 0.769, dense_120_loss: 0.876, model_output_loss: 7.523, dense_118_categorical_accuracy: 0.686
, dense_119_categorical_accuracy: 0.626, dense_120_categorical_accuracy: 0.620, model_output_categorical_accuracy: 0.532] : 66% 265184/400000 [05:13<02:40, 840.90it/s]
I am trying to detect the single pixel location of a single object in an image. I have a keras CNN regression network with my image tensor as the input, and a 3 item vector as the output.
First item: Is a 1 (if an object was found) or 0 (no object was found)
Second item: Is a number between 0 and 1 which indicates how far along the x axis is the object
Third item: Is a number between 0 and 1 which indicates how far along the y axis is the object
I have trained the network on 2000 test images and 500 validation images, and the val_loss is far less than 1, and the val_acc is best at around 0.94. Excellent.
But then when I predict the output, I find the values for all three output items are not between 0 and 1, they are actually between -2 and 3 approximately. All three items should be between 0 and 1.
I have not used any non-linear activation functions on the output layer, and have used relus for all non-output layers. Should I be using a softmax, even though it is non-linear? The second and third items are predicting the x and y axis of the image, which appear to me as linear quantities.
Here is my keras network:
inputs = Input((256, 256, 1))
base_kernels = 64
# 256
conv1 = Conv2D(base_kernels, 3, activation='relu', padding='same', kernel_initializer='he_normal')(inputs)
conv1 = BatchNormalization()(conv1)
conv1 = Conv2D(base_kernels, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv1)
conv1 = BatchNormalization()(conv1)
conv1 = Dropout(0.2)(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
# 128
conv2 = Conv2D(base_kernels * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool1)
conv2 = BatchNormalization()(conv2)
conv2 = Conv2D(base_kernels * 2, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv2)
conv2 = BatchNormalization()(conv2)
conv2 = Dropout(0.2)(conv2)
pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
# 64
conv3 = Conv2D(base_kernels * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(pool2)
conv3 = BatchNormalization()(conv3)
conv3 = Conv2D(base_kernels * 4, 3, activation='relu', padding='same', kernel_initializer='he_normal')(conv3)
conv3 = BatchNormalization()(conv3)
conv3 = Dropout(0.2)(conv3)
pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
flat = Flatten()(pool3)
dense = Dense(256, activation='relu')(flat)
output = Dense(3)(dense)
model = Model(inputs=[inputs], outputs=[output])
optimizer = Adam(lr=1e-4)
model.compile(optimizer=optimizer, loss='mean_absolute_error', metrics=['accuracy'])
Can anyone please help? Thanks! :)
Chris
The sigmoid activation produces outputs between zero and one, so if you use it as activation of your last layer(the output), the network's output will be between zero and one.
output = Dense(3, activation="sigmoid")(dense)