I have been training this model for 1 epoch to see if it will work. What I want to do is take an image and create global features for the image. Right now after each training session, the features are all 0. Can someone please tell me the best way to create global features using Keras and cnn?
Here is my model so far.
def create_base_network():
"""
Base network to be shared.
"""
model_input = Input(shape=(224,224,3))
x = Conv2D(64, (3, 3), activation='relu', padding='same')(model_input)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Dropout(0.5)(x)
x = BatchNormalization()(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Dropout(0.5)(x)
x = BatchNormalization()(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(256, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Dropout(0.5)(x)
x = BatchNormalization()(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(2,2))(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
x = Conv2D(512, (3, 3), activation='relu', padding='same')(x)
#x = MaxPooling2D(pool_size=(2,2))(x)
x = Dropout(0.5)(x)
#x = Flatten()(x)
x = Dense(1024, activation='relu')(x)
x = BatchNormalization()(x)
x = Dropout(0.7)(x)
x = Dense(1024, activation='relu')(x)
x = Dense(1024, activation='relu')(x)
x = BatchNormalization()(x)
x = Dropout(0.7)(x)
x = Dense(4096, activation='relu')(x)
x = Dense(4096, activation='relu')(x)
x = BatchNormalization()(x)
x = Dropout(0.7)(x)
# This layer is what the features are
x = Dense(4096, activation='relu')(x)
model = Model(inputs=model_input, outputs=x)
return model
Please and thank you.
Related
I am working on a double task model to distinguish handwritten digits and their chirographies, but there is a problem in merging convolution layer and sharing the parameters. How do I need to modify my model?
Here is my code of model:
write_input = Input(shape=(28, 28, 1), name='write_input')
num_input = Input(shape=(28, 28, 1), name='num_input')
x = Conv2D(32, (3, 3), activation='relu')(num_input)
x = MaxPooling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu')(x)
x = MaxPooling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu')(x)
y = Conv2D(32, (3, 3), activation='relu')(write_input)
y = MaxPooling2D((2, 2))(y)
y = Conv2D(64, (3, 3), activation='relu')(y)
y = MaxPooling2D((2, 2))(y)
y = Conv2D(64, (3, 3), activation='relu')(y)
merge = Concatenate()([x, y])
merge = Bidirectional(LSTM(128,return_sequences=True))(merge)
x = Flatten()(merge)
x = Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')(x)
x = Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')(x)
output_01 = Dense(10, activation='softmax', name='output01')(x)
y = Flatten()(merge)
y = Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')(y)
y = Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')(y)
output_02 = Dense(5, activation='softmax', name='output02')(y)
Packages Version:
Tensorflow==2.5
Python==3.8
Keras==2.3
Here is the code:
# Pipe Line
(x_train, y_train), (x_test, y_test), (x_val, y_val) = (X_train, Y_train), (X_test, Y_test), (X_val, Y_val)
def model_seg():
# Convolution Layers (BatchNorm after non-linear activation)
img_input = Input(shape= (192, 256, 3))
x = Conv2D(16, (3, 3), padding='same', name='conv1')(img_input)
x = BatchNormalization(name='bn1')(x)
x = Activation('relu')(x)
x = Conv2D(32, (3, 3), padding='same', name='conv2')(x)
x = BatchNormalization(name='bn2')(x)
x = Activation('relu')(x)
x = MaxPooling2D()(x)
x = Conv2D(64, (3, 3), padding='same', name='conv3')(x)
x = BatchNormalization(name='bn3')(x)
x = Activation('relu')(x)
x = Conv2D(64, (3, 3), padding='same', name='conv4')(x)
x = BatchNormalization(name='bn4')(x)
x = Activation('relu')(x)
x = MaxPooling2D()(x)
x = Conv2D(128, (3, 3), padding='same', name='conv5')(x)
x = BatchNormalization(name='bn5')(x)
x = Activation('relu')(x)
x = Conv2D(128, (4, 4), padding='same', name='conv6')(x)
x = BatchNormalization(name='bn6')(x)
x = Activation('relu')(x)
x = MaxPooling2D()(x)
x = Conv2D(256, (3, 3), padding='same', name='conv7')(x)
x = BatchNormalization(name='bn7')(x)
x = Dropout(0.5)(x)
x = Activation('relu')(x)
x = Conv2D(256, (3, 3), padding='same', name='conv8')(x)
x = BatchNormalization(name='bn8')(x)
x = Activation('relu')(x)
x = MaxPooling2D()(x)
x = Conv2D(512, (3, 3), padding='same', name='conv9')(x)
x = BatchNormalization(name='bn9')(x)
x = Activation('relu')(x)
x = Dense(1024, activation = 'relu', name='fc1')(x)
x = Dense(1024, activation = 'relu', name='fc2')(x)
# Deconvolution Layers (BatchNorm after non-linear activation)
x = Conv2DTranspose(256, (3, 3), padding='same', name='deconv1')(x)
x = BatchNormalization(name='bn19')(x)
x = Activation('relu')(x)
x = UpSampling2D()(x)
x = Conv2DTranspose(256, (3, 3), padding='same', name='deconv2')(x)
x = BatchNormalization(name='bn12')(x)
x = Activation('relu')(x)
x = Conv2DTranspose(128, (3, 3), padding='same', name='deconv3')(x)
x = BatchNormalization(name='bn13')(x)
x = Activation('relu')(x)
x = UpSampling2D()(x)
x = Conv2DTranspose(128, (4, 4), padding='same', name='deconv4')(x)
x = BatchNormalization(name='bn14')(x)
x = Activation('relu')(x)
x = Conv2DTranspose(128, (3, 3), padding='same', name='deconv5')(x)
x = BatchNormalization(name='bn15')(x)
x = Activation('relu')(x)
x = UpSampling2D()(x)
x = Conv2DTranspose(64, (3, 3), padding='same', name='deconv6')(x)
x = BatchNormalization(name='bn16')(x)
x = Activation('relu')(x)
x = Conv2DTranspose(32, (3, 3), padding='same', name='deconv7')(x)
x = BatchNormalization(name='bn20')(x)
x = Activation('relu')(x)
x = UpSampling2D()(x)
x = Conv2DTranspose(16, (3, 3), padding='same', name='deconv8')(x)
x = BatchNormalization(name='bn17')(x)
x = Dropout(0.5)(x)
x = Activation('relu')(x)
x = Conv2DTranspose(1, (3, 3), padding='same', name='deconv9')(x)
x = BatchNormalization(name='bn18')(x)
x = Activation('sigmoid')(x)
pred = Reshape((192,256))(x)
model = Model(inputs=img_input, outputs=pred)
model.compile(optimizer= Adam(lr = 0.003), loss= [jaccard_distance], metrics=[iou])
hist = model.fit(x_train, y_train, epochs= 300, batch_size= 16,validation_data=(x_test, y_test), verbose=1)
model.save("model.h5")
accuracy = model.evaluate(x=x_test,y=y_test,batch_size=16)
print("Accuracy: ",accuracy[1])
Gives me this error in type conversion and I don't know how to fix it:
return gen_math_ops.mul(x, y, name)
D:\road-damage\road-damage-detection\rdd\lib\site-packages\tensorflow\python\ops\gen_math_ops.py:6248 mul
_, _, _op, _outputs = _op_def_library._apply_op_helper(
D:\road-damage\road-damage-detection\rdd\lib\site-packages\tensorflow\python\framework\op_def_library.py:555 _apply_op_helper
raise TypeError(
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type uint8 of argument 'x'.
Somewhere in your code, there's a tensor that is floats and a tensor that is integers, and it's not sure which one the result should be.
The architecture of your network does not tell us much; it is most likely to do with the way your data is being prepared.
If it is OK to treat your X and Y as floats, try explicitly converting them to floats like this before passing them to fit:
x_train = x_train.astype(np.float)
x_test = x_test.astype(np.float)
y_train = y_train.astype(np.float)
y_test = y_test.astype(np.float)
I have a convolutional neural network, that does better job than others for my dataset. The problem is with the ZeroPadding2D I need to place to account for down/up sampling; it creates artifacts in the output (zero samples). So, How can I avoid ZeroPadding2D option without changing the network structure (Layers). I need to maintain the structure as it's (no.layers) and may change the
1- Filter
2- kernel
3- first dimension in my data (e.g 96)
4- any other options
Bellow is my CNN
input_img = Input(shape=(96, 44, 1), name='full')
x = GaussianNoise(.1)(input_img)
x = Conv2D(64, (5, 5), activation='relu', padding='same')(x)
x = AveragePooling2D((2, 2), padding='same')(x)
x = Dropout(0.1)(x)
x = Conv2D(128, (5, 5), activation='relu', padding='same')(x)
x = AveragePooling2D((2, 2), padding='same')(x)
x = Dropout(0.2)(x)
x = Conv2D(512, (5, 5), activation='relu', padding='same')(x)
encoded = AveragePooling2D((2, 2), padding='same')(x)
x = Dropout(0.2)(x)
# at this point the representation is (4, 4, 8) i.e. 128-dimensional
x = Conv2D(512, (5, 5), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Dropout(0.2)(x)
x = Conv2D(128, (5, 5), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Dropout(0.12)(x)
x = Conv2D(64, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
x = Dropout(0.12)(x)
x = ZeroPadding2D(((4, 0), (0, 0)))(x)
decoded = Conv2D(1, (5, 5), activation='tanh', padding='same',
name='out')(x)
autoencoder = Model(input_img, decoded)
I guess if you replace your Upsampling+ ZeroPadding sections with Conv2DTranspose , that might help to solve your issue.
Have a look here :
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2DTranspose
# import the necessary packages
import keras
from keras.initializers import glorot_uniform
from keras.layers import AveragePooling2D, Input, Add
from keras.models import Model
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dropout
from keras.layers.core import Dense
class SmallerVGGNet:
#staticmethod
def build(width, height, depth, classes, finalact):
X1 = Input(shape=(height, width, depth))
# # CONV => RELU => POOL
X = Conv2D(16, (3, 3), padding="same", strides=(1, 1), name="con_layer1")(X1)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = MaxPooling2D(pool_size=(3, 3), strides=(1, 1))(X)
X = Conv2D(32, (3, 3), padding="same", strides=(2, 2), name="con_layer2")(X)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = Conv2D(32, (3, 3), padding="same", strides=(1, 1), name="con_layer3")(X)
X = Activation("relu")(X)
X = BatchNormalization(axis=3)(X)
X = MaxPooling2D(pool_size=(3, 3), strides=(1, 1))(X)
# First component
X0 = Conv2D(256, (5, 5), strides=(1, 1), padding='same', kernel_initializer=glorot_uniform(seed=0))(X)
X0 = BatchNormalization(axis=3)(X0)
X0 = Activation("relu")(X0)
# (CONV => RELU) * 2 => POOL
X = Conv2D(64, (3, 3), padding="same", strides=(2, 2), name="con_layer4")(X0)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = Conv2D(64, (3, 3), padding="same", strides=(1, 1), name="con_layer5")(X)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = AveragePooling2D(pool_size=(3, 3), strides=(1, 1))(X)
# Second Component
X0 = Conv2D(512, (5, 5), strides=(1, 1), padding='valid', kernel_initializer=glorot_uniform(seed=0))(X)
X0 = BatchNormalization(axis=3)(X0)
X0 = Activation("relu")(X0)
# (CONV => RELU) * 2 => POOL
X = Conv2D(128, (3, 3), padding="same", strides=(2, 2), name="con_layer6")(X0)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = Conv2D(128, (3, 3), padding="same", strides=(1, 1), name="con_layer7")(X)
X = BatchNormalization(axis=3)(X)
X = Activation("relu")(X)
X = MaxPooling2D(pool_size=(3, 3), strides=(1, 1))(X)
# Third Component
X0 = Conv2D(1024, (7, 7), strides=(2, 2), padding='valid', kernel_initializer=glorot_uniform(seed=0))(X)
X0 = BatchNormalization(axis=3)(X0)
X0 = Dense(128, activation="relu")(X0)
X0 = Activation("relu")(X0)
X = Add()([X0])
X = Flatten()(X1)
X = BatchNormalization()(X)
X = Dropout(0.5)(X)
output = Dense(classes, activation=finalact)(X)
model = Model(inputs=[X1], outputs=output)
print(model.summary())
return model
I want to add third components last Activation function for that i created a add function to add all the X0 values. but while adding this this error occurs.
This happens while adding ADD function.
raise ValueError('A merge layer should be called '
ValueError: A merge layer should be called on a list of inputs.
Add() generally takes 2 values in list. You have only given one.
Trying to understand the autoencoders, I am looking for an algorithm of an autoencodeur and the principle of autoencoders function, mathematical formulas ..
I'm currently working on autoencoders and trying to take the encoder output the compressed data and i'm not sure if that's the good result
i'm i in the right way?
code :
nput_img = Input(shape=(64, 64, 3))
x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)
encoder_mode = Model(input_img, encoded)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
...
autoencoder.fit
...
...
encoded_imgs = autoencoder.predict(X_test)
plt.imshow(encoded_imgs[i])
is it the encoded input it must be a the characteristic and compressed data ?
Autoencoders are used to encode the main features of the input data. You can think of it as a feature extractor. The result will be blurred because there is data loss when you encode. The principle is to represent the input with less data.
Your input data is 64x64x3 = 12288 pixels. And your encoded is 8x8x64 = 4096. Which is 1/3 of the input data.
Encoded size calculation:
input_img = Input(shape=(64, 64, 3)) # 64x64x3
x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img) # 64x64x32
x = MaxPooling2D((2, 2), padding='same')(x) # 32x32x32
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x) # 32x32x64
x = MaxPooling2D((2, 2), padding='same')(x) # 16x16x64
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x) # 16x16x64
encoded = MaxPooling2D((2, 2), padding='same')(x) # 8x8x64
So you are reconstructing the original image from 33% of its data. It is quite impressive and of course there will be a little blur.