We Keep Coding

Can CNN do better than pretrained CNN?

With all I know. pretrained CNN can do way better than CNN. I have a dataset of 855 images. I have applied CNN and got 94% accuracy.Then I applied Pretrained model (VGG16, ResNet50, Inception_V3, MobileNet)also with fine tuning but still i got highest 60% and two of them are doing very bad on classification. Can CNN really do better than pretrained model or my implementation is wrong. I've converted my image into 100 by 100 dimensions and followed the way of keras application. Then What is the issue ??
Naive CNN approach :
def cnn_model():
size = (100,100,1)
num_cnn_layers =2
NUM_FILTERS = 32
KERNEL = (3, 3)
MAX_NEURONS = 120
model = Sequential()
for i in range(1, num_cnn_layers+1):
if i == 1:
model.add(Conv2D(NUM_FILTERS*i, KERNEL, input_shape=size,
activation='relu', padding='same'))
else:
model.add(Conv2D(NUM_FILTERS*i, KERNEL, activation='relu',
padding='same'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(int(MAX_NEURONS), activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(int(MAX_NEURONS/2), activation='relu'))
model.add(Dropout(0.4))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])
return model
VGG16 approach:
def vgg():
` `vgg_model = keras.applications.vgg16.VGG16(weights='imagenet',include_top=False,input_shape = (100,100,3))
model = Sequential()
for layer in vgg_model.layers:
model.add(layer)
# Freeze the layers
for layer in model.layers:
layer.trainable = False
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(3, activation='softmax'))
model.compile(optimizer=keras.optimizers.Adam(lr=1e-5),
loss='categorical_crossentropy',
metrics=['accuracy'])
return model

What you're referring to as CNN in both cases talk about the same thing, which is a type of a neural network model. It's just that the pre-trained model has been trained on some other data instead of the dataset you're working on and trying to classify.
What is usually used here is called Transfer Learning. Instead of freezing all the layers, trying leaving the last few layers open so they can be retrained with your own data, so that the pretrained model can edit its weights and biases to match your needs as well. It could be the case that the dataset you're trying to classify is foreign to the pretrained models.
Here's an example from my own work, there are additional pieces of code but you can make it work with your own code, the logic remains the same
#You extract the layer which you want to manipulate, usually the last few.
last_layer = pre_trained_model.get_layer(name_of_layer)
# Flatten the output layer to 1 dimension
x = layers.Flatten()(last_output)
# Add a fully connected layer with 1,024 hidden units and ReLU activation
x = layers.Dense(1024,activation='relu')(x)
# Add a dropout rate of 0.2
x = layers.Dropout(0.2)(x)
# Add a final sigmoid layer for classification
x = layers.Dense(1,activation='sigmoid')(x)
#Here we combine your newly added layers and the pre-trained model.
model = Model( pre_trained_model.input, x)
model.compile(optimizer = RMSprop(lr=0.0001),
loss = 'binary_crossentropy',
metrics = ['accuracy'])

Adding to what #Ilknur Mustafa mentioned, as your dataset may be foreign to the images used for pre-training, you can try to re-train few last layers of the pre-trained model instead of adding a whole new layers. The below example code doesn't add any additional trainable layer other than the output layer. In this way, you can benefit by retraining the last few layers on the existing weights, rather than training from scratch. This may be beneficial if you don't have a large dataset to train on.
# load model without classifier layers
vgg = VGG16(include_top=False, input_shape=(100, 100, 3), weights='imagenet', pooling='avg')
# make only last 2 conv layers trainable
for layer in vgg.layers[:-4]:
layer.trainable = False
# add output layer
out_layer = Dense(3, activation='softmax')(vgg.layers[-1].output)
model_pre_vgg = Model(vgg.input, out_layer)
# compile model
opt = SGD(lr=1e-5)
model_pre_vgg.compile(optimizer=opt, loss=keras.losses.categorical_crossentropy, metrics=['accuracy'])

#You extract the layer which you want to manipulate, usually the last few.
last_layer = pre_trained_model.get_layer(name_of_layer)
# Flatten the output layer to 1 dimension
x = layers.Flatten()(last_output)
# Add a fully connected layer with 1,024 hidden units and ReLU activation
x = layers.Dense(1024,activation='relu')(x)
# Add a dropout rate of 0.2
x = layers.Dropout(0.2)(x)
# Add a final sigmoid layer for classification
x = layers.Dense(1,activation='sigmoid')(x)
#Here we combine your newly added layers and the pre-trained model.
model = Model( pre_trained_model.input, x)
model.compile(optimizer = RMSprop(lr=0.0001),
loss = 'binary_crossentropy',
metrics = ['accuracy'])

How can I implement a 1D CNN in front of my LSTM network

At the moment I reshape my X_train like this:
X_train = input.reshape(1,1,12)
model = Sequential()
model.add(LSTM(100,input_shape=(1, 12)))
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(9, activation='sigmoid'))
But now I am thinking of implementing a 1D CNN in front of this LSTM layer. Does anybody know how this should be done?

You have a keras.layers.Conv1D (see doc) that you can applay to your network input.
If you input is of shape (1,1,12) and you apply K filters, you'll get an output of shape (1,1,K): so you might want to swap it to (1,12,1) to place the steps in 2nd position (check doc).
Note that model.summary() might help you debug the input and output shapes of your network.

My Keras CNN return the same output value, how can I fix/improve my code

I've created a code to train my CNN which will determine if a image be one or another class ("pdr or nonPdr")
This is my keras model:
model = Sequential()
model.add(Conv2D(input_shape=(605,700,3), filters=64, kernel_size=(3,3), padding="same",activation="tanh"))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(32, activation='tanh', input_dim=100))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
data, labels = ReadImages(TRAIN_DIR)
model.fit(np.array(data), np.array(labels), epochs=10, batch_size=16)
model.save('model.h5')
And this is my test_predict file:
model = load_model('model.h5')
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
test_image = cv2.imread("test_data/im0203.ppm")
test_image = test_image.reshape(1,605,700,3).astype('float')
test_image = np.array(test_image)
test_image2 = cv2.imread("test_data/im0001.ppm")
test_image2 = test_image2.reshape(1,605,700,3).astype('float')
#predict the result
print(model.predict(test_image))
print(model.predict(test_image2))
After run the code below I get the same value for my 2 images (that are different, one is pdr and the other is nonPdr)
[[0.033681]]
[[0.033681]]
How can I fix it and improve my CNN. I appreciate your help.
Update
I tried to remove one dense layer and change the last one in a (2, activation='sigmoid') but it not works too.. I really don't have any ideia what to do**

This could be because of multiple reasons. Most plausible ones are:
The input fed to network and the image fed during prediction may not have the same preprocessing steps.
The model is over-fitted and when it finds a little different prediction image, it tends to bias its decision based on its training.
Model is not good
To avoid such complications, possible future solutions could be:
1. Try feeding a normalized or scaled image only during training and testing.
Do not use too many Dense layers (Especially for image classification) because dense layers are good at learning spatial information and for an image to classify correctly, the model needs to look the image in 2d matrix rather than 1d spatial matrix (since dense layers flatten the 2d matrix to 1d matrix).
Even if you are trying to use a dense layer, I would recommend using a dense layer only at the last layer, it, for the final number_of_classes layers.
A better architecture would be
[ Conv followed by few transition blocks and bottleneck layers ] x 2
flatten
Dense(number_of_classes)

how to make a complicated output of a neural network keras

I would like to combine 2 neural networks which are showing probabilities of classes.
One says that it is a cat on the image.
The second says that the cat has a collar.
How to use softmax activation function on the output of the neural network?
Please, see the picture to understand the main idea:

You can use the functional API to create a multi-output network. Essentially every output will be a separate prediction. Something along the lines of:
in = Input(shape=(w,h,c)) # image input
latent = Conv...(...)(in) # some convolutional layers to extract features
# How share the underlying features to predict
animal = Dense(2, activation='softmax')(latent)
collar = Dense(2, activation='softmax')(latent)
model = Model(in, [animal, coller])
model.compile(loss='categorical_crossentropy', optimiser='adam')
You can have as many separate outputs you like. If you have only binary features you can have a single vector output as well, Dense(2, activation='sigmoid') and first entry could predict cat or not, while second whether it has a collar. This would be multi-class multi-label setup.

Juste create two separate dense layers (with sofmax activation) at the end of your model, e.g.:
from keras.layers import Input, Dense, Conv2D
from keras.models import Model
# Input example:
inputs = Input(shape=(64, 64, 3))
# Example of model:
x = Conv2D(16, (3, 3), padding='same')(inputs)
x = Dense(512, activation='relu')(x)
x = Dense(64, activation='relu')(x)
# ... (replace with your actual layers)
# Then add two separate layers taking the previous output and generating two estimations:
cat_predictions = Dense(2, activation='softmax')(x)
collar_predictions = Dense(2, activation='softmax')(x)
model = Model(inputs=inputs, outputs=[cat_predictions, collar_predictions])

Merge two different deep learning models in Keras

I have two different types of data (image volumes and coordinates) and I would like to use a convolutional neural network on the image volume data and then after this I would like to append some additional information (ie. the coordinates of the volume).
Independently this should create a pretty solid predictor for my function. How can I implement this using Keras.
The only answers I have found online are either ambiguous or are using the deprecated methods which I have got to work. But I would really like to implement this using the current API that way I can more easily save the model for later use.
model = Sequential()
model.add(Conv3D(32, kernel_size=(3, 3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv3D(64, (3, 3, 3), activation='relu'))
model.add(MaxPooling3D(pool_size=(2, 2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
print(model.output_shape)
# The additional data (the coordinates x,y,z)
extra = Sequential()
extra.add(Activation('sigmoid', input_shape=(3,)))
print(extra.output_shape)
merged = Concatenate([model, extra])
# New model should encompass the outputs of the convolutional network and the coordinates that have been merged.
# But how?
new_model = Sequential()
new_model.add(Dense(128, activation='relu'))
new_model.add(Dropout(0.8))
new_model.add(Dense(32, activation='sigmoid'))
new_model.add(Dense(num_classes, activation='softmax'))
new_model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])

Sequential models are not suited for creating models with branches.
You can have the two independent models as Sequential models, as you did, but from the Concatenate on, you should start using the functional Model API.
The idea is to get the output tensors of the two models and feed them in other layers to get new output tensors.
So, considering you have model and extra:
mergedOutput = Concatenate()([model.output, extra.output])
This mergetOutput is a tensor. You can either create the last part of the model using this tensor, or create the last part independently, and call it on this tensor. The second approach may be good if you want to train each model separately (doesn't seem to be your case).
Now, creating the new model as a functional API model:
out = Dense(128, activation='relu')(mergetOutput)
out = Dropout(0.8)(out)
out = Dense(32, activation='sigmoid')(out)
out = Dense(num_classes, activation='softmax')(out)
new_model = Model(
[model.input, extra.input], #model with two input tensors
out #and one output tensor
)
An easier approach is to take all three models you have already created and use them to create a combined model:
model = Sequential() #your first model
extra = Sequential() #your second model
new_model = Sequential() #all these three exactly as you did
#in this case, you just need to add an input shape to new_model, compatible with the concatenated output of the previous models.
new_model.add(FirstNewModelLayer(...,input_shape=(someValue,)))
Join them like this:
mergedOutput = Concatenate()([model.output, extra.output])
finalOutput = new_model(mergedOutput)
fullModel = Model([model.input,extra.input],finalOutput)

Use the functional API of Keras (https://keras.io/models/model/). You can just apply layers to your merged layer in Keras. The functional API works like this. You have a tensor and you apply a function to this Tensor. Then this is recursively evaluated. Because pretty much everything is a tensor in Keras this works quite nicely.
An example for this is:
activation = Dense(128, activation='relu')(merged)