I am using keras to add layers, for example:
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(16, 16, 32)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu', padding="same"))
model.add(BatchNormalization())
model.add(layers.Dropout(0.5))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(BatchNormalization())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(10))
Now I am implementing LRN. However, the keras library does not have LRN to my limited knowledge. The old tf.nn library does have a LRN function called tf.nn.local_response_normalization.
Is it possible to mix tf.nn with keras?
Yes, tf.nn.local_response_normalization can be used in a lambda layer. See the code below:
...
model.add(BatchNormalization())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(10))
model.add(layers.Lambda(tf.nn.local_response_normalization))
...
Related
I'm following this tutorial here.
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(32, 32, 3)))
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(10, activation='softmax'))
I am trying to understand the given code which uses the CIFAR-10 dataset.
why is he using kernel_initializer='he_uniform'?
why did he choose the 128 for the dense layer?
what will happen if we add more dense layer to the code like:
model.add(Dense(512, activation='relu', kernel_initializer='he_uniform'))
is there any way to increase the accuracy of the model?
what would be a suitable dropout rate?
why is he using kernel_initializer='he_uniform'?
The weights in a layer of a neural network are initialized randomly. How though? Which distribution should they follow? he_uniform is a strategy for initializing the weights of that layer.
why did he choose the 128 for the dense layer?
This was chosen arbitrarily.
What will happen if we add more dense layer to the code like:
model.add(Dense(512, activation='relu', kernel_initializer='he_uniform'))
I assime you mean to add them where the other 128-neuron Dense layer is (there it won't break the model) The model will become deeper and have a much higher number of parameters (i.e. your model will become more complex) with whatever positives or negatives come along with this.
what would be a suitable dropout rate?
Usually you see rates in the range of [0.2, 0.5]. Higher rates reduce overfitting but might cause training to become more unstable.
Im currently working on a little CNN with the Cifar-10 Dataset. I just updatet my code a little bit here and there and now it isnt working. I cant figure out the mistake. The prediction tells me "not an number". Couldnt find a answer for my proplem. So i cant post the Question without adding a little bit more text sooooo. Idk what i should write here. A good breakfast would be nice now. Coffe and Pancakes something like that. I hope i can poste the question now.
from keras.datasets import cifar10
import numpy as np
(x_training, y_training), (x_test,y_test) = cifar10.load_data()
x_training = x_training / 255.0
x_test = x_test / 255.0
%matplotlib inline
import matplotlib.pyplot as plt
plt.imshow(x_training[3])
plt.show
from keras.models import Sequential
from keras.layers import Dense, Flatten,Conv2D , MaxPooling2D, Dropout
import tensorflow as tf
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation="relu", padding="same"))
model.add(Conv2D(32, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(1, activation="sigmoid"))
model.compile(optimizer='RMSProp', loss="binary_crossentropy", metrics=['accuracy'])
model.summary()
model.fit(x_training, y_training,batch_size=128, epochs=10, shuffle = True )
model.evaluate(x_training, y_training)
results = model.predict(x_training[1].reshape(-1, 32, 32, 3))
results
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
'dog', 'frog', 'horse', 'ship', 'truck']
max = np.max(results)
max_position = np.argmax(results)
class_name_predict = class_names[max_position]
plt.imshow(x_training[1])
plt.show
test = class_name_predict
test
plt.imshow(x_training[1])
plt.show
x = class_names[y_training[1][0]]
x
There are some mistakes in your model:
The output layer for a multiclass problem must have a dimension equal to the number of classes with a softmax activation function
The standard losses for a multiclass problem are categorical_crossentropy and sparse_categoricalcrossentropy. categorical_crossentropy can be used when your target is one-hot encoded, sparse_categoricalcrossentropy is used when you have integer encoded labels (this is your case)
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation="relu", padding="same"))
model.add(Conv2D(32, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(Conv2D(128, kernel_size=(3, 3), activation="relu", padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(len(class_names), activation="softmax"))
model.compile(optimizer='RMSProp', loss="sparse_categorical_crossentropy", metrics=['accuracy'])
model.summary()
here the running notebook
I have a classifier model that I trained using 'theano' backend. The model works properly and I got the expected classification perforamance. The tensor size is Nx3x28x112 However, I would like to use the same classifier in another file (main_file.py) which contains a GANs implementation (with'tensorflow' backend). Thereby, I want to use the same classificer in the main_file.py and to change the input size of the tensor in order to be Nx28x112x3 (that is the proper input for the tensorflow backend). While the training procedure starts the performance is not close to the one I got with 'theano' and is close to random performance. My model looks like:
def createModel():
model = Sequential()
# The first two layers with 32 filters of window size 3x3
model.add(Conv2D(28, (3, 3), padding='same', activation='relu', input_shape=(28, 112, 3)))
# or input_shape = (3, 28, 112) in case of theano backend
model.add(Conv2D(28, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(nClasses, activation='softmax'))
return model
What should I do in order to make the model perform properly? Is there any fundamental difference when the backend is changing except the order of the input tensors?
I got this error when using Keras:
Is it because input_size not larger than the filter?
If input_shape=(64,64,3))), there will be no error.
``ValueError: Negative dimension size caused by subtracting 3 from 2 for
'conv2d_24/convolution' (op: 'Conv2D') with input shapes: [?,2,2,128],
[3,3,128,128].
My code are here:
from keras import layers
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
Default layer padding is valid, which means no padding. This way dimension reduce from 32 not to 16 but to 15. You can use padding='same' instead. In this case output has the same length as the original input.
from keras import layers
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',padding='same',
input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu',padding='same'))
model.add(layers.MaxPooling2D((2, 2),padding='same'))
model.add(layers.Conv2D(128, (3, 3), activation='relu',padding='same'))
model.add(layers.MaxPooling2D((2, 2),padding='same'))
model.add(layers.Conv2D(128, (3, 3), activation='relu',padding='same'))
model.add(layers.MaxPooling2D((2, 2),padding='same'))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
Hello I am new comer in Keras. I choose keras to Implement this paper : http://mmlab.ie.cuhk.edu.hk/projects/TCDCN.html . I just change the input size to 48x48 then for the output I just need the 68 landmark coordinate. Here is my network:
def mtfl40New(size):
model = Sequential()
model.add(Conv2D(16, (5, 5), padding='valid', input_shape=(3, size, size)))
model.add(Activation('tanh'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(48, (3, 3), padding='valid'))
model.add(Activation('tanh'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), padding='valid'))
model.add(Activation('tanh'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (2, 2), padding='valid'))
model.add(Activation('tanh'))
model.add(Flatten())
model.summary()
#model.count_params()
model.add(Dense(100, kernel_initializer="normal", input_shape=(576,)))
model.add(Activation('tanh'))
model.add(Dense(136, kernel_initializer="normal"))
model.add(Activation('tanh'))
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
return model
However I get this error :
Anyone can help?
-Thank you-
This is again an incompatibility between your input shape and the format how it is interpreted. You have set in your Keras configuration the image ordering to channels first, while the input shape has the channels at the end. To fix it simply replace this line:
model.add(Conv2D(16, (5, 5), padding='valid', input_shape=(3, size, size)))
With:
model.add(Conv2D(16, (5, 5), padding='valid', input_shape=(size, size, 3)))