I'm using Keras to build a a convolutional neural network which takes images with the shape (200, 200, 3) and should output a number representing the predicted amount of likes this image would get on social media. y_train and y_test are dimensional vectors. Here is my code so far:
model = Sequential()
model.add(Conv2D(10, 1, 1, input_shape=(200, 200, 3)))
model.add(Activation('relu'))
model.add(Conv2D(10, 1, 1))
model.add(Activation('relu'))
model.add(MaxPool2D())
# model.add(Flatten())
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train, y_train)
score = model.evaluate(x_test, y_test)
I'm getting an error saying
expected activation_2 to have 4 dimensions, but got array with shape
(7, 1)
Where (7, 1) is the shape of y_train. Why is this happening?
Since you are using a fixed size input images, you can add fully connected layers
Categorical cross entropy is a function to compare probabilistic distributions (i.e.for classification problems not regression), you can use Mean Squared Error loss function instead.
Here is a sample code:
model = Sequential()
model.add(Conv2D(10, 1, 1, input_shape=(200, 200, 3)))
model.add(Activation('relu'))
model.add(Conv2D(10, 1, 1))
model.add(Activation('relu'))
model.add(MaxPooling2D())
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(1))
model.compile(loss='mse',optimizer='rmsprop')
model.fit(x_train, y_train)
test_mse = model.evaluate(x_test, y_test)
print('test set mse is {}'.format(test_mse))
Related
Being new to Keras sequential models is causing me a few troubles!
I have an x_train of shape : 17755 x 500 x 12
and y_train of shape: 17755 x 15 (labels are already one-hot encoded)
And I made the next model to be trained on this data:
model = Sequential()
model.add(Conv2D(32,3,padding="same", activation="relu", input_shape=(17755,500,12)))
model.add(MaxPool2D())
model.add(Conv2D(32, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(128,activation="relu"))
model.add(Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)
1- when I don’t use np.expand_dims to add an axis for batch, I get this error:
ValueError: Input 0 of layer "sequential" is incompatible with the
layer: expected shape=(None, 17755, 500, 12), found shape=(None, 500,
12)
2- when I do use np.expand_dims and the shape of x_train became: 1x17755x500x12
I get this error:
Data cardinality is ambiguous:
x sizes: 1
y sizes: 17755
Make sure all arrays contain the same number of samples.
3- when I use np.expand_dims for y_train too and its shape became: 1x17755x15
I get this error:
ValueError: Shapes (None, 17755, 15) and (None, 15) are incompatible
I know I’m doing something fundamentally wrong, but what what is that? Can anyone please help me out with the shape of data please?
Regarding x_train try adding a new dimension at the end to represent the channel dimension needed for Conv2D layers. Note also that you do not provide the number of samples to your input shape. Here is a working example:
import tensorflow as tf
import numpy as np
x_train = np.random.random((17755,500,12))
x_train = np.expand_dims(x_train, axis=-1)
y_train = np.random.random((17755,15))
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32,3,padding="same", activation="relu", input_shape=(500, 12, 1)))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(32, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Dropout(0.4))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128,activation="relu"))
model.add(tf.keras.layers.Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)
I'm quite new to CNN.
I'm trying to create a the following model. but I get the following error: "ValueError: logits and labels must have the same shape ((1, 7, 7, 2) vs (1, 2))"
Below the code I'm trying to implement
#create the training data set
train_data=scaled_data[0:training_data_len,:]
#define the number of periods
n_periods=28
#split the data into x_train and y_train data set
x_train=[]
y_train=[]
for i in range(n_periods,len(train_data)):
x_train.append(train_data[i-n_periods:i,:28])
y_train.append(train_data[i,29])
x_train=np.array(x_train)
y_train=np.array(y_train)
#Reshape the train data
x_train=x_train.reshape(x_train.shape[0],x_train.shape[1],x_train.shape[2],1)
x_train.shape
y_train = keras.utils.to_categorical(y_train,2)
# x_train as the folllowing shape (3561, 28, 28, 1)
# y_train as the following shape (3561, 2, 2)
#Build the 2 D CNN model for regression
model= Sequential()
model.add(Conv2D(32,kernel_size=(3,3),padding='same',activation='relu',input_shape=(x_train.shape[1],x_train.shape[2],1)))
model.add(Conv2D(64,kernel_size=(3,3),padding='same',activation='relu'))
model.add(MaxPooling2D(pool_size=(4,4)))
model.add(Dropout(0.25))
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='sigmoid'))
model.add(Dense(2, activation='sigmoid'))
model.summary()
#compile the model
model.compile(optimizer='ADADELTA', loss='binary_crossentropy', metrics=['accuracy'])
#train the model
model.fit(x_train, y_train, batch_size=1, epochs=1, verbose=2)
There are two problems in your approach:
You're using Convolutional/MaxPooling layers in which the inputs/outputs are as matrices, i.e., with the shape of (Batch_Size, Height, Width, Depth). You then add some Dense layers which usually expect vectors, not matrices as inputs. Therefore, you have to first flatten the outputs of MaxPooling before giving it to Dense layer, i.e., add a model.add(Flatten()) after model.add(Dropout(0.25)) and before model.add(Dense(128,activation='relu')).
You are doing binary classification, i.e., you have two classes. You are using binary_crossentropy as the loss function, for this to work, you should keep your targets as they are (0 and 1) and not use y_train = keras.utils.to_categorical(y_train,2). Your final layer should have 1 neuron and not 2 (Change model.add(Dense(2, activation='sigmoid')) into model.add(Dense(1, activation='sigmoid')) )
I have built up a NN with following architecture:
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=0)
print(X_train.shape, X_test.shape, Y_train.shape, Y_test.shape)
(1901, 456, 3) (476, 456, 3) (1901, 3, 3) (476, 3, 3)
model = Sequential()
model.add(Flatten(input_shape=(456,3)))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(3 * 3))
model.add(Reshape((3, 3)))
model.compile('adam', 'mse')
history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=100)
Now I want to replace this architecture with a analogue CNN which does the same; but when trying to implement this I always get problems with the dimensions of the different layers. And my error is always like this
ValueError: Error when checking input: expected conv2d_3_input to have 4 dimensions, but got array with shape (x, x, x)
the dataset remains the same, just the NN architecture changes and this is my first approach:
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(1901,456,3)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))
Can someone help me out to replace my first NN into a CNN?
Your network is well defined, the error you're getting is during the fit operation. And why is that the case.
Well Conv2D is looking for data with 4D shape as you can see here : doc
X_train shape must then be (samples, channels, rows, cols)
When you gave input_shape=(1901,456,3), you didn't have to specify the number of samples.
But during the fit operation you need to have a data shaped as (samples, channels, rows, cols) .
And now you see that you have a problem. Why is your X_train shaped like that, it seems that you only have one image. You can feed it by reshaping it using :
X_train = X_train.reshape((1, 1901, 456, 3))
But that seems odd, you're only feeding one image to your network.
Edit : after clarification on the comments, conv1D will be better in this type of case, here is how to do it:
model = Sequential()
model.add(Conv1D(32, kernel_size=3,
activation='relu',
input_shape=(456,3)))
model.add(Conv1D(64, 3, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3 * 3, activation='softmax'))
model.add(Reshape((3, 3))
now everything worked with the architecture and there is also no problem when compiling the NN;
batch_size = 128
epochs = 12
model.compile(
optimizer='rmsprop',
loss=tf.keras.losses.MeanSquaredError(),
metrics=['mse'],
)
model.fit(X_test, Y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(X_test, Y_test))
score = model.evaluate(X_test, Y_test, verbose=0)
but when trying to fit I get following error :
ValueError: Input arrays should have the same number of samples as target
arrays. Found 476 input samples and 1901 target samples.
what am I missing here?
I am learning Keras using audio classification, Actually, I am implementing the code with modification from https://github.com/deepsound-project/genre-recognition/blob/master/train_model.py using Keras.
The shape of the dataset is
X_train shape = (800, 32, 1)
y_train shape = (800, 10)
X_test shape = (200, 32, 1)
y_test shape = (200, 10)
The model
model = Sequential()
model.add(Conv1D(filters=256, kernel_size=5, input_shape=(32,1), activation="relu"))
model.add(BatchNormalization(momentum=0.9))
model.add(MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(Conv1D(filters=256, kernel_size=5, activation="relu"))
model.add(BatchNormalization(momentum=0.9))
model.add(MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation="relu", ))
model.add(Dense(10, activation='softmax'))
model.compile(
loss='categorical_crossentropy',
optimizer = Adam(lr=0.001),
metrics = ['accuracy'],
)
model.summary()
red_lr= ReduceLROnPlateau(monitor='val_loss',patience=2,verbose=2,factor=0.5,min_delta=0.01)
check=ModelCheckpoint(filepath=r'/content/drive/My Drive/Colab Notebooks/gen/cnn.hdf5', verbose=1, save_best_only = True)
History = model.fit(X_train,
y_train,
epochs=100,
#batch_size=512,
validation_data = (X_test, y_test),
verbose = 2,
callbacks=[check, red_lr],
shuffle=True )
The accuracy graph
Loss graph
I do not understand, Why the val_acc is in the range of 70%. I tried to modify the model architecture including optimizer, but no improvement.
And, Is it good to have a lot of difference between loss and val_loss.
how to improve the accuracy above 80... any help...
Thank you
I found it, I use concatenate function from Keras to concatenate all convolution layers and, it gives the best performance.
I was trying to run a 10 fold cross validation on my dataset.I had reshaped my data before training as follows
data = data.reshape(500,1,1028,1)
data_y = np_utils.to_categorical(data_y, 3)
After this i described my model
for train,test in kf.split(data):
fold+=1
print("Fold #{}".format(fold))
x_train = data[train]
y_train = data_y[train]
x_test = data[test]
y_test = data_y[test]
print(x_train.shape)
model.add(Conv2D(32, (1, 3),input_shape=(1,1028,1)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
#model.add(MaxPooling2D(pool_size=(1,2)))
model.add(Conv2D(34, (1, 4)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(1,2)))
model.add(Conv2D(64,(1, 3)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
#model.add(MaxPooling2D(pool_size=(1,2)))
model.add(Conv2D(64, (1, 4)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(1,2)))
model.add(Flatten())
#fully connected for new model
model.add(Dense(550))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.25))
model.add(Dense(250))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.25))
model.add(Dense(100))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.25))
model.add(Dense(25))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.25))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer=Adam(), metrics=['accuracy'])
model.fit(x_train.reshape(450,1,1028,1), y_train,
batch_size=5,
epochs=1,
verbose=1,
validation_data=(x_test, y_test))
pred = model.predict(x_test)
oos_y.append(y_test)
pred = np.argmax(pred, axis=1) # raw probabilities to chosen class (highest probability)
oos_pred.append(pred)
# Measure this fold's accuracy
y_compare = np.argmax(y_test, axis=1) # For accuracy calculation
score = metrics.accuracy_score(y_compare, pred)
print("Fold score (accuracy): {}".format(score))
The problem is that when I run my code the code runs properly for fold 1 but for fold 2 it gives me the following error
ValueError: Input 0 is incompatible with layer conv2d_5: expected ndim=4, found ndim=2
When I checked the dimensions of x_train it was (450, 1, 1028, 1)
I am not sure what the error is.
You are adding model layers inside loop over and over. The error was produced when you try to add convolutional layer(for second iteration of loop) after softmax activation layer(the last layer for first iteration of the loop). After careful inspection I've come to the following solution to your question.
First split the dataset into train and test
for train_index, test_index in kf.split(data):
X_train, X_test = data[train_index], data[test_index]
y_train, y_test = data_y[train_index], data_y[test_index]
then add layers to model outside of the loop.
model.add(Conv2D(32, (1, 3),input_shape=(1,1028,1)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
#model.add(MaxPooling2D(pool_size=(1,2)))
model.add(Conv2D(34, (1, 4)))
model.add(BatchNormalization(axis=-1))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(1,2)))
# ... The reset of the code