I want to make a neural network for a classification problem. The training set takes 225 input of 48 dimensions and has a validation set of 50. the code is:
#building model
def build_model():
model = Sequential()
model.add(Dense(128, activation = "relu"))
model.add(Dropout(0.5))
model.add(Dense(64, activation = "relu"))
model.add(Dropout(0.2))
model.add(Dense(32, activation = "relu"))
model.add(Dropout(0.1))
model.add(Dense(16, activation = "softmax"))
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=0.00001),
loss=['mean_squared_error'],
metrics=['accuracy']
)
return model
model = build_model()
#train model
history = model.fit(
x_train,
y_train,
epochs=10,
batch_size=32,
validation_data=(
x_val,
y_val
),
callbacks=[ProgbarLogger(count_mode='steps',stateful_metrics=None)
]
)
but i get as output weird logs such as :
Train on 225 samples, validate on 50 samples
Epoch 1/10
Epoch 1/10
8/8 [==============================] - 2s 247ms/step - loss: 0.6517 - accuracy: 0.0178 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 2s 9ms/sample - loss: 0.7298 - accuracy: 0.0178 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 2/10
Epoch 2/10
8/8 [==============================] - 0s 10ms/step - loss: 0.6516 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 359us/sample - loss: 0.7297 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 3/10
Epoch 3/10
8/8 [==============================] - 0s 9ms/step - loss: 0.6517 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 341us/sample - loss: 0.7297 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 4/10
Epoch 4/10
8/8 [==============================] - 0s 10ms/step - loss: 0.6515 - accuracy: 0.0178 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 346us/sample - loss: 0.7296 - accuracy: 0.0178 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 5/10
Epoch 5/10
8/8 [==============================] - 0s 9ms/step - loss: 0.7457 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 319us/sample - loss: 0.7295 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 6/10
Epoch 6/10
8/8 [==============================] - 0s 10ms/step - loss: 1.0552 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 359us/sample - loss: 0.7296 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 7/10
Epoch 7/10
8/8 [==============================] - 0s 9ms/step - loss: 0.6514 - accuracy: 0.0089 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 328us/sample - loss: 0.7295 - accuracy: 0.0089 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 8/10
Epoch 8/10
8/8 [==============================] - 0s 9ms/step - loss: 0.6516 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 324us/sample - loss: 0.7295 - accuracy: 0.0133 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 9/10
Epoch 9/10
8/8 [==============================] - 0s 10ms/step - loss: 0.6515 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 355us/sample - loss: 0.7296 - accuracy: 0.0222 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
Epoch 10/10
Epoch 10/10
8/8 [==============================] - 0s 9ms/step - loss: 0.6515 - accuracy: 0.0267 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
225/225 [==============================] - 0s 328us/sample - loss: 0.7295 - accuracy: 0.0267 - val_loss: 0.2842 - val_accuracy: 0.0000e+00
can you help me understand this behavior of the losses and the excess of accuracy. shouldnt it be lower?
Related
I'm new to this technology, so I was trying to build a model on image dataset.
I have used this architecture -
model = keras.Sequential()
model.add(layers.Conv2D(filters=6, kernel_size=(3, 3), activation='relu', input_shape=(32,32,1)))
model.add(layers.AveragePooling2D())
model.add(layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu'))
model.add(layers.AveragePooling2D())
model.add(layers.Flatten())
model.add(layers.Dense(units=120, activation='relu'))
model.add(layers.Dense(units=84, activation='relu'))
model.add(layers.Dense(units=1, activation = 'sigmoid'))
The accuracy and loss seems pretty good but not the validation accuracy -
Epoch 1/50
10/10 [==============================] - 17s 2s/step - loss: 20.8554 - accuracy: 0.5170 -
val_loss: 0.8757 - val_accuracy: 0.5946
Epoch 2/50
10/10 [==============================] - 14s 1s/step - loss: 1.5565 - accuracy: 0.5612 -
val_loss: 0.8725 - val_accuracy: 0.5811
Epoch 3/50
10/10 [==============================] - 14s 1s/step - loss: 0.8374 - accuracy: 0.6293 -
val_loss: 0.8483 - val_accuracy: 0.5405
Epoch 4/50
10/10 [==============================] - 14s 1s/step - loss: 1.0340 - accuracy: 0.5748 -
val_loss: 1.6252 - val_accuracy: 0.5135
Epoch 5/50
10/10 [==============================] - 14s 1s/step - loss: 1.1054 - accuracy: 0.5816 -
val_loss: 0.7324 - val_accuracy: 0.6486
Epoch 6/50
10/10 [==============================] - 15s 1s/step - loss: 0.5942 - accuracy: 0.7041 -
val_loss: 0.7412 - val_accuracy: 0.6351
Epoch 7/50
10/10 [==============================] - 15s 2s/step - loss: 0.6041 - accuracy: 0.6939 -
val_loss: 0.6918 - val_accuracy: 0.6622
Epoch 8/50
10/10 [==============================] - 14s 1s/step - loss: 0.4944 - accuracy: 0.7687 -
val_loss: 0.7083 - val_accuracy: 0.6216
Epoch 9/50
10/10 [==============================] - 14s 1s/step - loss: 0.5231 - accuracy: 0.7007 -
val_loss: 1.0332 - val_accuracy: 0.5270
Epoch 10/50
10/10 [==============================] - 14s 1s/step - loss: 0.5133 - accuracy: 0.7313 -
val_loss: 0.6859 - val_accuracy: 0.5811
Epoch 11/50
10/10 [==============================] - 14s 1s/step - loss: 0.6177 - accuracy: 0.6735 -
val_loss: 1.0781 - val_accuracy: 0.5135
Epoch 12/50
10/10 [==============================] - 14s 1s/step - loss: 0.9852 - accuracy: 0.6701 -
val_loss: 3.0853 - val_accuracy: 0.4865
Epoch 13/50
10/10 [==============================] - 13s 1s/step - loss: 1.0099 - accuracy: 0.6259 -
val_loss: 1.8193 - val_accuracy: 0.5000
Epoch 14/50
10/10 [==============================] - 13s 1s/step - loss: 0.7179 - accuracy: 0.7041 -
val_loss: 1.5659 - val_accuracy: 0.5135
Epoch 15/50
10/10 [==============================] - 14s 1s/step - loss: 0.4575 - accuracy: 0.7857 -
val_loss: 0.6865 - val_accuracy: 0.5946
Epoch 16/50
10/10 [==============================] - 14s 1s/step - loss: 0.6540 - accuracy: 0.7177 -
val_loss: 1.7108 - val_accuracy: 0.5405
Epoch 17/50
10/10 [==============================] - 13s 1s/step - loss: 1.3617 - accuracy: 0.6156 -
val_loss: 1.1215 - val_accuracy: 0.5811
Epoch 18/50
10/10 [==============================] - 14s 1s/step - loss: 0.6983 - accuracy: 0.7245 -
val_loss: 2.1121 - val_accuracy: 0.5135
Epoch 19/50
10/10 [==============================] - 15s 1s/step - loss: 0.6669 - accuracy: 0.7415 -
val_loss: 0.8061 - val_accuracy: 0.6216
Epoch 20/50
10/10 [==============================] - 14s 1s/step - loss: 0.3853 - accuracy: 0.8129 -
val_loss: 0.7368 - val_accuracy: 0.6757
Epoch 21/50
10/10 [==============================] - 13s 1s/step - loss: 0.5672 - accuracy: 0.7347 -
val_loss: 1.4207 - val_accuracy: 0.5270
Epoch 22/50
10/10 [==============================] - 14s 1s/step - loss: 0.4770 - accuracy: 0.7551 -
val_loss: 1.6060 - val_accuracy: 0.5135
Epoch 23/50
10/10 [==============================] - 14s 1s/step - loss: 0.7212 - accuracy: 0.7041 -
val_loss: 1.1835 - val_accuracy: 0.5811
Epoch 24/50
10/10 [==============================] - 14s 1s/step - loss: 0.5231 - accuracy: 0.7483 -
val_loss: 0.6802 - val_accuracy: 0.7027
Epoch 25/50
10/10 [==============================] - 13s 1s/step - loss: 0.3185 - accuracy: 0.8367 -
val_loss: 0.6644 - val_accuracy: 0.7027
Epoch 26/50
10/10 [==============================] - 14s 1s/step - loss: 0.2500 - accuracy: 0.8912 -
val_loss: 0.8569 - val_accuracy: 0.6486
Epoch 27/50
10/10 [==============================] - 14s 1s/step - loss: 0.2279 - accuracy: 0.9082 -
val_loss: 0.7515 - val_accuracy: 0.7162
Epoch 28/50
10/10 [==============================] - 14s 1s/step - loss: 0.2349 - accuracy: 0.9082 -
val_loss: 0.9439 - val_accuracy: 0.5811
Epoch 29/50
10/10 [==============================] - 13s 1s/step - loss: 0.2051 - accuracy: 0.9184 -
val_loss: 0.7895 - val_accuracy: 0.7027
Epoch 30/50
10/10 [==============================] - 14s 1s/step - loss: 0.1236 - accuracy: 0.9592 -
val_loss: 0.7387 - val_accuracy: 0.7297
Epoch 31/50
10/10 [==============================] - 14s 1s/step - loss: 0.1370 - accuracy: 0.9524 -
val_loss: 0.7387 - val_accuracy: 0.7297
Epoch 32/50
10/10 [==============================] - 14s 1s/step - loss: 0.0980 - accuracy: 0.9796 -
val_loss: 0.6901 - val_accuracy: 0.7162
Epoch 33/50
10/10 [==============================] - 14s 1s/step - loss: 0.0989 - accuracy: 0.9762 -
val_loss: 0.7754 - val_accuracy: 0.7162
Epoch 34/50
10/10 [==============================] - 14s 1s/step - loss: 0.1195 - accuracy: 0.9592 -
val_loss: 0.6639 - val_accuracy: 0.6622
Epoch 35/50
10/10 [==============================] - 14s 1s/step - loss: 0.0805 - accuracy: 0.9898 -
val_loss: 0.7666 - val_accuracy: 0.7162
Epoch 36/50
10/10 [==============================] - 14s 1s/step - loss: 0.0649 - accuracy: 0.9966 -
val_loss: 0.7543 - val_accuracy: 0.7162
Epoch 37/50
10/10 [==============================] - 14s 1s/step - loss: 0.0604 - accuracy: 0.9898 -
val_loss: 0.7472 - val_accuracy: 0.7297
Epoch 38/50
10/10 [==============================] - 14s 1s/step - loss: 0.0538 - accuracy: 1.0000 -
val_loss: 0.7287 - val_accuracy: 0.7432
Epoch 39/50
10/10 [==============================] - 13s 1s/step - loss: 0.0430 - accuracy: 0.9966 -
val_loss: 0.8989 - val_accuracy: 0.6622
Epoch 40/50
10/10 [==============================] - 14s 1s/step - loss: 0.0386 - accuracy: 1.0000 -
val_loss: 0.6951 - val_accuracy: 0.6892
Epoch 41/50
10/10 [==============================] - 13s 1s/step - loss: 0.0379 - accuracy: 1.0000 -
val_loss: 0.8485 - val_accuracy: 0.6892
Epoch 42/50
10/10 [==============================] - 14s 1s/step - loss: 0.0276 - accuracy: 1.0000 -
val_loss: 0.9726 - val_accuracy: 0.6486
Epoch 43/50
10/10 [==============================] - 13s 1s/step - loss: 0.0329 - accuracy: 1.0000 -
val_loss: 0.7336 - val_accuracy: 0.7568
Epoch 44/50
10/10 [==============================] - 14s 1s/step - loss: 0.0226 - accuracy: 1.0000 -
val_loss: 0.8846 - val_accuracy: 0.6892
Epoch 45/50
10/10 [==============================] - 13s 1s/step - loss: 0.0249 - accuracy: 1.0000 -
val_loss: 0.9542 - val_accuracy: 0.6892
Epoch 46/50
10/10 [==============================] - 14s 1s/step - loss: 0.0171 - accuracy: 1.0000 -
val_loss: 0.8792 - val_accuracy: 0.6892
Epoch 47/50
10/10 [==============================] - 15s 1s/step - loss: 0.0122 - accuracy: 1.0000 -
val_loss: 0.8564 - val_accuracy: 0.7162
Epoch 48/50
10/10 [==============================] - 13s 1s/step - loss: 0.0114 - accuracy: 1.0000 -
val_loss: 0.8900 - val_accuracy: 0.7027
Epoch 49/50
10/10 [==============================] - 13s 1s/step - loss: 0.0084 - accuracy: 1.0000 -
val_loss: 0.8981 - val_accuracy: 0.7027
I tried changing the parameters too yet no result. Would be helpful if I can get to know what's wrong with the val_accuracy. Thanks in advance.
You are using less dataset specially test dataset for validation. Try adding some more data to train the model and for validation, then you can see the difference in val_accuracy. You can also try by adding more layers to the model.
There are some other methods available like, data augmentation, dropout, regularizers to increase the accuracy of the model by avoiding overfitting problem.
Please follow this reference to overcome the overfitting problem and to best train your model.
I am trying to run an autoencoder for dimensionality reduction on a Fraud Detection dataset (https://www.kaggle.com/kartik2112/fraud-detection?select=fraudTest.csv) and am receiving very high loss values for each iteration. Below is the autoencoder code.
nb_epoch = 100
batch_size = 128
input_dim = X_train.shape[1]
encoding_dim = 14
hidden_dim = int(encoding_dim / 2)
learning_rate = 1e-7
input_layer = Input(shape=(input_dim, ))
encoder = Dense(encoding_dim, activation="tanh", activity_regularizer=regularizers.l1(learning_rate))(input_layer)
encoder = Dense(hidden_dim, activation="relu")(encoder)
decoder = Dense(hidden_dim, activation='tanh')(encoder)
decoder = Dense(input_dim, activation='relu')(decoder)
autoencoder = Model(inputs=input_layer, outputs=decoder)
autoencoder.compile(metrics=['accuracy'],
loss='mean_squared_error',
optimizer='adam')
cp = ModelCheckpoint(filepath="autoencoder_fraud.h5",
save_best_only=True,
verbose=0)
tb = TensorBoard(log_dir='./logs',
histogram_freq=0,
write_graph=True,
write_images=True)
history = autoencoder.fit(X_train, X_train,
epochs=nb_epoch,
batch_size=batch_size,
shuffle=True,
validation_data=(X_test, X_test),
verbose=1,
callbacks=[cp, tb]).history
here is a snippet of the loss values.
Epoch 1/100
10131/10131 [==============================] - 32s 3ms/step - loss: 52445827358.6230 - accuracy: 0.3389 - val_loss: 9625651200.0000 - val_accuracy: 0.5083
Epoch 2/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52393605025.8066 - accuracy: 0.5083 - val_loss: 9621398528.0000 - val_accuracy: 0.5083
Epoch 3/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52486496629.1354 - accuracy: 0.5082 - val_loss: 9617147904.0000 - val_accuracy: 0.5083
Epoch 4/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52514002255.9432 - accuracy: 0.5070 - val_loss: 9612887040.0000 - val_accuracy: 0.5083
Epoch 5/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52436489238.6388 - accuracy: 0.5076 - val_loss: 9608664064.0000 - val_accuracy: 0.5083
Epoch 6/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52430005774.7556 - accuracy: 0.5081 - val_loss: 9604417536.0000 - val_accuracy: 0.5083
Epoch 7/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52474495714.5898 - accuracy: 0.5079 - val_loss: 9600195584.0000 - val_accuracy: 0.5083
Epoch 8/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52423052560.0695 - accuracy: 0.5076 - val_loss: 9595947008.0000 - val_accuracy: 0.5083
Epoch 9/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52442358260.0742 - accuracy: 0.5072 - val_loss: 9591708672.0000 - val_accuracy: 0.5083
Epoch 10/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52402494704.5369 - accuracy: 0.5089 - val_loss: 9587487744.0000 - val_accuracy: 0.5083
Epoch 11/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52396583628.3553 - accuracy: 0.5081 - val_loss: 9583238144.0000 - val_accuracy: 0.5083
Epoch 12/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52349824708.2700 - accuracy: 0.5076 - val_loss: 9579020288.0000 - val_accuracy: 0.5083
Epoch 13/100
10131/10131 [==============================] - 31s 3ms/step - loss: 52332072133.6850 - accuracy: 0.5083 - val_loss: 9574786048.0000 - val_accuracy: 0.5083
Epoch 14/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52353680011.6731 - accuracy: 0.5086 - val_loss: 9570555904.0000 - val_accuracy: 0.5083
Epoch 15/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52347432594.5456 - accuracy: 0.5088 - val_loss: 9566344192.0000 - val_accuracy: 0.5083
Epoch 16/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52327825554.3435 - accuracy: 0.5076 - val_loss: 9562103808.0000 - val_accuracy: 0.5083
Epoch 17/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52347251610.1255 - accuracy: 0.5080 - val_loss: 9557892096.0000 - val_accuracy: 0.5083
Epoch 18/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52292632667.3636 - accuracy: 0.5079 - val_loss: 9553654784.0000 - val_accuracy: 0.5083
Epoch 19/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52354135093.7671 - accuracy: 0.5083 - val_loss: 9549425664.0000 - val_accuracy: 0.5083
Epoch 20/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52295668148.2006 - accuracy: 0.5086 - val_loss: 9545219072.0000 - val_accuracy: 0.5083
Epoch 21/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52314219115.3320 - accuracy: 0.5079 - val_loss: 9540980736.0000 - val_accuracy: 0.5083
Epoch 22/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52328022934.0829 - accuracy: 0.5079 - val_loss: 9536788480.0000 - val_accuracy: 0.5083
Epoch 23/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52268139834.5172 - accuracy: 0.5074 - val_loss: 9532554240.0000 - val_accuracy: 0.5083
Epoch 24/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52308370726.3040 - accuracy: 0.5077 - val_loss: 9528341504.0000 - val_accuracy: 0.5083
Epoch 25/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52224468101.4070 - accuracy: 0.5081 - val_loss: 9524126720.0000 - val_accuracy: 0.5083
Epoch 26/100
10131/10131 [==============================] - 30s 3ms/step - loss: 52200100823.1694 - accuracy: 0.5080 - val_loss: 9519915008.0000 - val_accuracy: 0.5083
Any advice/solution will be highly appreciated. Thank you
I have scaled the numarical data using StandardScaler and encoded
categorical data using LabelEncoder
First of all, check what numerical data you scaled.
I think you wrongly scaled cc_num, because cc_num is a categorical column.
This should solve your problem with high loss, but it doen't mean your model will be good.
You should first make a good check on the features and try to get some useful relationships between label and features (data preprocessing/featurezation)
I state that I am not at all familiar with neural networks and this is the first time that I have tried to develop one.
The problem lies in predicting a week's pollution forecast, based on the previous month.
Unstructured data with 15 features are:
Start data
The data to be predicted is 'gas', for a total of 168 hours in the next week, is the hours in a week.
MinMaxScaler(feature_range (0,1)) is applied to the data. And then the data is split into train and test data. Since only one year of hourly measurements is available, the data is resampled in series of 672 hourly samples that each starts from every day of the year at midnight. Therefore, from about 8000 starting hourly surveys, about 600 series of 672 samples are obtained.
The 'date' is removed from the initial data and the form of train_x and train_y is:
Shape of train_x and train_y
In train_x[0] there are 672 hourly readings for the first 4 weeks of the data set and consist of all features including 'gas'.
In train_y [0], on the other hand, there are 168 hourly readings for the following week which begins when the month ends in train_x [0].
Train_X[0] where column 0 is 'gas' and Train_y[0] with only gas column for the next week after train_x[0]
TRAIN X SHAPE = (631, 672, 14)
TRAIN Y SHAPE = (631, 168, 1)
After organizing the data in this way (if it's wrong please let me know), I built the neural network as the following:
train_x, train_y = to_supervised(train, n_input)
train_x = train_x.astype(float)
train_y = train_y.astype(float)
# define parameters
verbose, epochs, batch_size = 1, 200, 50
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
# define model
model = Sequential()
opt = optimizers.RMSprop(learning_rate=1e-3)
model.add(layers.GRU(14, activation='relu', input_shape=(n_timesteps, n_features),return_sequences=False, stateful=False))
model.add(layers.Dense(1, activation='relu'))
#model.add(layers.Dense(14, activation='linear'))
model.add(layers.Dense(n_outputs, activation='sigmoid'))
model.summary()
model.compile(loss='mse', optimizer=opt, metrics=['accuracy'])
train_y = np.concatenate(train_y).reshape(len(train_y), 168)
callback_early_stopping = EarlyStopping(monitor='val_loss',
patience=5, verbose=1)
callback_tensorboard = TensorBoard(log_dir='./23_logs/',
histogram_freq=0,
write_graph=False)
callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',
factor=0.1,
min_lr=1e-4,
patience=0,
verbose=1)
callbacks = [callback_early_stopping,
callback_tensorboard,
callback_reduce_lr]
history = model.fit(train_x, train_y, epochs=epochs, batch_size=batch_size, verbose=verbose, shuffle=False
, validation_split=0.2, callbacks=callbacks)
When i fit the network i get:
11/11 [==============================] - 5s 305ms/step - loss: 0.1625 - accuracy: 0.0207 - val_loss: 0.1905 - val_accuracy: 0.0157
Epoch 2/200
11/11 [==============================] - 2s 179ms/step - loss: 0.1594 - accuracy: 0.0037 - val_loss: 0.1879 - val_accuracy: 0.0157
Epoch 3/200
11/11 [==============================] - 2s 169ms/step - loss: 0.1571 - accuracy: 0.0040 - val_loss: 0.1855 - val_accuracy: 0.0079
Epoch 4/200
11/11 [==============================] - 2s 165ms/step - loss: 0.1550 - accuracy: 0.0092 - val_loss: 0.1832 - val_accuracy: 0.0079
Epoch 5/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1529 - accuracy: 0.0102 - val_loss: 0.1809 - val_accuracy: 0.0079
Epoch 6/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1508 - accuracy: 0.0085 - val_loss: 0.1786 - val_accuracy: 0.0079
Epoch 7/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1487 - accuracy: 0.0023 - val_loss: 0.1763 - val_accuracy: 0.0079
Epoch 8/200
11/11 [==============================] - 2s 158ms/step - loss: 0.1467 - accuracy: 0.0023 - val_loss: 0.1740 - val_accuracy: 0.0079
Epoch 9/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1446 - accuracy: 0.0034 - val_loss: 0.1718 - val_accuracy: 0.0000e+00
Epoch 10/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1426 - accuracy: 0.0034 - val_loss: 0.1695 - val_accuracy: 0.0000e+00
Epoch 11/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1406 - accuracy: 0.0034 - val_loss: 0.1673 - val_accuracy: 0.0000e+00
Epoch 12/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1387 - accuracy: 0.0034 - val_loss: 0.1651 - val_accuracy: 0.0000e+00
Epoch 13/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1367 - accuracy: 0.0052 - val_loss: 0.1629 - val_accuracy: 0.0000e+00
Epoch 14/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1348 - accuracy: 0.0052 - val_loss: 0.1608 - val_accuracy: 0.0000e+00
Epoch 15/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1328 - accuracy: 0.0052 - val_loss: 0.1586 - val_accuracy: 0.0000e+00
Epoch 16/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1309 - accuracy: 0.0052 - val_loss: 0.1565 - val_accuracy: 0.0000e+00
Epoch 17/200
11/11 [==============================] - 2s 171ms/step - loss: 0.1290 - accuracy: 0.0052 - val_loss: 0.1544 - val_accuracy: 0.0000e+00
Epoch 18/200
11/11 [==============================] - 2s 174ms/step - loss: 0.1271 - accuracy: 0.0052 - val_loss: 0.1523 - val_accuracy: 0.0000e+00
Epoch 19/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1253 - accuracy: 0.0052 - val_loss: 0.1502 - val_accuracy: 0.0000e+00
Epoch 20/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1234 - accuracy: 0.0052 - val_loss: 0.1482 - val_accuracy: 0.0000e+00
Epoch 21/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1216 - accuracy: 0.0052 - val_loss: 0.1461 - val_accuracy: 0.0000e+00
Epoch 22/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1198 - accuracy: 0.0052 - val_loss: 0.1441 - val_accuracy: 0.0000e+00
Epoch 23/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1180 - accuracy: 0.0052 - val_loss: 0.1421 - val_accuracy: 0.0000e+00
Epoch 24/200
11/11 [==============================] - 2s 163ms/step - loss: 0.1162 - accuracy: 0.0052 - val_loss: 0.1401 - val_accuracy: 0.0000e+00
Epoch 25/200
11/11 [==============================] - 2s 167ms/step - loss: 0.1145 - accuracy: 0.0052 - val_loss: 0.1381 - val_accuracy: 0.0000e+00
Epoch 26/200
11/11 [==============================] - 2s 188ms/step - loss: 0.1127 - accuracy: 0.0052 - val_loss: 0.1361 - val_accuracy: 0.0000e+00
Epoch 27/200
11/11 [==============================] - 2s 169ms/step - loss: 0.1110 - accuracy: 0.0052 - val_loss: 0.1342 - val_accuracy: 0.0000e+00
Epoch 28/200
11/11 [==============================] - 2s 189ms/step - loss: 0.1093 - accuracy: 0.0052 - val_loss: 0.1323 - val_accuracy: 0.0000e+00
Epoch 29/200
11/11 [==============================] - 2s 183ms/step - loss: 0.1076 - accuracy: 0.0079 - val_loss: 0.1304 - val_accuracy: 0.0000e+00
Epoch 30/200
11/11 [==============================] - 2s 172ms/step - loss: 0.1059 - accuracy: 0.0079 - val_loss: 0.1285 - val_accuracy: 0.0000e+00
Epoch 31/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1042 - accuracy: 0.0079 - val_loss: 0.1266 - val_accuracy: 0.0000e+00
Epoch 32/200
Accuracy always remains very low and sometimes (like this case) val_accuracy becomes 0 and never changes. While loss and val_loss do not converge well but decrease. I realize that I am certainly doing many things wrong and I cannot understand how I can fix it. I have obviously tried with other hyperparameters and also with other networks like LSTM, but I didn't get satisfactory results.
How can I improve the model so that the accuracy is at least decent? Any advice is welcome, thank you very much!
I am building a DNN with keras to classify between background or signal events (HEP). Nevertheless the loss and the accuracy are not changing.
I already tried changing the parameters on the optimizer, normalizing the data, changing the number of layers, neurons, epochs, initializing the weights, etc.
Here's the model:
epochs = 20
num_features = 2
num_classes = 2
batch_size = 32
# model
print("\n Building model...")
model = Sequential()
model.add(Dropout(0.2))
model.add(Dense(128, input_shape=(2,), activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes,activation=tf.nn.softmax))
print("\n Compiling model...")
opt = adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0,
amsgrad=False)
# compile model
model.compile(
loss='sparse_categorical_crossentropy',
optimizer=opt,
metrics=['accuracy'])
print("\n Fitting model...")
history = model.fit(x_train, y_train, epochs = epochs,
batch_size = batch_size, validation_data = (x_test, y_test))
I'm expecting a change in the loss but it won't decrease from 0.69-ish.
The epochs report
Building model...
Compiling model...
Fitting model...
Train on 18400 samples, validate on 4600 samples
Epoch 1/20
18400/18400 [==============================] - 1s 71us/step - loss: 0.6939 - acc: 0.4965 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 2/20
18400/18400 [==============================] - 1s 60us/step - loss: 0.6935 - acc: 0.5045 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 3/20
18400/18400 [==============================] - 1s 69us/step - loss: 0.6937 - acc: 0.4993 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 4/20
18400/18400 [==============================] - 1s 65us/step - loss: 0.6939 - acc: 0.4984 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 5/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6936 - acc: 0.5000 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 6/20
18400/18400 [==============================] - 1s 57us/step - loss: 0.6937 - acc: 0.4913 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 7/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6935 - acc: 0.5008 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 8/20
18400/18400 [==============================] - 1s 63us/step - loss: 0.6936 - acc: 0.5013 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 9/20
18400/18400 [==============================] - 1s 67us/step - loss: 0.6936 - acc: 0.4924 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 10/20
18400/18400 [==============================] - 1s 61us/step - loss: 0.6933 - acc: 0.5067 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 11/20
18400/18400 [==============================] - 1s 64us/step - loss: 0.6938 - acc: 0.4972 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 12/20
18400/18400 [==============================] - 1s 64us/step - loss: 0.6936 - acc: 0.4991 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 13/20
18400/18400 [==============================] - 1s 70us/step - loss: 0.6937 - acc: 0.4960 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 14/20
18400/18400 [==============================] - 1s 63us/step - loss: 0.6935 - acc: 0.4992 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 15/20
18400/18400 [==============================] - 1s 61us/step - loss: 0.6937 - acc: 0.4940 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 16/20
18400/18400 [==============================] - 1s 68us/step - loss: 0.6933 - acc: 0.5067 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 17/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6938 - acc: 0.4997 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 18/20
18400/18400 [==============================] - 1s 56us/step - loss: 0.6936 - acc: 0.4972 - val_loss: 0.6941 - val_acc: 0.5000
Epoch 19/20
18400/18400 [==============================] - 1s 57us/step - loss: 0.6934 - acc: 0.5061 - val_loss: 0.6954 - val_acc: 0.5000
Epoch 20/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6936 - acc: 0.5037 - val_loss: 0.6939 - val_acc: 0.5000
Update: My data preparation contains this
np.random.shuffle(x_train)
np.random.shuffle(y_train)
np.random.shuffle(x_test)
np.random.shuffle(y_test)
And I'm thinking it's changing the class for each data point cause the shuffle is done separately.
I'm trying to do Simple classification of road picture ( 1way/2way) with CNN, my dataset is composed of 4k images of class 1 and ~4K of class 2, so normaly the classes are equilibrate, each class is stored in different folder.
But the metrics do some kind of "jump" ? i tryed different size of the input_shape, different optimizer ( 'adam','rmsprop'), batch size ( 10,16,20) and i get the same result... any one know what causes this beavior ?
code:
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(300, 300,3)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(128, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
batch_size = 10
train_datagen = ImageDataGenerator(
# rescale=1./255,
# shear_range=0.2,
# zoom_range=0.2,
# horizontal_flip=True
featurewise_std_normalization=True,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
test_datagen = ImageDataGenerator(featurewise_std_normalization=True,
rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(300, 300),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'data/validation',
target_size=(300, 300),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=2000 // batch_size,
epochs=50,
validation_data=validation_generator,
validation_steps=800 // batch_size)
When i run this code i obtain this result:
Epoch 1/50
125/125 [==============================] - 253s 2s/step - loss: 0.8142 - acc: 0.5450 - val_loss: 0.4937 - val_acc: 0.8662
Epoch 2/50
125/125 [==============================] - 254s 2s/step - loss: 0.6748 - acc: 0.5980 - val_loss: 0.5782 - val_acc: 0.7859
Epoch 3/50
125/125 [==============================] - 255s 2s/step - loss: 0.6679 - acc: 0.6580 - val_loss: 0.5068 - val_acc: 0.8562
Epoch 4/50
125/125 [==============================] - 255s 2s/step - loss: 0.6438 - acc: 0.6780 - val_loss: 0.5018 - val_acc: 0.8766
Epoch 5/50
125/125 [==============================] - 257s 2s/step - loss: 0.6427 - acc: 0.7245 - val_loss: 0.3760 - val_acc: 0.9213
Epoch 6/50
125/125 [==============================] - 256s 2s/step - loss: 0.5635 - acc: 0.7435 - val_loss: 0.6140 - val_acc: 0.6398
Epoch 7/50
125/125 [==============================] - 254s 2s/step - loss: 0.6226 - acc: 0.7320 - val_loss: 0.1852 - val_acc: 0.9433
Epoch 8/50
125/125 [==============================] - 252s 2s/step - loss: 0.4858 - acc: 0.7765 - val_loss: 0.1617 - val_acc: 0.9437
Epoch 9/50
125/125 [==============================] - 253s 2s/step - loss: 0.4433 - acc: 0.8120 - val_loss: 0.5577 - val_acc: 0.6788
Epoch 10/50
125/125 [==============================] - 252s 2s/step - loss: 0.4621 - acc: 0.7935 - val_loss: 0.1000 - val_acc: 0.9762
Epoch 11/50
125/125 [==============================] - 254s 2s/step - loss: 0.4572 - acc: 0.8035 - val_loss: 0.3797 - val_acc: 0.8161
Epoch 12/50
125/125 [==============================] - 257s 2s/step - loss: 0.4707 - acc: 0.8105 - val_loss: 0.0903 - val_acc: 0.9761
Epoch 13/50
125/125 [==============================] - 254s 2s/step - loss: 0.4134 - acc: 0.8390 - val_loss: 0.1587 - val_acc: 0.9437
Epoch 14/50
125/125 [==============================] - 252s 2s/step - loss: 0.4023 - acc: 0.8355 - val_loss: 0.1149 - val_acc: 0.9584
Epoch 15/50
125/125 [==============================] - 253s 2s/step - loss: 0.4286 - acc: 0.8255 - val_loss: 0.0897 - val_acc: 0.9700
Epoch 16/50
125/125 [==============================] - 253s 2s/step - loss: 0.4665 - acc: 0.8140 - val_loss: 0.6411 - val_acc: 0.8136
Epoch 17/50
125/125 [==============================] - 252s 2s/step - loss: 0.4010 - acc: 0.8315 - val_loss: 0.1205 - val_acc: 0.9736
Epoch 18/50
125/125 [==============================] - 253s 2s/step - loss: 0.3790 - acc: 0.8550 - val_loss: 0.0993 - val_acc: 0.9613
Epoch 19/50
125/125 [==============================] - 251s 2s/step - loss: 0.3717 - acc: 0.8620 - val_loss: 0.1154 - val_acc: 0.9748
Epoch 20/50
125/125 [==============================] - 250s 2s/step - loss: 0.4434 - acc: 0.8405 - val_loss: 0.1251 - val_acc: 0.9537
Epoch 21/50
125/125 [==============================] - 253s 2s/step - loss: 0.4535 - acc: 0.7545 - val_loss: 0.6766 - val_acc: 0.3640
Epoch 22/50
125/125 [==============================] - 252s 2s/step - loss: 0.7482 - acc: 0.7140 - val_loss: 0.4803 - val_acc: 0.7950
Epoch 23/50
125/125 [==============================] - 252s 2s/step - loss: 0.3712 - acc: 0.8585 - val_loss: 0.1056 - val_acc: 0.9685
Epoch 24/50
125/125 [==============================] - 251s 2s/step - loss: 0.3836 - acc: 0.8545 - val_loss: 0.1267 - val_acc: 0.9673
Epoch 25/50
125/125 [==============================] - 250s 2s/step - loss: 0.3879 - acc: 0.8805 - val_loss: 0.8669 - val_acc: 0.8100
Epoch 26/50
125/125 [==============================] - 250s 2s/step - loss: 0.3735 - acc: 0.8825 - val_loss: 0.1472 - val_acc: 0.9685
Epoch 27/50
125/125 [==============================] - 250s 2s/step - loss: 0.4577 - acc: 0.8620 - val_loss: 0.3285 - val_acc: 0.8925
Epoch 28/50
125/125 [==============================] - 252s 2s/step - loss: 0.3805 - acc: 0.8875 - val_loss: 0.3930 - val_acc: 0.7821
Epoch 29/50
125/125 [==============================] - 250s 2s/step - loss: 0.3565 - acc: 0.8930 - val_loss: 0.1087 - val_acc: 0.9647
Epoch 30/50
125/125 [==============================] - 250s 2s/step - loss: 0.4680 - acc: 0.8845 - val_loss: 0.1012 - val_acc: 0.9688
Epoch 31/50
125/125 [==============================] - 250s 2s/step - loss: 0.3293 - acc: 0.9080 - val_loss: 0.0700 - val_acc: 0.9811
Epoch 32/50
125/125 [==============================] - 250s 2s/step - loss: 0.4197 - acc: 0.9060 - val_loss: 0.1464 - val_acc: 0.9700
Epoch 33/50
125/125 [==============================] - 251s 2s/step - loss: 0.3656 - acc: 0.9005 - val_loss: 8.8236 - val_acc: 0.4307
Epoch 34/50
125/125 [==============================] - 249s 2s/step - loss: 0.4593 - acc: 0.9015 - val_loss: 4.3916 - val_acc: 0.6826
Epoch 35/50
125/125 [==============================] - 250s 2s/step - loss: 0.4824 - acc: 0.8605 - val_loss: 0.0748 - val_acc: 0.9850
Epoch 36/50
125/125 [==============================] - 250s 2s/step - loss: 0.4629 - acc: 0.8875 - val_loss: 0.2257 - val_acc: 0.8728
Epoch 37/50
125/125 [==============================] - 250s 2s/step - loss: 0.3708 - acc: 0.9075 - val_loss: 0.1196 - val_acc: 0.9537
Epoch 38/50
125/125 [==============================] - 250s 2s/step - loss: 0.9151 - acc: 0.8605 - val_loss: 0.1266 - val_acc: 0.9559
Epoch 39/50
125/125 [==============================] - 250s 2s/step - loss: 0.3700 - acc: 0.9035 - val_loss: 0.1038 - val_acc: 0.9812
Epoch 40/50
125/125 [==============================] - 249s 2s/step - loss: 0.5900 - acc: 0.8625 - val_loss: 0.0838 - val_acc: 0.9887
Epoch 41/50
125/125 [==============================] - 250s 2s/step - loss: 0.4409 - acc: 0.9065 - val_loss: 0.0828 - val_acc: 0.9773
Epoch 42/50
125/125 [==============================] - 250s 2s/step - loss: 0.3415 - acc: 0.9115 - val_loss: 0.8084 - val_acc: 0.8788
Epoch 43/50
125/125 [==============================] - 250s 2s/step - loss: 0.5181 - acc: 0.8440 - val_loss: 0.0998 - val_acc: 0.9786
Epoch 44/50
125/125 [==============================] - 249s 2s/step - loss: 0.3270 - acc: 0.8970 - val_loss: 0.1155 - val_acc: 0.9625
Epoch 45/50
125/125 [==============================] - 250s 2s/step - loss: 0.3810 - acc: 0.9125 - val_loss: 0.2881 - val_acc: 0.9484
Epoch 46/50
125/125 [==============================] - 249s 2s/step - loss: 0.3499 - acc: 0.9220 - val_loss: 0.3109 - val_acc: 0.8564
Epoch 47/50
125/125 [==============================] - 250s 2s/step - loss: 0.3505 - acc: 0.9160 - val_loss: 0.0861 - val_acc: 0.9788
Epoch 48/50
125/125 [==============================] - 250s 2s/step - loss: 0.3073 - acc: 0.9225 - val_loss: 0.0999 - val_acc: 0.9874
Epoch 49/50
125/125 [==============================] - 250s 2s/step - loss: 0.4418 - acc: 0.9000 - val_loss: 0.0301 - val_acc: 0.9925
Epoch 50/50
125/125 [==============================] - 250s 2s/step - loss: 0.3501 - acc: 0.9190 - val_loss: 0.0351 - val_acc: 0.9861
it's overfit ? or just the random set parameters position of my loss function ? i will try to found other picture to build new validation dataset ...
"each class is stored in different folder"
So do you mean 1 class is inside 'train' folder,
and another class is inside 'validate' folder?
try setting batch size to 32
and size of training vs validate sets to the ratio of 0.8 vs 0.2
EDIT
I found a link that you may refer to:
https://stats.stackexchange.com/questions/187335/validation-error-less-than-training-error
EDIT
Try getting more samples.
If there's difficulty getting more samples,
try creating/modifying from the existing samples.