I have a classification model that is clearly overfitting and the validation accuracy doesn't change.
I've tried using feature selection and feature extraction methods but they didn't help.
feature selection method:
fs = SelectKBest(f_classif, k=10)
fs.fit(x, y)
feature_index = fs.get_support(True)
feature_index = feature_index.tolist()
best_features = []
# makes list of best features
for index in feature_index:
best_features.append(x[:, index])
x = (np.array(best_features, dtype=np.float)).T
model:
def model_and_print(x, y, Epochs, Batch_Size, loss, opt, class_weight, callback):
# fix random seed for reproducibility
seed = 7
np.random.seed(seed)
# define 10-fold cross validation test harness
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=seed)
# K-fold Cross Validation model evaluation
fold_no = 1
for train, test in kfold.split(x, y):
# create model
model = Sequential()
model.add(Dropout(0.2, input_shape=(len(x[0]),)))
model.add(Dense(6, activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.2))
model.add(Dense(8, activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.4))
model.add(Dense(1, activation=tf.nn.sigmoid))
# compile model
model.compile(optimizer=opt,
loss=loss, metrics=['accuracy']
)
history = model.fit(x[train], y[train], validation_data=(x[test], y[test]), epochs=Epochs,
batch_size=Batch_Size, verbose=1)
def main():
data = ["data.pkl", "data_list.pkl", "data_mean.pkl"]
df = pd.read_pickle(data[2])
x, y = data_frame_to_feature_and_target_arrays(df)
# hyper meters
Epochs = 200
Batch_Size = 1
learning_rate = 0.003
optimizer = optimizers.Adam(learning_rate=learning_rate)
loss = "binary_crossentropy"
model_and_print(x, y, Epochs, Batch_Size, loss, optimizer, class_weight, es_callback)
if __name__ == "__main__":
main()
output for part of one fold:
1/73 [..............................] - ETA: 0s - loss: 0.6470 - accuracy: 1.0000
62/73 [========================>.....] - ETA: 0s - loss: 0.5665 - accuracy: 0.7097
73/73 [==============================] - 0s 883us/step - loss: 0.5404 - accuracy: 0.7534 - val_loss: 0.5576 - val_accuracy: 0.5000
Epoch 100/200
1/73 [..............................] - ETA: 0s - loss: 0.4743 - accuracy: 1.0000
69/73 [===========================>..] - ETA: 0s - loss: 0.6388 - accuracy: 0.6522
73/73 [==============================] - 0s 806us/step - loss: 0.6316 - accuracy: 0.6575 - val_loss: 0.5592 - val_accuracy: 0.5000
Epoch 101/200
1/73 [..............................] - ETA: 0s - loss: 0.6005 - accuracy: 1.0000
69/73 [===========================>..] - ETA: 0s - loss: 0.5656 - accuracy: 0.7101
73/73 [==============================] - 0s 806us/step - loss: 0.5641 - accuracy: 0.7123 - val_loss: 0.5629 - val_accuracy: 0.5000
Epoch 102/200
1/73 [..............................] - ETA: 0s - loss: 0.2126 - accuracy: 1.0000
65/73 [=========================>....] - ETA: 0s - loss: 0.5042 - accuracy: 0.8000
73/73 [==============================] - 0s 847us/step - loss: 0.5340 - accuracy: 0.7671 - val_loss: 0.5608 - val_accuracy: 0.5000
Epoch 103/200
1/73 [..............................] - ETA: 0s - loss: 0.8801 - accuracy: 0.0000e+00
68/73 [==========================>...] - ETA: 0s - loss: 0.5754 - accuracy: 0.6471
73/73 [==============================] - 0s 819us/step - loss: 0.5780 - accuracy: 0.6575 - val_loss: 0.5639 - val_accuracy: 0.5000
Epoch 104/200
1/73 [..............................] - ETA: 0s - loss: 0.0484 - accuracy: 1.0000
70/73 [===========================>..] - ETA: 0s - loss: 0.5711 - accuracy: 0.7571
73/73 [==============================] - 0s 806us/step - loss: 0.5689 - accuracy: 0.7534 - val_loss: 0.5608 - val_accuracy: 0.5000
Epoch 105/200
1/73 [..............................] - ETA: 0s - loss: 0.1237 - accuracy: 1.0000
69/73 [===========================>..] - ETA: 0s - loss: 0.5953 - accuracy: 0.7101
73/73 [==============================] - 0s 820us/step - loss: 0.5922 - accuracy: 0.7260 - val_loss: 0.5672 - val_accuracy: 0.5000
Epoch 106/200
1/73 [..............................] - ETA: 0s - loss: 0.3360 - accuracy: 1.0000
67/73 [==========================>...] - ETA: 0s - loss: 0.5175 - accuracy: 0.7313
73/73 [==============================] - 0s 847us/step - loss: 0.5320 - accuracy: 0.7397 - val_loss: 0.5567 - val_accuracy: 0.5000
Epoch 107/200
1/73 [..............................] - ETA: 0s - loss: 0.1384 - accuracy: 1.0000
67/73 [==========================>...] - ETA: 0s - loss: 0.5435 - accuracy: 0.6866
73/73 [==============================] - 0s 833us/step - loss: 0.5541 - accuracy: 0.6575 - val_loss: 0.5629 - val_accuracy: 0.5000
Epoch 108/200
1/73 [..............................] - ETA: 0s - loss: 0.2647 - accuracy: 1.0000
69/73 [===========================>..] - ETA: 0s - loss: 0.6047 - accuracy: 0.6232
73/73 [==============================] - 0s 820us/step - loss: 0.5948 - accuracy: 0.6301 - val_loss: 0.5660 - val_accuracy: 0.5000
Epoch 109/200
1/73 [..............................] - ETA: 0s - loss: 0.8837 - accuracy: 0.0000e+00
66/73 [==========================>...] - ETA: 0s - loss: 0.5250 - accuracy: 0.7576
73/73 [==============================] - 0s 861us/step - loss: 0.5357 - accuracy: 0.7397 - val_loss: 0.5583 - val_accuracy: 0.5000
Epoch 110/200
final accuracy:
Score for fold 10: loss of 0.5600861310958862; accuracy of 50.0%
my question is what can I do about the overfitting if I've tried feature extraction and dropout layers and why is the validation accuracy not changing?
Related
So I am trying to create an LSTM that can predict the next time step of a double pendulum. The data that I am trying to train with is a (2001, 4) numpy array. (i.e. the first 5 rows will look like:
array([[ 1.04719755, 0. , 1.04719755, 0. ],
[ 1.03659984, -0.42301933, 1.04717544, -0.00178865],
[ 1.00508218, -0.83475539, 1.04682248, -0.01551541],
[ 0.95354768, -1.22094052, 1.04514269, -0.05838011],
[ 0.88372305, -1.56345555, 1.04009056, -0.15443162]])
where each row is a unique representation of the state of the double pendulum.)
So I wanted to created an LSTM that could learn to predict the next state given the current one.
Here was my code for it so far (full_sol is the (2001, 4) matrix:
import numpy as np
from tensorflow import keras
import tensorflow as tf
# full_sol = np.random.rand(2001, 4)
full_sol = full_sol.reshape((full_sol.shape[0], 1, full_sol.shape[1]))
model = keras.Sequential()
model.add(keras.layers.LSTM(100, input_shape=(None, 4), return_sequences=True, dropout=0.2))
model.add(keras.layers.TimeDistributed(keras.layers.Dense(4, activation=tf.keras.layers.LeakyReLU(
alpha=0.3))))
model.compile(loss="mean_squared_error", optimizer="adam", metrics="accuracy")
history = model.fit(full_sol[:-1,:,:], full_sol[1:,:,:], epochs=20)
Then when I train, I get the following results:
Epoch 1/20
63/63 [==============================] - 3s 4ms/step - loss: 1.7181 - accuracy: 0.4200
Epoch 2/20
63/63 [==============================] - 0s 4ms/step - loss: 1.0481 - accuracy: 0.5155
Epoch 3/20
63/63 [==============================] - 0s 5ms/step - loss: 0.7584 - accuracy: 0.5715
Epoch 4/20
63/63 [==============================] - 0s 5ms/step - loss: 0.5134 - accuracy: 0.6420
Epoch 5/20
63/63 [==============================] - 0s 5ms/step - loss: 0.3944 - accuracy: 0.7260
Epoch 6/20
63/63 [==============================] - 0s 5ms/step - loss: 0.3378 - accuracy: 0.7605
Epoch 7/20
63/63 [==============================] - 0s 5ms/step - loss: 0.3549 - accuracy: 0.7825
Epoch 8/20
63/63 [==============================] - 0s 4ms/step - loss: 0.3528 - accuracy: 0.7995
Epoch 9/20
63/63 [==============================] - 0s 5ms/step - loss: 0.3285 - accuracy: 0.8020
Epoch 10/20
63/63 [==============================] - 0s 5ms/step - loss: 0.2874 - accuracy: 0.8030
Epoch 11/20
63/63 [==============================] - 0s 4ms/step - loss: 0.3072 - accuracy: 0.8135
Epoch 12/20
63/63 [==============================] - 0s 4ms/step - loss: 0.3075 - accuracy: 0.8035
Epoch 13/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2942 - accuracy: 0.8030
Epoch 14/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2637 - accuracy: 0.8170
Epoch 15/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2675 - accuracy: 0.8150
Epoch 16/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2644 - accuracy: 0.8085
Epoch 17/20
63/63 [==============================] - 0s 5ms/step - loss: 0.2479 - accuracy: 0.8200
Epoch 18/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2475 - accuracy: 0.8215
Epoch 19/20
63/63 [==============================] - 0s 4ms/step - loss: 0.2243 - accuracy: 0.8340
Epoch 20/20
63/63 [==============================] - 0s 5ms/step - loss: 0.2430 - accuracy: 0.8240
So, quite high accuracy. But when I test it on the training set, the predictions aren't very good.
E.g. when I predict the first value:
model.predict(tf.expand_dims(full_sol[0], axis = 0))
I get array([[[ 1.0172144 , -0.3535697 , 1.1287913 , -0.23707283]]],dtype=float32)
Instead of array([[ 1.03659984, -0.42301933, 1.04717544, -0.00178865]]).
Where have I gone wrong?
I don't think you are doing anything wrong. What you are getting is still fairly close to the actual value. You can either change your choice of metric so it accurately represents the degree of error in your predictions, or you could try to increase the accuracy further.
I state that I am not at all familiar with neural networks and this is the first time that I have tried to develop one.
The problem lies in predicting a week's pollution forecast, based on the previous month.
Unstructured data with 15 features are:
Start data
The data to be predicted is 'gas', for a total of 168 hours in the next week, is the hours in a week.
MinMaxScaler(feature_range (0,1)) is applied to the data. And then the data is split into train and test data. Since only one year of hourly measurements is available, the data is resampled in series of 672 hourly samples that each starts from every day of the year at midnight. Therefore, from about 8000 starting hourly surveys, about 600 series of 672 samples are obtained.
The 'date' is removed from the initial data and the form of train_x and train_y is:
Shape of train_x and train_y
In train_x[0] there are 672 hourly readings for the first 4 weeks of the data set and consist of all features including 'gas'.
In train_y [0], on the other hand, there are 168 hourly readings for the following week which begins when the month ends in train_x [0].
Train_X[0] where column 0 is 'gas' and Train_y[0] with only gas column for the next week after train_x[0]
TRAIN X SHAPE = (631, 672, 14)
TRAIN Y SHAPE = (631, 168, 1)
After organizing the data in this way (if it's wrong please let me know), I built the neural network as the following:
train_x, train_y = to_supervised(train, n_input)
train_x = train_x.astype(float)
train_y = train_y.astype(float)
# define parameters
verbose, epochs, batch_size = 1, 200, 50
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
# define model
model = Sequential()
opt = optimizers.RMSprop(learning_rate=1e-3)
model.add(layers.GRU(14, activation='relu', input_shape=(n_timesteps, n_features),return_sequences=False, stateful=False))
model.add(layers.Dense(1, activation='relu'))
#model.add(layers.Dense(14, activation='linear'))
model.add(layers.Dense(n_outputs, activation='sigmoid'))
model.summary()
model.compile(loss='mse', optimizer=opt, metrics=['accuracy'])
train_y = np.concatenate(train_y).reshape(len(train_y), 168)
callback_early_stopping = EarlyStopping(monitor='val_loss',
patience=5, verbose=1)
callback_tensorboard = TensorBoard(log_dir='./23_logs/',
histogram_freq=0,
write_graph=False)
callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',
factor=0.1,
min_lr=1e-4,
patience=0,
verbose=1)
callbacks = [callback_early_stopping,
callback_tensorboard,
callback_reduce_lr]
history = model.fit(train_x, train_y, epochs=epochs, batch_size=batch_size, verbose=verbose, shuffle=False
, validation_split=0.2, callbacks=callbacks)
When i fit the network i get:
11/11 [==============================] - 5s 305ms/step - loss: 0.1625 - accuracy: 0.0207 - val_loss: 0.1905 - val_accuracy: 0.0157
Epoch 2/200
11/11 [==============================] - 2s 179ms/step - loss: 0.1594 - accuracy: 0.0037 - val_loss: 0.1879 - val_accuracy: 0.0157
Epoch 3/200
11/11 [==============================] - 2s 169ms/step - loss: 0.1571 - accuracy: 0.0040 - val_loss: 0.1855 - val_accuracy: 0.0079
Epoch 4/200
11/11 [==============================] - 2s 165ms/step - loss: 0.1550 - accuracy: 0.0092 - val_loss: 0.1832 - val_accuracy: 0.0079
Epoch 5/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1529 - accuracy: 0.0102 - val_loss: 0.1809 - val_accuracy: 0.0079
Epoch 6/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1508 - accuracy: 0.0085 - val_loss: 0.1786 - val_accuracy: 0.0079
Epoch 7/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1487 - accuracy: 0.0023 - val_loss: 0.1763 - val_accuracy: 0.0079
Epoch 8/200
11/11 [==============================] - 2s 158ms/step - loss: 0.1467 - accuracy: 0.0023 - val_loss: 0.1740 - val_accuracy: 0.0079
Epoch 9/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1446 - accuracy: 0.0034 - val_loss: 0.1718 - val_accuracy: 0.0000e+00
Epoch 10/200
11/11 [==============================] - 2s 160ms/step - loss: 0.1426 - accuracy: 0.0034 - val_loss: 0.1695 - val_accuracy: 0.0000e+00
Epoch 11/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1406 - accuracy: 0.0034 - val_loss: 0.1673 - val_accuracy: 0.0000e+00
Epoch 12/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1387 - accuracy: 0.0034 - val_loss: 0.1651 - val_accuracy: 0.0000e+00
Epoch 13/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1367 - accuracy: 0.0052 - val_loss: 0.1629 - val_accuracy: 0.0000e+00
Epoch 14/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1348 - accuracy: 0.0052 - val_loss: 0.1608 - val_accuracy: 0.0000e+00
Epoch 15/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1328 - accuracy: 0.0052 - val_loss: 0.1586 - val_accuracy: 0.0000e+00
Epoch 16/200
11/11 [==============================] - 2s 162ms/step - loss: 0.1309 - accuracy: 0.0052 - val_loss: 0.1565 - val_accuracy: 0.0000e+00
Epoch 17/200
11/11 [==============================] - 2s 171ms/step - loss: 0.1290 - accuracy: 0.0052 - val_loss: 0.1544 - val_accuracy: 0.0000e+00
Epoch 18/200
11/11 [==============================] - 2s 174ms/step - loss: 0.1271 - accuracy: 0.0052 - val_loss: 0.1523 - val_accuracy: 0.0000e+00
Epoch 19/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1253 - accuracy: 0.0052 - val_loss: 0.1502 - val_accuracy: 0.0000e+00
Epoch 20/200
11/11 [==============================] - 2s 161ms/step - loss: 0.1234 - accuracy: 0.0052 - val_loss: 0.1482 - val_accuracy: 0.0000e+00
Epoch 21/200
11/11 [==============================] - 2s 159ms/step - loss: 0.1216 - accuracy: 0.0052 - val_loss: 0.1461 - val_accuracy: 0.0000e+00
Epoch 22/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1198 - accuracy: 0.0052 - val_loss: 0.1441 - val_accuracy: 0.0000e+00
Epoch 23/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1180 - accuracy: 0.0052 - val_loss: 0.1421 - val_accuracy: 0.0000e+00
Epoch 24/200
11/11 [==============================] - 2s 163ms/step - loss: 0.1162 - accuracy: 0.0052 - val_loss: 0.1401 - val_accuracy: 0.0000e+00
Epoch 25/200
11/11 [==============================] - 2s 167ms/step - loss: 0.1145 - accuracy: 0.0052 - val_loss: 0.1381 - val_accuracy: 0.0000e+00
Epoch 26/200
11/11 [==============================] - 2s 188ms/step - loss: 0.1127 - accuracy: 0.0052 - val_loss: 0.1361 - val_accuracy: 0.0000e+00
Epoch 27/200
11/11 [==============================] - 2s 169ms/step - loss: 0.1110 - accuracy: 0.0052 - val_loss: 0.1342 - val_accuracy: 0.0000e+00
Epoch 28/200
11/11 [==============================] - 2s 189ms/step - loss: 0.1093 - accuracy: 0.0052 - val_loss: 0.1323 - val_accuracy: 0.0000e+00
Epoch 29/200
11/11 [==============================] - 2s 183ms/step - loss: 0.1076 - accuracy: 0.0079 - val_loss: 0.1304 - val_accuracy: 0.0000e+00
Epoch 30/200
11/11 [==============================] - 2s 172ms/step - loss: 0.1059 - accuracy: 0.0079 - val_loss: 0.1285 - val_accuracy: 0.0000e+00
Epoch 31/200
11/11 [==============================] - 2s 164ms/step - loss: 0.1042 - accuracy: 0.0079 - val_loss: 0.1266 - val_accuracy: 0.0000e+00
Epoch 32/200
Accuracy always remains very low and sometimes (like this case) val_accuracy becomes 0 and never changes. While loss and val_loss do not converge well but decrease. I realize that I am certainly doing many things wrong and I cannot understand how I can fix it. I have obviously tried with other hyperparameters and also with other networks like LSTM, but I didn't get satisfactory results.
How can I improve the model so that the accuracy is at least decent? Any advice is welcome, thank you very much!
This is my model.
def get_model2():
lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(
0.001,
decay_steps=100000,
decay_rate=0.96,
staircase=True)
model = Sequential()
model.add(Dense(1024,activation='relu',input_shape=[44]))
model.add(Dropout(0.2))
model.add(Dense(2048,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(4098,activation='relu'))
model.add(Dense(2048,activation='relu'))
model.add(Dense(1024,activation='relu'))
model.add(Dense(512,activation='relu'))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss=my_binary_crossentropy,optimizer=tf.keras.optimizers.Adam(learning_rate=lr_schedule),metrics=['accuracy'])
return model
And this is my training:
from sklearn.model_selection import RepeatedStratifiedKFold
model = get_model2()
model.save_weights('model.h5')
o = 0
hlavny_list = []
skf = RepeatedStratifiedKFold(n_splits=10,n_repeats=10,random_state=2027)
for train_index,test_index in skf.split(X, Y):
o = o + 1
X_test, X_train = X[train_index], X[test_index]
y_test, y_train = Y[train_index], Y[test_index]
model.load_weights('model.h5')
model.fit(X_train,y_train,epochs=10000,batch_size=256,validation_data=(X_test,y_test),callbacks=[early_stop])
vys = model.predict_classes(X_test)
a,b,c,d = potrebne_miery(y_true=y_test,y_pred=vys)
hl_list = [a,b,c,d]
hlavny_list.append(hl_list)
if (o % 4 == 0):
np.savetxt('/content/drive/My Drive/siete/model_t_t_1_9_moja_loss_v2.csv',np.array(hlavny_list),delimiter=',')
np.savetxt('/content/drive/My Drive/siete/model_t_t_1_9_moja_loss_v2.csv',np.array(hlavny_list),delimiter=',')
Nothing special here, except my own loss function, which looks like:
import tensorflow as tf
from tensorflow.python.ops import clip_ops
def my_binary_crossentropy(target, output, from_logits=False):
target = ops.convert_to_tensor_v2(target)
output = ops.convert_to_tensor_v2(output)
target = tf.cast(target, tf.float32)
epsilon_ = 0.01
output = clip_ops.clip_by_value(output, epsilon_, 1. - epsilon_)
# Compute cross entropy from probabilities.
bce = 8 * target * math_ops.log(output + epsilon_)
bce += (1 - target) * math_ops.log(1 - output + epsilon_)
return -bce
Before i was using Binary crossentropy and everything worked fine, no problem. But when i change loss function, some problem occured which i don't understand. Model is training in loop, and sometimes and i don't why or when, model behave like doesn't train.
Epoch 1/10000
124/124 [==============================] - 6s 47ms/step - loss: 1.1125 - accuracy: 0.4600 - val_loss: 0.7640 - val_accuracy: 0.9312
Epoch 2/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.6418 - accuracy: 0.8598 - val_loss: 0.5307 - val_accuracy: 0.8718
Epoch 3/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.5434 - accuracy: 0.8768 - val_loss: 0.5416 - val_accuracy: 0.8736
Epoch 4/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.5167 - accuracy: 0.8820 - val_loss: 0.5383 - val_accuracy: 0.9165
Epoch 5/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4948 - accuracy: 0.8898 - val_loss: 0.5136 - val_accuracy: 0.9156
Epoch 6/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4693 - accuracy: 0.8910 - val_loss: 0.5088 - val_accuracy: 0.9130
Epoch 7/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4533 - accuracy: 0.8925 - val_loss: 0.5163 - val_accuracy: 0.8551
Epoch 8/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4257 - accuracy: 0.8883 - val_loss: 0.5490 - val_accuracy: 0.9189
Epoch 9/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4237 - accuracy: 0.8919 - val_loss: 0.5302 - val_accuracy: 0.8172
Epoch 10/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4072 - accuracy: 0.8859 - val_loss: 0.5591 - val_accuracy: 0.9278
Epoch 11/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.3831 - accuracy: 0.8908 - val_loss: 0.5563 - val_accuracy: 0.8937
Epoch 00011: early stopping
32695
24925
221726
5339
Epoch 1/10000
124/124 [==============================] - 6s 48ms/step - loss: 4.1699 - accuracy: 0.8661 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 2/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 3/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 4/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 5/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 6/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 00006: early stopping
Loss, val_loss nor accuracy is not decreasing. I think that is some problem in loss function, because this problem occured after new loss function, this loop i have done maybe 10 000 times without error. It happens maybe 1 in 4 cycles. What is wrong? I will be very grateful for help. Thank you
Your dropout values are too low and make learning harder for the model. Use higher dropout value to overcome problem you have.
Start by building simple model with one hidden layer and use popular hyper parameters.
You can start fine tunning hyper parameters while updating the model as you move.
That is the simple and best way to debug this from my point of view.
Let me know if you need further help.
So I'm trying to learn how machine learning works, and I started with a simple stock price prediction program. I've tried for days to narrow down issues, but I am at a halt with my research and progress. My issue is that the Accuracy does not increase, the val_accuracy does not change either. I've narrowed my data-set to be a smaller set to observe behavior, and of course, still not changing...
I have tried switching the loss, the activation, and tried a number of changes to preparing the data... I don't get what is going on? This is on one stock ticker only... (trying to use one model for top 100 prices)
My layers / model
def createModel(X_train):
'''
#description -
'''
# Model
model = Sequential()
model.add(LSTM(512, activation = 'relu', return_sequences = True, input_shape = (X_train.shape[1:])))
model.add(Dropout(0.3))
model.add(LSTM(512, activation = 'relu', return_sequences = True))
model.add(Dropout(0.3))
model.add(LSTM(256, activation = 'relu', return_sequences = True))
model.add(Dropout(0.3))
model.add(LSTM(128, activation = 'relu', return_sequences = False))
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid'))
# print(model.summary())
# opt = tf.keras.optimizers.Adam(learning_rate = 0.01)
model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
return model
I have the following code snippets to read in the data and prepare the data.
if filename.endswith('.csv'):
data = pd.read_csv(filename)
# Clean up file name to extract ticker
filename = filename.replace('.csv', '')
data = data.drop(['Dividends', 'Stock Splits'], axis = 1)
data['Date'] = list(map(convertDateToNumber, data['Date']))
data.set_index('Date', inplace = True)
# Shift for a new column to do calculations on, then drop the shifted column after
data['Per Change'] = data['Open'].shift(1)
data['Percent Change'] = list(map(calculatePercentChange, data['Open'], data['Close']))
data['Class'] = (list(map(classify, data['Open'], data['Close'])))
# Drop the unnecessary headers now...
data = data.drop('Per Change', 1)
data.fillna(method = "ffill", inplace = True)
data.dropna(inplace = True)
trainingData = int(len(data) * 0.75)
training_data = data.head(trainingData).values.tolist()
training_data = scaler.fit_transform(training_data)
testingData = int(len(data) * 0.25)
testing_data = data.tail(testingData).values.tolist()
testing_data = scaler.fit_transform(testing_data)
X_train = []
y_train = []
for i in range(training_data.shape[0]):
X_train.append(training_data[i])
y_train.append(training_data[i, 2])
X_train, y_train = np.array(X_train), np.array(y_train)
X_train = X_train.reshape(X_train.shape[0], 1, X_train.shape[1])
# y_train = y_train.reshape(y_train.shape[0], 1)
# Test Data
X_test = []
y_test = []
for i in range(testing_data.shape[0]):
X_test.append(testing_data[i])
y_test.append(testing_data[i, 2])
X_test, y_test = np.array(X_test), np.array(y_test)
X_test = X_test.reshape(X_test.shape[0], 1, X_test.shape[1])
# y_test = y_test.reshape(y_test.shape[0], 1)
# Create the model
model = createModel(X_train)
# Evaluate the model
print('')
loss, acc = model.evaluate(X_test, y_test)
print("\n---------- Untrained model, accuracy: {:5.2f}% ----------\n".format(100 * acc))
if os.path.isdir(modelPath.replace('data model.h5', '')):
try:
model = tf.keras.models.load_model(modelPath, compile = True)
# Re-evaluate the model
loss, acc = model.evaluate(X_test, y_test)
print("\n---------- Restored model, accuracy: {:5.2f}% ----------\n".format(100 * acc))
except:
pass
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir = log_dir, histogram_freq = 1)
model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath = checkpointPath, save_weights_only = True, monitor = 'accuracy', mode = 'max', save_freq = 5)
# To throw it all together... fit() trains the model
model.fit(X_train, y_train, validation_data = (X_test, y_test), shuffle = True, epochs = 50, batch_size = 500, callbacks = [tensorboard_callback, model_checkpoint_callback])
model.save(modelPath)
# # Call the model protocol
y_pred = model.predict(X_test)
scale = 1 / scaler.scale_[0]
y_test = y_test * scale
y_pred = y_pred * scale
plt.plot(y_test, color = 'blue', label = '{} Real Stock Price'.format(filename + ' ' + companyNameToTicker[filename]))
plt.plot(y_pred, color = 'red', label = '{} Predicted Stock Price'.format(filename + ' ' + companyNameToTicker[filename]))
plt.title('{} Stock Price Prediction'.format(filename + ' ' + companyNameToTicker[filename]))
plt.xlabel('Time??')
plt.ylabel('{} Stock Prediction'.format(filename + ' ' + companyNameToTicker[filename]))
plt.legend()
# plt.ion()
# plt.pause(0.05)
plt.show()
Here is the output the above code produces...
1/1 [==============================] - ETA: 0s - loss: 0.5488 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (10.187773). Check your callbacks.
1/1 [==============================] - 1s 822ms/step - loss: 0.5488 - accuracy: 0.0476 - val_loss: 0.4729 - val_accuracy: 0.1429
Epoch 3/50
1/1 [==============================] - 1s 517ms/step - loss: 0.5472 - accuracy: 0.0476 - val_loss: 0.4725 - val_accuracy: 0.1429
Epoch 4/50
1/1 [==============================] - 0s 485ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4723 - val_accuracy: 0.1429
Epoch 5/50
1/1 [==============================] - ETA: 0s - loss: 0.5484 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.438490). Check your callbacks.
1/1 [==============================] - 1s 507ms/step - loss: 0.5484 - accuracy: 0.0476 - val_loss: 0.4725 - val_accuracy: 0.1429
Epoch 6/50
1/1 [==============================] - 1s 527ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4732 - val_accuracy: 0.1429
Epoch 7/50
1/1 [==============================] - 0s 413ms/step - loss: 0.5481 - accuracy: 0.0476 - val_loss: 0.4738 - val_accuracy: 0.1429
Epoch 8/50
1/1 [==============================] - 0s 491ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4743 - val_accuracy: 0.1429
Epoch 9/50
1/1 [==============================] - 0s 408ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4748 - val_accuracy: 0.1429
Epoch 10/50
1/1 [==============================] - ETA: 0s - loss: 0.5478 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.434556). Check your callbacks.
1/1 [==============================] - 0s 482ms/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4751 - val_accuracy: 0.1429
Epoch 11/50
1/1 [==============================] - 1s 535ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4754 - val_accuracy: 0.1429
Epoch 12/50
1/1 [==============================] - 0s 408ms/step - loss: 0.5485 - accuracy: 0.0476 - val_loss: 0.4758 - val_accuracy: 0.1429
Epoch 13/50
1/1 [==============================] - 0s 392ms/step - loss: 0.5487 - accuracy: 0.0476 - val_loss: 0.4764 - val_accuracy: 0.1429
Epoch 14/50
1/1 [==============================] - 0s 460ms/step - loss: 0.5488 - accuracy: 0.0476 - val_loss: 0.4768 - val_accuracy: 0.1429
Epoch 15/50
1/1 [==============================] - ETA: 0s - loss: 0.5486 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.388608). Check your callbacks.
1/1 [==============================] - 0s 397ms/step - loss: 0.5486 - accuracy: 0.0476 - val_loss: 0.4770 - val_accuracy: 0.1429
Epoch 16/50
1/1 [==============================] - 1s 573ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4770 - val_accuracy: 0.1429
Epoch 17/50
1/1 [==============================] - 0s 456ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4766 - val_accuracy: 0.1429
Epoch 18/50
1/1 [==============================] - 0s 392ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4763 - val_accuracy: 0.1429
Epoch 19/50
1/1 [==============================] - 0s 404ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4760 - val_accuracy: 0.1429
Epoch 20/50
1/1 [==============================] - ETA: 0s - loss: 0.5479 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.362628). Check your callbacks.
1/1 [==============================] - ETA: 0s - loss: 0.5488 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (10.187773). Check your callbacks.
1/1 [==============================] - 1s 822ms/step - loss: 0.5488 - accuracy: 0.0476 - val_loss: 0.4729 - val_accuracy: 0.1429
Epoch 3/50
1/1 [==============================] - 1s 517ms/step - loss: 0.5472 - accuracy: 0.0476 - val_loss: 0.4725 - val_accuracy: 0.1429
Epoch 4/50
1/1 [==============================] - 0s 485ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4723 - val_accuracy: 0.1429
Epoch 5/50
1/1 [==============================] - ETA: 0s - loss: 0.5484 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.438490). Check your callbacks.
1/1 [==============================] - 1s 507ms/step - loss: 0.5484 - accuracy: 0.0476 - val_loss: 0.4725 - val_accuracy: 0.1429
Epoch 6/50
1/1 [==============================] - 1s 527ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4732 - val_accuracy: 0.1429
Epoch 7/50
1/1 [==============================] - 0s 413ms/step - loss: 0.5481 - accuracy: 0.0476 - val_loss: 0.4738 - val_accuracy: 0.1429
Epoch 8/50
1/1 [==============================] - 0s 491ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4743 - val_accuracy: 0.1429
Epoch 9/50
1/1 [==============================] - 0s 408ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4748 - val_accuracy: 0.1429
Epoch 10/50
1/1 [==============================] - ETA: 0s - loss: 0.5478 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.434556). Check your callbacks.
1/1 [==============================] - 0s 482ms/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4751 - val_accuracy: 0.1429
Epoch 11/50
1/1 [==============================] - 1s 535ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4754 - val_accuracy: 0.1429
Epoch 12/50
1/1 [==============================] - 0s 408ms/step - loss: 0.5485 - accuracy: 0.0476 - val_loss: 0.4758 - val_accuracy: 0.1429
Epoch 13/50
1/1 [==============================] - 0s 392ms/step - loss: 0.5487 - accuracy: 0.0476 - val_loss: 0.4764 - val_accuracy: 0.1429
Epoch 14/50
1/1 [==============================] - 0s 460ms/step - loss: 0.5488 - accuracy: 0.0476 - val_loss: 0.4768 - val_accuracy: 0.1429
Epoch 15/50
1/1 [==============================] - ETA: 0s - loss: 0.5486 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.388608). Check your callbacks.
1/1 [==============================] - 0s 397ms/step - loss: 0.5486 - accuracy: 0.0476 - val_loss: 0.4770 - val_accuracy: 0.1429
Epoch 16/50
1/1 [==============================] - 1s 573ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4770 - val_accuracy: 0.1429
Epoch 17/50
1/1 [==============================] - 0s 456ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4766 - val_accuracy: 0.1429
Epoch 18/50
1/1 [==============================] - 0s 392ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4763 - val_accuracy: 0.1429
Epoch 19/50
1/1 [==============================] - 0s 404ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4760 - val_accuracy: 0.1429
Epoch 20/50
1/1 [==============================] - ETA: 0s - loss: 0.5479 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.362628). Check your callbacks.
1/1 [==============================] - 0s 452ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4758 - val_accuracy: 0.1429
Epoch 21/50
1/1 [==============================] - 0s 473ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4753 - val_accuracy: 0.1429
Epoch 22/50
1/1 [==============================] - 0s 428ms/step - loss: 0.5496 - accuracy: 0.0476 - val_loss: 0.4744 - val_accuracy: 0.1429
Epoch 23/50
1/1 [==============================] - 1s 584ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4741 - val_accuracy: 0.1429
Epoch 24/50
1/1 [==============================] - 0s 446ms/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4743 - val_accuracy: 0.1429
Epoch 25/50
1/1 [==============================] - ETA: 0s - loss: 0.5476 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.530422). Check your callbacks.
1/1 [==============================] - 1s 646ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4746 - val_accuracy: 0.1429
Epoch 26/50
1/1 [==============================] - 1s 506ms/step - loss: 0.5487 - accuracy: 0.0476 - val_loss: 0.4756 - val_accuracy: 0.1429
Epoch 27/50
1/1 [==============================] - 0s 413ms/step - loss: 0.5482 - accuracy: 0.0476 - val_loss: 0.4765 - val_accuracy: 0.1429
Epoch 28/50
1/1 [==============================] - 0s 382ms/step - loss: 0.5481 - accuracy: 0.0476 - val_loss: 0.4772 - val_accuracy: 0.1429
Epoch 29/50
1/1 [==============================] - 0s 421ms/step - loss: 0.5487 - accuracy: 0.0476 - val_loss: 0.4774 - val_accuracy: 0.1429
Epoch 30/50
1/1 [==============================] - ETA: 0s - loss: 0.5483 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.657228). Check your callbacks.
1/1 [==============================] - 1s 955ms/step - loss: 0.5483 - accuracy: 0.0476 - val_loss: 0.4782 - val_accuracy: 0.1429
Epoch 31/50
1/1 [==============================] - 1s 634ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4792 - val_accuracy: 0.1429
Epoch 32/50
1/1 [==============================] - 0s 364ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4800 - val_accuracy: 0.1429
Epoch 33/50
1/1 [==============================] - 0s 404ms/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4808 - val_accuracy: 0.1429
Epoch 34/50
1/1 [==============================] - 0s 381ms/step - loss: 0.5477 - accuracy: 0.0476 - val_loss: 0.4812 - val_accuracy: 0.1429
Epoch 35/50
1/1 [==============================] - ETA: 0s - loss: 0.5476 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.342873). Check your callbacks.
1/1 [==============================] - 1s 524ms/step - loss: 0.5476 - accuracy: 0.0476 - val_loss: 0.4810 - val_accuracy: 0.1429
Epoch 36/50
1/1 [==============================] - 0s 442ms/step - loss: 0.5485 - accuracy: 0.0476 - val_loss: 0.4808 - val_accuracy: 0.1429
Epoch 37/50
1/1 [==============================] - 1s 514ms/step - loss: 0.5493 - accuracy: 0.0476 - val_loss: 0.4805 - val_accuracy: 0.1429
Epoch 38/50
1/1 [==============================] - 1s 630ms/step - loss: 0.5503 - accuracy: 0.0476 - val_loss: 0.4806 - val_accuracy: 0.1429
Epoch 39/50
1/1 [==============================] - ETA: 0s - loss: 0.5478 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.724169). Check your callbacks.
1/1 [==============================] - 1s 1s/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4812 - val_accuracy: 0.1429
Epoch 40/50
1/1 [==============================] - ETA: 0s - loss: 0.5475 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.356633). Check your callbacks.
1/1 [==============================] - 0s 400ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4813 - val_accuracy: 0.1429
Epoch 41/50
1/1 [==============================] - 1s 625ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4814 - val_accuracy: 0.1429
Epoch 42/50
1/1 [==============================] - 1s 671ms/step - loss: 0.5481 - accuracy: 0.0476 - val_loss: 0.4810 - val_accuracy: 0.1429
Epoch 43/50
1/1 [==============================] - 1s 527ms/step - loss: 0.5482 - accuracy: 0.0476 - val_loss: 0.4803 - val_accuracy: 0.1429
Epoch 44/50
1/1 [==============================] - 1s 688ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4797 - val_accuracy: 0.1429
Epoch 45/50
1/1 [==============================] - ETA: 0s - loss: 0.5475 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.479657). Check your callbacks.
1/1 [==============================] - 1s 505ms/step - loss: 0.5475 - accuracy: 0.0476 - val_loss: 0.4789 - val_accuracy: 0.1429
Epoch 46/50
1/1 [==============================] - 1s 637ms/step - loss: 0.5479 - accuracy: 0.0476 - val_loss: 0.4776 - val_accuracy: 0.1429
Epoch 47/50
1/1 [==============================] - 0s 383ms/step - loss: 0.5490 - accuracy: 0.0476 - val_loss: 0.4772 - val_accuracy: 0.1429
Epoch 48/50
1/1 [==============================] - 0s 420ms/step - loss: 0.5486 - accuracy: 0.0476 - val_loss: 0.4769 - val_accuracy: 0.1429
Epoch 49/50
1/1 [==============================] - 0s 428ms/step - loss: 0.5478 - accuracy: 0.0476 - val_loss: 0.4769 - val_accuracy: 0.1429
Epoch 50/50
1/1 [==============================] - ETA: 0s - loss: 0.5482 - accuracy: 0.0476WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.509478). Check your callbacks.
1/1 [==============================] - 0s 417ms/step - loss: 0.5482 - accuracy: 0.0476 - val_loss: 0.4772 - val_accuracy: 0.1429
1 Physical GPUs, 1 Logical GPU
WARNING:tensorflow:Layer lstm_16 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_17 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_18 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_19 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
1/1 [==============================] - 0s 67ms/step - loss: 0.6931 - accuracy: 0.0714
---------- Untrained model, accuracy: 7.14% ----------
WARNING:tensorflow:Layer lstm will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_1 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_2 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU
1/1 [==============================] - 0s 3ms/step - loss: 0.4772 - accuracy: 0.1429
---------- Restored model, accuracy: 14.29% ----------
Shapes of datasets:
X_train: (9538, 1, 7)
y_train: (9538,)
X_test: (3179, 1, 7)
y_test (3179,)
I am building a DNN with keras to classify between background or signal events (HEP). Nevertheless the loss and the accuracy are not changing.
I already tried changing the parameters on the optimizer, normalizing the data, changing the number of layers, neurons, epochs, initializing the weights, etc.
Here's the model:
epochs = 20
num_features = 2
num_classes = 2
batch_size = 32
# model
print("\n Building model...")
model = Sequential()
model.add(Dropout(0.2))
model.add(Dense(128, input_shape=(2,), activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes,activation=tf.nn.softmax))
print("\n Compiling model...")
opt = adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0,
amsgrad=False)
# compile model
model.compile(
loss='sparse_categorical_crossentropy',
optimizer=opt,
metrics=['accuracy'])
print("\n Fitting model...")
history = model.fit(x_train, y_train, epochs = epochs,
batch_size = batch_size, validation_data = (x_test, y_test))
I'm expecting a change in the loss but it won't decrease from 0.69-ish.
The epochs report
Building model...
Compiling model...
Fitting model...
Train on 18400 samples, validate on 4600 samples
Epoch 1/20
18400/18400 [==============================] - 1s 71us/step - loss: 0.6939 - acc: 0.4965 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 2/20
18400/18400 [==============================] - 1s 60us/step - loss: 0.6935 - acc: 0.5045 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 3/20
18400/18400 [==============================] - 1s 69us/step - loss: 0.6937 - acc: 0.4993 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 4/20
18400/18400 [==============================] - 1s 65us/step - loss: 0.6939 - acc: 0.4984 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 5/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6936 - acc: 0.5000 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 6/20
18400/18400 [==============================] - 1s 57us/step - loss: 0.6937 - acc: 0.4913 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 7/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6935 - acc: 0.5008 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 8/20
18400/18400 [==============================] - 1s 63us/step - loss: 0.6936 - acc: 0.5013 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 9/20
18400/18400 [==============================] - 1s 67us/step - loss: 0.6936 - acc: 0.4924 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 10/20
18400/18400 [==============================] - 1s 61us/step - loss: 0.6933 - acc: 0.5067 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 11/20
18400/18400 [==============================] - 1s 64us/step - loss: 0.6938 - acc: 0.4972 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 12/20
18400/18400 [==============================] - 1s 64us/step - loss: 0.6936 - acc: 0.4991 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 13/20
18400/18400 [==============================] - 1s 70us/step - loss: 0.6937 - acc: 0.4960 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 14/20
18400/18400 [==============================] - 1s 63us/step - loss: 0.6935 - acc: 0.4992 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 15/20
18400/18400 [==============================] - 1s 61us/step - loss: 0.6937 - acc: 0.4940 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 16/20
18400/18400 [==============================] - 1s 68us/step - loss: 0.6933 - acc: 0.5067 - val_loss: 0.6936 - val_acc: 0.5000
Epoch 17/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6938 - acc: 0.4997 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 18/20
18400/18400 [==============================] - 1s 56us/step - loss: 0.6936 - acc: 0.4972 - val_loss: 0.6941 - val_acc: 0.5000
Epoch 19/20
18400/18400 [==============================] - 1s 57us/step - loss: 0.6934 - acc: 0.5061 - val_loss: 0.6954 - val_acc: 0.5000
Epoch 20/20
18400/18400 [==============================] - 1s 58us/step - loss: 0.6936 - acc: 0.5037 - val_loss: 0.6939 - val_acc: 0.5000
Update: My data preparation contains this
np.random.shuffle(x_train)
np.random.shuffle(y_train)
np.random.shuffle(x_test)
np.random.shuffle(y_test)
And I'm thinking it's changing the class for each data point cause the shuffle is done separately.