We Keep Coding

Neural network errors don't change

I am training a model using TensorFlow. I was getting weird results when looking at my model performance. I built two models to classify images, one using a CNN and the other using a traditional ANN. Below is the code setup for each of them.
#CNN model
model = Sequential()
model.add(Reshape((20, 60, 3)))
#model.add(Conv2D(128, (5, 5), (2, 2), activation='elu'))
#model.add(Conv2D(64, (4, 4), (2, 2), activation='elu'))
#model.add(Flatten())
#model.add(Dense(1, activation = 'elu'))
#model.add(Dense(25, activation = 'elu'))
#model.add(Dense(10, activation = 'elu'))
#model.add(Dense(1))
opt = keras.optimizers.RMSprop(lr=0.0009, decay=1e-6)
model.compile(Adam(lr = 0.0001), loss='mse', metrics = ['mae'])
history = model.fit(X_train, y_train, epochs = 20, validation_data=(X_val, y_val), batch_size= 32)
#ANN model
model = Sequential()
model.add(Reshape((20, 60, 3)))
#model.add(Flatten())
#model.add(Dense(10, activation = 'elu'))
#model.add(Dense(1))
opt = keras.optimizers.RMSprop(lr=0.0009, decay=1e-6)
model.compile(Adam(lr = 0.0001), loss='mse', metrics = ['mae'])
history = model.fit(X_train, y_train, epochs = 20, validation_data=(X_val, y_val), batch_size= 32)
However, the problem is that I am getting nearly identical loss, and mean absolute error metrics using both of these models, when I am expecting the mae to be MUCH higher for the 2nd model. Does anyone know why this is happening? Could it be something wrong with my input data?
P.S. This network is trying to do regression to predict steering angle for a self-driving rc car from a image
EDIT:
Here is the ending error with the CNN:
Epoch 20/20 113/113 [==============================] - 1s 5ms/step - loss: 0.0382 - mae: 0.1582 - val_loss: 0.0454 - val_mae: 0.1727 dict_keys(['loss', 'mae', 'val_loss', 'val_mae'])
Here is the ending error with the ANN:
Epoch 20/20 113/113 [==============================] - 0s 3ms/step - loss: 0.0789 - mae: 0.2187 - val_loss: 0.0854 - val_mae: 0.2300 dict_keys(['loss', 'mae', 'val_loss', 'val_mae'])

I think the issue is from your training data, try using another data and check the results again

Keras custom loss becomes NaN after a while for regression

I am trying to use a custom loss function for my model. I am scaling y values previously and in my loss function I inverse scale them.(Using the answer from scaling back data in customized keras training loss function) After a random amount of epochs the loss starts to come as NaN also mean_absolute_error val_mean_absolute_error and val_loss are all NaN. Heres my model and custom loss function:
model = Sequential()
model.add(LSTM(units=512, activation="tanh", return_sequences=True, input_shape=(X_train.shape[1],X_train.shape[2])))
model.add(Dropout(0.2))
model.add(LSTM(units=256, activation="tanh", return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=128, activation="tanh", return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=64, activation="tanh"))
model.add(Dropout(0.2))
model.add(Dense(units = 2))
model.compile(optimizer = "Adam", loss = my_loss_function , metrics=['mean_absolute_error'])
model.summary()
I have 2 outputs as you can see.
def my_loss_function(y_actual, y_predicted):
y_actual = (y_actual - K.constant(y_scaler.min_)) / K.constant(y_scaler.scale_)
y_predicted = (y_predicted - K.constant(y_scaler.min_)) / K.constant(y_scaler.scale_)
a_loss = abs(y_actual[0]-y_predicted[0])*128000
b_loss = abs(y_actual[1]-y_predicted[1])*27000
loss= tf.math.sqrt(tf.square(a_loss) + tf.square(b_loss))
return loss
y_scaler is used earlier:
y_scaler = MinMaxScaler(feature_range = (0, 1))
y_scaler.fit(y_data)
y_data=y_scaler.transform(y_data)
y_testdata=y_scaler.transform(y_testdata)
Can anyone help?
When I use MSE, MAE etc. it works fine

CNN accuracy doesn't change through multiple epochs

I'm trying to create a CNN which differentiates b/w pictures of eyes with/without symptoms of diabetic retinopathy. When I tried to run my model, the accuracy didn't improve at all. I tried using different learning rates, but it hasn't worked. Since this is my first time making a CNN, I think I might have made a mistake elsewhere. If you see the problems within my code, please let me know, I would really appreciate the help.
Train on 980 samples, validate on 327 samples
Epoch 1/5
980/980 [==============================] - 777s 792ms/step - loss: 8.1986 - accuracy: 0.4653 - val_loss: 8.8154 - val_accuracy: 0.4251
Epoch 2/5
980/980 [==============================] - 666s 679ms/step - loss: 8.1986 - accuracy: 0.4653 - val_loss: 8.8154 - val_accuracy: 0.4251
Epoch 3/5
980/980 [==============================] - 672s 686ms/step - loss: 8.1986 - accuracy: 0.4653 - val_loss: 8.8154 - val_accuracy: 0.4251
Here is my code:
DATADIR = "C:\\Users.."
CATEGORIES = ["nosymptoms", "symptoms"]
training_data = []
IMG_SIZE = 512
for category in CATEGORIES:
path = os.path.join(DATADIR, category) #brings us to folder with categories
class_num = CATEGORIES.index(category)
for img in os.listdir(path):
img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE)) #img resized and becomes array
training_data.append([new_array, class_num]) #classification is appended to image
import random
random.shuffle(training_data)
for sample in training_data[:10]:
print(sample[1])#0 is the image array, 1 is the label
X = [] #features set
y = [] #labels
X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1)#shape of features (-1 means any), img size, 1 (b/c it is a grayscale)
#Save data
import pickle
pickle_out = open("X.pickle","wb")
pickle.dump(X, pickle_out)
pickle_out.close()
pickle_out = open("y.pickle","wb")
pickle.dump(y, pickle_out)
pickle_out.close()
X = pickle.load(open("X.pickle", "rb"))
y = pickle.load(open("y.pickle", "rb"))
X = X/255.0
model = Sequential()
model.add(Convolution2D(32, (3,3),input_shape=(X.shape[1:]),activation='relu'))
model.add(Convolution2D(32, (3,3),activation='relu'))
model.add(Convolution2D(32, (3,3),activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(16, activation='relu'))
model.add(Dense(12, activation='relu'))
model.add(Dense(1, activation='softmax'))
from keras.optimizers import SGD
opt = SGD(lr=0.01)
model.compile(loss = "binary_crossentropy", optimizer = opt, metrics=['accuracy'])
print(model.summary())
model.fit(X, y, batch_size = 16, epochs = 5, validation_split=.25)

Since you have 2 classes ,change your last layer to 2 neurons
model = Sequential()
model.add(Convolution2D(32, (3,3),input_shape=(X.shape[1:]),activation='relu'))
model.add(Convolution2D(32, (3,3),activation='relu'))
model.add(Convolution2D(32, (3,3),activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(16, activation='relu'))
model.add(Dense(12, activation='relu'))
model.add(Dense(2, activation='softmax'))
from keras.optimizers import SGD
opt = SGD(lr=0.01)
model.compile(loss = "categorical_crossentropy", optimizer = opt, metrics=['accuracy'])
print(model.summary())
model.fit(X, y, batch_size = 16, epochs = 5, validation_split=.25)

model.fit does not take all the samples of X_train

hi I experience a strange problem. I run a model and I expected that fit model take 6783 samples but instead, it takes 212.
so the model is below. First, as you can see X_train must have these values
timesteps = len(X_train[0])
input_dim = len(X_train[0][0])
n_classes = 6
100
6
6783
The model goes like this:
model = Sequential()
model.add(LSTM(64, return_sequences=True, recurrent_regularizer=l2(0.0015), input_shape=(timesteps, input_dim)))
model.add(Dropout(0.5))
model.add(LSTM(64, recurrent_regularizer=l2(0.0015), input_shape=(timesteps, input_dim)))
model.add(Dense(64, activation='relu'))
model.add(Dense(n_classes, activation='softmax'))
model.compile(optimizer=Adam(learning_rate = 0.0025), loss = 'sparse_categorical_crossentropy', metrics = ['accuracy'])
and I fit the model
model.fit(X_train, y_train, batch_size=32, epochs=50)
and I take
212/212 [==============================] - 21s 99ms/step - loss: 1.1047 - accuracy: 0.5210
any ideas? thank you
Update:
I change the batch size to 1 and it takes all the samples. But is this an acceptable behaviour? I use Keras 2.2.4 and TensorFlow 2.2.0

Negative Loss function when using cross_val_score in sklearn/keras. Works when not using k fold

I am trying to implement a model which takes in an array of 167 categorical variables (0 or 1), and outputs an estimated value between 0 and 1. Over 300 datapoints are available.
The model works when using a basic model, below:
classifier = Sequential()
classifier.add(Dense(units = 80, kernel_initializer = 'uniform', activation = 'relu', input_dim = 167))
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
classifier.fit(X_train, y_train, batch_size = 10, epochs = 200)
y_pred = classifier.predict(X_test)
Output is similar to:
Epoch 105/200
253/253 [==============================] - 0s - loss: 0.5582 - acc: 0.0079
Epoch 106/200
253/253 [==============================] - 0s - loss: 0.5583 - acc: 0.0079
Unfortunately, when I try to use cross validation, model stops working, and loss function becomes large and negative. Code is below:
def build_classifier():
classifier = Sequential()
classifier.add(Dense(units = 80, kernel_initializer = 'uniform', activation = 'relu', input_dim = 167))
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
return classifier
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, epochs = 100)
accuracies = cross_val_score(estimator = classifier, X=X_train, y=y_train, cv=3,n_jobs=1)
Output looks like:
Epoch 59/100
168/168 [==============================] - 0s - loss: -1106.9519 - acc: 0.0060
Epoch 60/100
168/168 [==============================] - 0s - loss: -1106.9519 - acc: 0.0060
I have toyed with different parameters, but I cannot seem to find what is causing the issue. Still learning, so any help is very appreciated.

It can happen if the data is sparse. A lot NaNs and Infs can cause this problem. If you are doing a 3 fold validation, it may be possible that in one of the folds, the data selected doesn't contain enough info. Possible solutions can be:
Change the random seed.
Increase the data set.