"Same" network on MATLAB and Keras has very different results - python
I have been trying to replicate the same simple network structure in MATLAB and Keras. The problem is the accuracy I get is very different. MATLAB code gets accuracy near 0.84 and loss near 17 and Keras code gets accuracy near 0.63 and loss near 130, with Keras using double epochs to train and the same data. I think the difference is too big to be a matter of implementation, so I think I'm missing something.
The original code is from a MATLAB example in which I have made a little change to avoid normalization in the first layer.
Here is the MATLAB code:
% Load the digit training set as 4-D array data using
% |digitTrain4DArrayData|.
[trainImages,~,trainAngles] = digitTrain4DArrayData;
disp("Train Images:")
disp(trainImages(:,:,:,1))
% Display 20 random sample training digits using |imshow|.
numTrainImages = size(trainImages,4);
figure
idx = randperm(numTrainImages,20);
for i = 1:numel(idx)
subplot(4,5,i)
imshow(trainImages(:,:,:,idx(i)))
drawnow
end
%%
% Combine all the layers together in a |Layer| array.
layers = [ ...
imageInputLayer([28 28 1], 'Normalization', 'none')
convolution2dLayer(12,25)
reluLayer
fullyConnectedLayer(1)
regressionLayer];
%% Train Network'
options = trainingOptions('sgdm','InitialLearnRate',0.001, ...
'MaxEpochs',15)
net = trainNetwork(trainImages,trainAngles,layers,options)
net.Layers
%% Test Network
[testImages,~,testAngles] = digitTest4DArrayData;
predictedTestAngles = predict(net,testImages);
% *Evaluate Performance*
predictionError = testAngles - predictedTestAngles;
thr = 10;
numCorrect = sum(abs(predictionError) < thr);
numTestImages = size(testImages,4);
accuracy = numCorrect/numTestImages
%%
% Use the root-mean-square error (RMSE) to measure the differences between
% the predicted and actual angles of rotation.
squares = predictionError.^2;
rmse = sqrt(mean(squares))
%%
% *Display Box Plot of Residuals for Each Digit Class*
residuals = testAngles - predictedTestAngles;
residualMatrix = reshape(residuals,500,10);
figure
boxplot(residualMatrix, ...
'Labels',{'0','1','2','3','4','5','6','7','8','9'})
xlabel('Digit Class')
ylabel('Degrees Error')
title('Residuals')
idx = randperm(numTestImages,49);
for i = 1:numel(idx)
image = testImages(:,:,:,idx(i));
predictedAngle = predictedTestAngles(idx(i));
imagesRotated(:,:,:,i) = imrotate(image,predictedAngle,'bicubic','crop');
end
figure
subplot(1,2,1)
montage(testImages(:,:,:,idx))
title('Original')
subplot(1,2,2)
montage(imagesRotated)
title('Corrected')
Here is the Keras code:
import numpy as np
import scipy.io
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense, Activation, Conv2D, BatchNormalization, AveragePooling2D
from keras.optimizers import SGD
from keras.utils import np_utils
from keras import regularizers
np.random.seed(1671) # for reproducibility
# network and training
NB_EPOCH = 30
BATCH_SIZE = 128
VERBOSE = 1
NB_CLASSES = 10 # number of outputs = number of digits
OPTIMIZER = SGD() # SGD optimizer, explained later in this chapter
N_HIDDEN = 128
VALIDATION_SPLIT=0.2 # how much TRAIN is reserved for VALIDATION
data = scipy.io.loadmat('RegressionImageData.mat')
XTrain = np.rollaxis(data['XTrain'],3,0)
XTest = np.rollaxis(data['XTest'],3,0)
YTest = np.squeeze(data['YTest'])
YTrain = np.squeeze(data['YTrain'])
print("Train Images:")
print(XTrain.shape)
print(type(XTrain))
print(XTrain)
XTrain_test = np.reshape(XTrain, (5000,28,28))
with open("./test.txt", "a+") as file:
np.set_printoptions(threshold=np.nan)
file.write(np.array2string(XTrain_test[0], max_line_width=np.inf))
model = Sequential()
model.add(Conv2D(25,(12,12), input_shape=(28,28,1), strides=(1,1), activation = "relu"))
model.add(Flatten())
model.add(Dense(1))
model.summary()
sgd = SGD(lr=0.001, decay=0.1, momentum=0.9, nesterov=False)
model.compile(loss='mean_squared_error', optimizer=sgd)
history = model.fit(XTrain, YTrain,
batch_size=BATCH_SIZE, epochs=NB_EPOCH,
verbose=VERBOSE, validation_split=VALIDATION_SPLIT,
shuffle=False)
predictions= model.predict(XTrain)
[np.transpose(predictions[1:50]), np.transpose(YTrain[1:50]), np.abs(np.transpose(predictions[1:50])- np.transpose(YTrain[1:50]))]
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('rmse')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
Ypred_test= model.predict(XTest)
Ypred_test=np.reshape(Ypred_test, (5000,))
predictionError = Ypred_test - YTest
thr = 10;
numCorrect = np.sum((np.abs(predictionError) < thr)*1)
numValidationImages = len(YTest)
accuracy = numCorrect/numValidationImages
print(accuracy)
squares = np.power(predictionError,2)
rmse = np.sqrt(np.mean(squares))
print(rmse)
Anyone know where could be the gap?
Related
Getting Only Zeros from Model.Predict() When Using Individual Data Points But Getting both 1's and 0's when Using Entire Test Dataset
We trained a binary classification nn with ~90% accuracy/~20% loss. When we use the model by calling model.predict() function with the testing data (which is 20% of our entire dataset), we get a relatively even distribution of 1's and 0's. But when we input individual points from the testing data as a numpy array, we only get 0's regardless of shuffling or not. Can anyone help us explain why we are getting this behavior?
When we use X_test (from the split above) instead of dummyTest, binary_prediction equals 1 for the respective data point array from dummyTest), but when we use dummyTest's data individually, binary_prediction only equals 0.
Full Code Shown Below:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from pandas.plotting import scatter_matrix
from sklearn import model_selection
from keras.utils.np_utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from keras.layers import Dropout
import os
full_dataset = pd.read_csv('noQMarkDataset.csv')
data = full_dataset[~full_dataset.isin(['?'])]
data = data.dropna(axis=0)
data = data.apply(pd.to_numeric)
X = np.array(data.drop(['target'], 1))
y = np.array(data['target'])
mean = X.mean(axis=0)
X -= mean
std = X.std(axis=0)
X /= std
X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, random_state = 0, test_size = 0.2, shuffle=False) #stratify = y
# convert data to categorical labels
Y_train = to_categorical(y_train, num_classes=None)
Y_test = to_categorical(y_test, num_classes=None)
Y_train_binary = y_train.copy()
Y_test_binary = y_test.copy()
Y_train_binary[Y_train_binary > 0] = 1
Y_test_binary[Y_test_binary > 0] = 1
def create_binary_model():
model = Sequential()
model.add(Dense(16, input_dim=7, activation='relu')) # prev input_dim =1 3
model.add(Dropout(0.25))
model.add(Dense(16, activation = 'relu'))
model.add(Dropout(0.25))
model.add(Dense(16))
model.add(Dropout(0.25))
model.add(Dense(12, activation = 'relu'))
model.add(Dense(12, activation = 'relu'))
model.add(Dropout(0.25))
model.add(Dense(8, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(1, activation='sigmoid'))
adam = Adam(lr=0.001)
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy']) #optimizer = 'rmsprop' previously
return model
binary_model = create_binary_model()
history=binary_model.fit(X_train, Y_train_binary, validation_data=(X_test, Y_test_binary), epochs=130, batch_size=15) #epochs = 130, batch_size = 20 previously (best)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'])
plt.show()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'])
plt.show()
from sklearn.metrics import classification_report, accuracy_score
dummyTest = np.array([[67.0,1.0,4.0,0.0,2.0,129.0,2.0]])
binary_pred = np.round(binary_model.predict(dummyTest)).astype(int)
#binary_pred = np.round(binary_model.predict(X_test)).astype(int)
print(f"Binary Pred: {binary_pred}")
binary_model.save(os.path.join(".", "test_model.h5"))
#after loading
from tensorflow.keras.models import load_model
model2 = load_model(os.path.join(".", "test_model.h5"))
binary_pred2 = np.round(binary_model.predict(dummyTest)).astype(int)
#binary_pred2 = np.round(binary_model.predict(X_test)).astype(int)
print(f"Binary Pred2 after load: {binary_pred2}")
Full Dataset Shown Below
age,sex,cp,fbs,restecg,thalach,ca,target
63.0,1.0,1.0,1.0,2.0,150.0,0.0,0
67.0,1.0,4.0,0.0,2.0,108.0,3.0,2
67.0,1.0,4.0,0.0,2.0,129.0,2.0,1
37.0,1.0,3.0,0.0,0.0,187.0,0.0,0
41.0,0.0,2.0,0.0,2.0,172.0,0.0,0
56.0,1.0,2.0,0.0,0.0,178.0,0.0,0
62.0,0.0,4.0,0.0,2.0,160.0,2.0,3
57.0,0.0,4.0,0.0,0.0,163.0,0.0,0
63.0,1.0,4.0,0.0,2.0,147.0,1.0,2
53.0,1.0,4.0,1.0,2.0,155.0,0.0,1
57.0,1.0,4.0,0.0,0.0,148.0,0.0,0
56.0,0.0,2.0,0.0,2.0,153.0,0.0,0
56.0,1.0,3.0,1.0,2.0,142.0,1.0,2
44.0,1.0,2.0,0.0,0.0,173.0,0.0,0
52.0,1.0,3.0,1.0,0.0,162.0,0.0,0
57.0,1.0,3.0,0.0,0.0,174.0,0.0,0
48.0,1.0,2.0,0.0,0.0,168.0,0.0,1
54.0,1.0,4.0,0.0,0.0,160.0,0.0,0
48.0,0.0,3.0,0.0,0.0,139.0,0.0,0
49.0,1.0,2.0,0.0,0.0,171.0,0.0,0
64.0,1.0,1.0,0.0,2.0,144.0,0.0,0
58.0,0.0,1.0,1.0,2.0,162.0,0.0,0
58.0,1.0,2.0,0.0,2.0,160.0,0.0,1
58.0,1.0,3.0,0.0,2.0,173.0,2.0,3
60.0,1.0,4.0,0.0,2.0,132.0,2.0,4
50.0,0.0,3.0,0.0,0.0,158.0,0.0,0
58.0,0.0,3.0,0.0,0.0,172.0,0.0,0
66.0,0.0,1.0,0.0,0.0,114.0,0.0,0
43.0,1.0,4.0,0.0,0.0,171.0,0.0,0
40.0,1.0,4.0,0.0,2.0,114.0,0.0,3
69.0,0.0,1.0,0.0,0.0,151.0,2.0,0
60.0,1.0,4.0,1.0,0.0,160.0,2.0,2
64.0,1.0,3.0,0.0,0.0,158.0,0.0,1
59.0,1.0,4.0,0.0,0.0,161.0,0.0,0
44.0,1.0,3.0,0.0,0.0,179.0,0.0,0
42.0,1.0,4.0,0.0,0.0,178.0,0.0,0
43.0,1.0,4.0,0.0,2.0,120.0,0.0,3
57.0,1.0,4.0,0.0,2.0,112.0,1.0,1
55.0,1.0,4.0,0.0,0.0,132.0,1.0,3
61.0,1.0,3.0,1.0,0.0,137.0,0.0,0
65.0,0.0,4.0,0.0,2.0,114.0,3.0,4
40.0,1.0,1.0,0.0,0.0,178.0,0.0,0
71.0,0.0,2.0,0.0,0.0,162.0,2.0,0
59.0,1.0,3.0,1.0,0.0,157.0,0.0,0
61.0,0.0,4.0,0.0,2.0,169.0,0.0,1
58.0,1.0,3.0,0.0,2.0,165.0,1.0,4
51.0,1.0,3.0,0.0,0.0,123.0,0.0,0
50.0,1.0,4.0,0.0,2.0,128.0,0.0,4
65.0,0.0,3.0,1.0,2.0,157.0,1.0,0
53.0,1.0,3.0,1.0,2.0,152.0,0.0,0
41.0,0.0,2.0,0.0,0.0,168.0,1.0,0
65.0,1.0,4.0,0.0,0.0,140.0,0.0,0
44.0,1.0,4.0,0.0,2.0,153.0,1.0,2
44.0,1.0,2.0,0.0,2.0,188.0,0.0,0
60.0,1.0,4.0,0.0,0.0,144.0,1.0,1
54.0,1.0,4.0,0.0,2.0,109.0,1.0,1
50.0,1.0,3.0,0.0,0.0,163.0,1.0,1
41.0,1.0,4.0,0.0,2.0,158.0,0.0,1
54.0,1.0,3.0,0.0,2.0,152.0,1.0,0
51.0,1.0,1.0,0.0,2.0,125.0,1.0,0
51.0,0.0,4.0,0.0,0.0,142.0,0.0,2
46.0,0.0,3.0,0.0,2.0,160.0,0.0,0
58.0,1.0,4.0,0.0,2.0,131.0,3.0,1
54.0,0.0,3.0,1.0,0.0,170.0,0.0,0
54.0,1.0,4.0,0.0,0.0,113.0,1.0,2
60.0,1.0,4.0,0.0,2.0,142.0,2.0,2
60.0,1.0,3.0,0.0,2.0,155.0,0.0,1
54.0,1.0,3.0,0.0,2.0,165.0,0.0,0
59.0,1.0,4.0,0.0,2.0,140.0,0.0,2
46.0,1.0,3.0,0.0,0.0,147.0,0.0,1
65.0,0.0,3.0,0.0,0.0,148.0,0.0,0
67.0,1.0,4.0,1.0,0.0,163.0,2.0,3
62.0,1.0,4.0,0.0,0.0,99.0,2.0,1
65.0,1.0,4.0,0.0,2.0,158.0,2.0,1
44.0,1.0,4.0,0.0,2.0,177.0,1.0,1
65.0,0.0,3.0,0.0,2.0,151.0,0.0,0
60.0,1.0,4.0,0.0,2.0,141.0,1.0,1
51.0,0.0,3.0,0.0,2.0,142.0,1.0,0
48.0,1.0,2.0,0.0,2.0,180.0,0.0,0
58.0,1.0,4.0,0.0,2.0,111.0,0.0,3
45.0,1.0,4.0,0.0,2.0,148.0,0.0,0
53.0,0.0,4.0,0.0,2.0,143.0,0.0,0
39.0,1.0,3.0,0.0,2.0,182.0,0.0,0
68.0,1.0,3.0,1.0,2.0,150.0,0.0,3
52.0,1.0,2.0,0.0,0.0,172.0,0.0,0
44.0,1.0,3.0,0.0,2.0,180.0,0.0,0
47.0,1.0,3.0,0.0,2.0,156.0,0.0,0
53.0,0.0,3.0,0.0,2.0,115.0,0.0,0
53.0,0.0,4.0,0.0,2.0,160.0,0.0,0
51.0,0.0,3.0,0.0,2.0,149.0,0.0,0
66.0,1.0,4.0,0.0,2.0,151.0,0.0,0
62.0,0.0,4.0,0.0,2.0,145.0,3.0,3
62.0,1.0,3.0,0.0,0.0,146.0,3.0,0
44.0,0.0,3.0,0.0,0.0,175.0,0.0,0
63.0,0.0,3.0,0.0,2.0,172.0,0.0,0
52.0,1.0,4.0,0.0,0.0,161.0,1.0,1
59.0,1.0,4.0,0.0,2.0,142.0,1.0,2
60.0,0.0,4.0,0.0,2.0,157.0,2.0,3
52.0,1.0,2.0,0.0,0.0,158.0,1.0,0
48.0,1.0,4.0,0.0,2.0,186.0,0.0,0
45.0,1.0,4.0,0.0,2.0,185.0,0.0,0
34.0,1.0,1.0,0.0,2.0,174.0,0.0,0
57.0,0.0,4.0,0.0,2.0,159.0,1.0,0
71.0,0.0,3.0,1.0,2.0,130.0,1.0,0
49.0,1.0,3.0,0.0,0.0,139.0,3.0,3
54.0,1.0,2.0,0.0,0.0,156.0,0.0,0
59.0,1.0,4.0,0.0,0.0,162.0,1.0,2
57.0,1.0,3.0,0.0,2.0,150.0,1.0,1
61.0,1.0,4.0,0.0,0.0,140.0,1.0,2
39.0,1.0,4.0,0.0,0.0,140.0,0.0,3
61.0,0.0,4.0,0.0,2.0,146.0,0.0,1
56.0,1.0,4.0,1.0,2.0,144.0,1.0,1
52.0,1.0,1.0,0.0,2.0,190.0,0.0,0
43.0,0.0,4.0,1.0,2.0,136.0,0.0,2
62.0,0.0,3.0,0.0,0.0,97.0,1.0,2
41.0,1.0,2.0,0.0,0.0,132.0,0.0,0
58.0,1.0,3.0,1.0,2.0,165.0,0.0,0
35.0,0.0,4.0,0.0,0.0,182.0,0.0,0
63.0,1.0,4.0,1.0,2.0,132.0,3.0,3
65.0,1.0,4.0,0.0,2.0,127.0,1.0,2
48.0,1.0,4.0,1.0,2.0,150.0,2.0,3
63.0,0.0,4.0,0.0,2.0,154.0,3.0,4
51.0,1.0,3.0,0.0,0.0,143.0,0.0,0
55.0,1.0,4.0,0.0,0.0,111.0,0.0,3
65.0,1.0,1.0,1.0,2.0,174.0,1.0,1
45.0,0.0,2.0,0.0,2.0,175.0,0.0,0
56.0,0.0,4.0,1.0,2.0,133.0,2.0,3
54.0,1.0,4.0,0.0,0.0,126.0,1.0,3
44.0,1.0,2.0,0.0,0.0,170.0,0.0,0
62.0,0.0,4.0,0.0,0.0,163.0,0.0,0
54.0,1.0,3.0,0.0,2.0,147.0,0.0,0
51.0,1.0,3.0,0.0,0.0,154.0,1.0,0
29.0,1.0,2.0,0.0,2.0,202.0,0.0,0
51.0,1.0,4.0,0.0,2.0,186.0,0.0,0
43.0,0.0,3.0,0.0,0.0,165.0,0.0,0
55.0,0.0,2.0,0.0,2.0,161.0,0.0,0
70.0,1.0,4.0,0.0,0.0,125.0,0.0,4
62.0,1.0,2.0,0.0,2.0,103.0,1.0,3
35.0,1.0,4.0,0.0,0.0,130.0,0.0,1
51.0,1.0,3.0,1.0,2.0,166.0,0.0,0
59.0,1.0,2.0,0.0,0.0,164.0,0.0,0
59.0,1.0,1.0,0.0,2.0,159.0,0.0,1
52.0,1.0,2.0,1.0,0.0,184.0,0.0,0
64.0,1.0,3.0,0.0,0.0,131.0,0.0,1
58.0,1.0,3.0,0.0,2.0,154.0,0.0,0
47.0,1.0,3.0,0.0,0.0,152.0,0.0,1
57.0,1.0,4.0,1.0,2.0,124.0,3.0,4
41.0,1.0,3.0,0.0,0.0,179.0,0.0,0
45.0,1.0,2.0,0.0,2.0,170.0,0.0,0
60.0,0.0,3.0,0.0,0.0,160.0,1.0,0
52.0,1.0,1.0,1.0,0.0,178.0,0.0,0
42.0,0.0,4.0,0.0,2.0,122.0,0.0,0
67.0,0.0,3.0,0.0,2.0,160.0,0.0,0
55.0,1.0,4.0,0.0,2.0,145.0,1.0,4
64.0,1.0,4.0,0.0,2.0,96.0,1.0,3
70.0,1.0,4.0,0.0,2.0,109.0,3.0,1
51.0,1.0,4.0,0.0,0.0,173.0,0.0,1
58.0,1.0,4.0,0.0,2.0,171.0,2.0,1
60.0,1.0,4.0,0.0,2.0,170.0,2.0,2
68.0,1.0,3.0,0.0,0.0,151.0,1.0,0
46.0,1.0,2.0,1.0,0.0,156.0,0.0,0
77.0,1.0,4.0,0.0,2.0,162.0,3.0,4
54.0,0.0,3.0,0.0,0.0,158.0,0.0,0
58.0,0.0,4.0,0.0,2.0,122.0,0.0,0
48.0,1.0,3.0,1.0,0.0,175.0,2.0,0
57.0,1.0,4.0,0.0,0.0,168.0,0.0,0
54.0,0.0,2.0,1.0,2.0,159.0,1.0,0
35.0,1.0,4.0,0.0,2.0,156.0,0.0,1
45.0,0.0,2.0,0.0,0.0,138.0,0.0,0
70.0,1.0,3.0,0.0,0.0,112.0,1.0,3
53.0,1.0,4.0,0.0,2.0,111.0,0.0,0
59.0,0.0,4.0,0.0,0.0,143.0,0.0,1
62.0,0.0,4.0,0.0,2.0,157.0,0.0,0
64.0,1.0,4.0,0.0,2.0,132.0,2.0,4
57.0,1.0,4.0,0.0,0.0,88.0,1.0,1
52.0,1.0,4.0,1.0,0.0,147.0,3.0,0
56.0,1.0,4.0,0.0,2.0,105.0,1.0,1
43.0,1.0,3.0,0.0,0.0,162.0,1.0,0
53.0,1.0,3.0,1.0,2.0,173.0,3.0,0
48.0,1.0,4.0,0.0,2.0,166.0,0.0,3
56.0,0.0,4.0,0.0,2.0,150.0,2.0,2
42.0,1.0,1.0,0.0,2.0,178.0,2.0,0
59.0,1.0,1.0,0.0,2.0,145.0,0.0,0
60.0,0.0,4.0,0.0,2.0,161.0,0.0,1
63.0,0.0,2.0,0.0,0.0,179.0,2.0,0
42.0,1.0,3.0,1.0,0.0,194.0,0.0,0
66.0,1.0,2.0,0.0,0.0,120.0,3.0,2
54.0,1.0,2.0,0.0,2.0,195.0,1.0,1
69.0,1.0,3.0,0.0,2.0,146.0,3.0,2
50.0,1.0,3.0,0.0,0.0,163.0,0.0,0
51.0,1.0,4.0,0.0,0.0,122.0,3.0,3
62.0,0.0,4.0,1.0,0.0,106.0,3.0,2
68.0,0.0,3.0,0.0,2.0,115.0,0.0,0
67.0,1.0,4.0,0.0,2.0,125.0,2.0,3
69.0,1.0,1.0,1.0,2.0,131.0,1.0,0
45.0,0.0,4.0,0.0,2.0,152.0,0.0,0
50.0,0.0,2.0,0.0,0.0,162.0,0.0,0
59.0,1.0,1.0,0.0,2.0,125.0,0.0,1
50.0,0.0,4.0,0.0,2.0,159.0,0.0,0
64.0,0.0,4.0,0.0,0.0,154.0,0.0,0
57.0,1.0,3.0,1.0,0.0,173.0,1.0,0
64.0,0.0,3.0,0.0,0.0,133.0,0.0,0
43.0,1.0,4.0,0.0,0.0,161.0,0.0,0
45.0,1.0,4.0,0.0,2.0,147.0,3.0,3
58.0,1.0,4.0,0.0,2.0,130.0,2.0,3
50.0,1.0,4.0,0.0,2.0,126.0,0.0,3
55.0,1.0,2.0,0.0,0.0,155.0,0.0,0
62.0,0.0,4.0,0.0,0.0,154.0,0.0,1
37.0,0.0,3.0,0.0,0.0,170.0,0.0,0
38.0,1.0,1.0,0.0,0.0,182.0,0.0,4
41.0,1.0,3.0,0.0,2.0,168.0,0.0,0
66.0,0.0,4.0,1.0,0.0,165.0,2.0,3
52.0,1.0,4.0,0.0,0.0,160.0,1.0,1
56.0,1.0,1.0,0.0,2.0,162.0,0.0,0
46.0,0.0,2.0,0.0,0.0,172.0,0.0,0
46.0,0.0,4.0,0.0,2.0,152.0,0.0,0
64.0,0.0,4.0,0.0,0.0,122.0,2.0,0
59.0,1.0,4.0,0.0,2.0,182.0,0.0,0
41.0,0.0,3.0,0.0,2.0,172.0,0.0,0
54.0,0.0,3.0,0.0,2.0,167.0,0.0,0
39.0,0.0,3.0,0.0,0.0,179.0,0.0,0
53.0,1.0,4.0,0.0,0.0,95.0,2.0,3
63.0,0.0,4.0,0.0,0.0,169.0,2.0,1
34.0,0.0,2.0,0.0,0.0,192.0,0.0,0
47.0,1.0,4.0,0.0,0.0,143.0,0.0,0
67.0,0.0,3.0,0.0,0.0,172.0,1.0,0
54.0,1.0,4.0,0.0,2.0,108.0,1.0,3
66.0,1.0,4.0,0.0,2.0,132.0,1.0,2
52.0,0.0,3.0,0.0,2.0,169.0,0.0,0
55.0,0.0,4.0,0.0,1.0,117.0,0.0,2
49.0,1.0,3.0,0.0,2.0,126.0,3.0,1
74.0,0.0,2.0,0.0,2.0,121.0,1.0,0
54.0,0.0,3.0,0.0,0.0,163.0,1.0,0
54.0,1.0,4.0,0.0,2.0,116.0,2.0,3
56.0,1.0,4.0,1.0,2.0,103.0,0.0,2
46.0,1.0,4.0,0.0,2.0,144.0,0.0,1
49.0,0.0,2.0,0.0,0.0,162.0,0.0,0
42.0,1.0,2.0,0.0,0.0,162.0,0.0,0
41.0,1.0,2.0,0.0,0.0,153.0,0.0,0
41.0,0.0,2.0,0.0,0.0,163.0,0.0,0
49.0,0.0,4.0,0.0,0.0,163.0,0.0,0
61.0,1.0,1.0,0.0,0.0,145.0,2.0,2
60.0,0.0,3.0,1.0,0.0,96.0,0.0,0
67.0,1.0,4.0,0.0,0.0,71.0,0.0,2
58.0,1.0,4.0,0.0,0.0,156.0,1.0,2
47.0,1.0,4.0,0.0,2.0,118.0,1.0,1
52.0,1.0,4.0,0.0,0.0,168.0,2.0,3
62.0,1.0,2.0,1.0,2.0,140.0,0.0,0
57.0,1.0,4.0,0.0,0.0,126.0,0.0,0
58.0,1.0,4.0,0.0,0.0,105.0,1.0,1
64.0,1.0,4.0,0.0,0.0,105.0,1.0,0
51.0,0.0,3.0,0.0,2.0,157.0,0.0,0
43.0,1.0,4.0,0.0,0.0,181.0,0.0,0
42.0,0.0,3.0,0.0,0.0,173.0,0.0,0
67.0,0.0,4.0,0.0,0.0,142.0,2.0,0
76.0,0.0,3.0,0.0,1.0,116.0,0.0,0
70.0,1.0,2.0,0.0,2.0,143.0,0.0,0
57.0,1.0,2.0,0.0,0.0,141.0,0.0,1
44.0,0.0,3.0,0.0,0.0,149.0,1.0,0
58.0,0.0,2.0,1.0,2.0,152.0,2.0,3
60.0,0.0,1.0,0.0,0.0,171.0,0.0,0
44.0,1.0,3.0,0.0,0.0,169.0,0.0,0
61.0,1.0,4.0,0.0,2.0,125.0,1.0,4
42.0,1.0,4.0,0.0,0.0,125.0,0.0,2
52.0,1.0,4.0,1.0,0.0,156.0,0.0,2
59.0,1.0,3.0,1.0,0.0,134.0,1.0,2
40.0,1.0,4.0,0.0,0.0,181.0,0.0,1
42.0,1.0,3.0,0.0,0.0,150.0,0.0,0
61.0,1.0,4.0,0.0,2.0,138.0,1.0,1
66.0,1.0,4.0,0.0,2.0,138.0,0.0,0
46.0,1.0,4.0,0.0,0.0,120.0,2.0,2
71.0,0.0,4.0,0.0,0.0,125.0,0.0,0
59.0,1.0,1.0,0.0,0.0,162.0,2.0,1
64.0,1.0,1.0,0.0,2.0,155.0,0.0,0
66.0,0.0,3.0,0.0,2.0,152.0,1.0,0
39.0,0.0,3.0,0.0,0.0,152.0,0.0,0
57.0,1.0,2.0,0.0,2.0,164.0,1.0,1
58.0,0.0,4.0,0.0,0.0,131.0,0.0,0
57.0,1.0,4.0,0.0,0.0,143.0,1.0,2
47.0,1.0,3.0,0.0,0.0,179.0,0.0,0
55.0,0.0,4.0,0.0,1.0,130.0,1.0,3
35.0,1.0,2.0,0.0,0.0,174.0,0.0,0
61.0,1.0,4.0,0.0,0.0,161.0,1.0,2
58.0,1.0,4.0,0.0,1.0,140.0,3.0,4
58.0,0.0,4.0,1.0,2.0,146.0,2.0,2
56.0,1.0,2.0,0.0,2.0,163.0,0.0,0
56.0,1.0,2.0,0.0,0.0,169.0,0.0,0
67.0,1.0,3.0,0.0,2.0,150.0,0.0,1
55.0,0.0,2.0,0.0,0.0,166.0,0.0,0
44.0,1.0,4.0,0.0,0.0,144.0,0.0,2
63.0,1.0,4.0,0.0,2.0,144.0,2.0,2
63.0,0.0,4.0,0.0,0.0,136.0,0.0,1
41.0,1.0,2.0,0.0,0.0,182.0,0.0,0
59.0,1.0,4.0,1.0,2.0,90.0,2.0,3
57.0,0.0,4.0,0.0,0.0,123.0,0.0,1
45.0,1.0,1.0,0.0,0.0,132.0,0.0,1
68.0,1.0,4.0,1.0,0.0,141.0,2.0,2
57.0,1.0,4.0,0.0,0.0,115.0,1.0,3
57.0,0.0,2.0,0.0,2.0,174.0,1.0,1
66.0,0.0,3.0,0.0,2.0,152.0,1.0,0
39.0,0.0,3.0,0.0,0.0,152.0,0.0,0
57.0,1.0,2.0,0.0,2.0,164.0,1.0,1
58.0,0.0,4.0,0.0,0.0,131.0,0.0,0
57.0,1.0,4.0,0.0,0.0,143.0,1.0,2
47.0,1.0,3.0,0.0,0.0,179.0,0.0,0
55.0,0.0,4.0,0.0,1.0,130.0,1.0,3
35.0,1.0,2.0,0.0,0.0,174.0,0.0,0
61.0,1.0,4.0,0.0,0.0,161.0,1.0,2
58.0,1.0,4.0,0.0,1.0,140.0,3.0,4
46.0,1.0,4.0,0.0,0.0,120.0,2.0,2
71.0,0.0,4.0,0.0,0.0,125.0,0.0,0
59.0,1.0,1.0,0.0,0.0,162.0,2.0,1
64.0,1.0,1.0,0.0,2.0,155.0,0.0,0
66.0,0.0,3.0,0.0,2.0,152.0,1.0,0
39.0,0.0,3.0,0.0,0.0,152.0,0.0,0
57.0,1.0,2.0,0.0,2.0,164.0,1.0,1
58.0,0.0,4.0,0.0,0.0,131.0,0.0,0
57.0,1.0,4.0,0.0,0.0,143.0,1.0,2
47.0,1.0,3.0,0.0,0.0,179.0,0.0,0
55.0,0.0,4.0,0.0,1.0,130.0,1.0,3
35.0,1.0,2.0,0.0,0.0,174.0,0.0,0
61.0,1.0,4.0,0.0,0.0,161.0,1.0,2
58.0,1.0,4.0,0.0,1.0,140.0,3.0,4
58.0,0.0,4.0,1.0,2.0,146.0,2.0,2
39.0,0.0,3.0,0.0,0.0,179.0,0.0,0
53.0,1.0,4.0,0.0,0.0,95.0,2.0,3
63.0,0.0,4.0,0.0,0.0,169.0,2.0,1
34.0,0.0,2.0,0.0,0.0,192.0,0.0,0
47.0,1.0,4.0,0.0,0.0,143.0,0.0,0
67.0,0.0,3.0,0.0,0.0,172.0,1.0,0
54.0,1.0,4.0,0.0,2.0,108.0,1.0,3
66.0,1.0,4.0,0.0,2.0,132.0,1.0,2
52.0,0.0,3.0,0.0,2.0,169.0,0.0,0
55.0,0.0,4.0,0.0,1.0,117.0,0.0,2
49.0,1.0,3.0,0.0,2.0,126.0,3.0,1
74.0,0.0,2.0,0.0,2.0,121.0,1.0,0
54.0,0.0,3.0,0.0,0.0,163.0,1.0,0
54.0,1.0,4.0,0.0,2.0,116.0,2.0,3
56.0,1.0,4.0,1.0,2.0,103.0,0.0,2
46.0,1.0,4.0,0.0,2.0,144.0,0.0,1
49.0,0.0,2.0,0.0,0.0,162.0,0.0,0
42.0,1.0,2.0,0.0,0.0,162.0,0.0,0
45.0,1.0,4.0,0.0,2.0,185.0,0.0,0
34.0,1.0,1.0,0.0,2.0,174.0,0.0,0
57.0,0.0,4.0,0.0,2.0,159.0,1.0,0
71.0,0.0,3.0,1.0,2.0,130.0,1.0,0
49.0,1.0,3.0,0.0,0.0,139.0,3.0,3
54.0,1.0,2.0,0.0,0.0,156.0,0.0,0
59.0,1.0,4.0,0.0,0.0,162.0,1.0,2
57.0,1.0,3.0,0.0,2.0,150.0,1.0,1
61.0,1.0,4.0,0.0,0.0,140.0,1.0,2
39.0,1.0,4.0,0.0,0.0,140.0,0.0,3
61.0,0.0,4.0,0.0,2.0,146.0,0.0,1
56.0,1.0,4.0,1.0,2.0,144.0,1.0,1
52.0,1.0,1.0,0.0,2.0,190.0,0.0,0
43.0,0.0,4.0,1.0,2.0,136.0,0.0,2
67.0,1.0,4.0,0.0,2.0,108.0,3.0,2
67.0,1.0,4.0,0.0,2.0,129.0,2.0,1
37.0,1.0,3.0,0.0,0.0,187.0,0.0,0
41.0,0.0,2.0,0.0,2.0,172.0,0.0,0
56.0,1.0,2.0,0.0,0.0,178.0,0.0,0
62.0,0.0,4.0,0.0,2.0,160.0,2.0,3
57.0,0.0,4.0,0.0,0.0,163.0,0.0,0
63.0,1.0,4.0,0.0,2.0,147.0,1.0,2
53.0,1.0,4.0,1.0,2.0,155.0,0.0,1
57.0,1.0,4.0,0.0,0.0,148.0,0.0,0
56.0,0.0,2.0,0.0,2.0,153.0,0.0,0
56.0,1.0,3.0,1.0,2.0,142.0,1.0,2
44.0,1.0,2.0,0.0,0.0,173.0,0.0,0
52.0,1.0,3.0,1.0,0.0,162.0,0.0,0
57.0,1.0,3.0,0.0,0.0,174.0,0.0,0
48.0,1.0,2.0,0.0,0.0,168.0,0.0,1
54.0,1.0,4.0,0.0,0.0,160.0,0.0,0
48.0,0.0,3.0,0.0,0.0,139.0,0.0,0
50.0,1.0,2.0,0.0,0.0,160.0,0.0,0
54.0,0.0,2.0,0.0,0.0,150.0,1.0,0
54.0,0.0,2.0,0.0,1.0,155.0,1.0,0
54.0,0.0,2.0,0.0,0.0,130.0,1.0,0
54.0,0.0,2.0,0.0,0.0,130.0,1.0,0
54.0,1.0,1.0,0.0,0.0,137.0,1.0,0
54.0,1.0,2.0,0.0,0.0,142.0,1.0,0
54.0,1.0,2.0,0.0,0.0,154.0,1.0,0
54.0,1.0,2.0,0.0,0.0,110.0,1.0,0
54.0,1.0,2.0,0.0,1.0,130.0,1.0,0
59.0,1.0,4.0,0.0,0.0,140.0,0.0,0
47.0,1.0,4.0,0.0,0.0,98.0,0.0,1
56.0,1.0,4.0,0.0,0.0,120.0,0.0,1
59.0,1.0,4.0,0.0,1.0,131.0,0.0,0
You are training your model with normalized data but predicting on data that is not normalized. X_test is normalized data so the predictions on it are as expected. dummyTest is not normalized. If you normalize the dummyTest variable before feeding it to your neural network like so: dummyTest[0] -= mean dummyTest[0] /= std You will receive the expected output(1)
LSTM model has constant accuracy and doesn't variate
i'm stuck as you can see, with my lstm model. I'm trying to predict the amount of tons to produce per month. When i run the model to train the accuracy is almost constant, it has a minimal variation like:
0.34406
0.34407
0.34408
I tried different combination of activations, initializers and parameters, and the acc don't increase.
I don't know if the problem here is my data, my model or this value is the max acc the model can reach.
Here is the code (if you notice some libraries unused, its because i made some changes by the first version)
import numpy as np
import pandas as pd
from pandas.tseries.offsets import DateOffset
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
from sklearn import preprocessing
import keras
%tensorflow_version 2.x
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Dropout
from keras.optimizers import Adam
import warnings
warnings.filterwarnings("ignore")
%matplotlib inline
from plotly.offline import iplot
import matplotlib.pyplot as plt
import chart_studio.plotly as py
import plotly.offline as pyoff
import plotly.graph_objs as go
df_ventas = pd.read_csv('/content/drive/My Drive/proyectoPanimex/DEOPE.csv', parse_dates=['Data Emissão'], index_col=0, squeeze=True)
#df_ventas = df_ventas.resample('M').sum().reset_index()
df_ventas = df_ventas.drop(columns= ['weekday', 'month'], axis=1)
df_ventas = df_ventas.reset_index()
df_ventas = df_ventas.rename(columns= {'Data Emissão':'Fecha','Un':'Cantidad'})
df_ventas['dia'] = [x.day for x in df_ventas.Fecha]
df_ventas['mes']=[x.month for x in df_ventas.Fecha]
df_ventas['anio']=[x.year for x in df_ventas.Fecha]
df_ventas = df_ventas[:-48]
df_ventas = df_ventas.drop(columns='Fecha')
df_diff = df_ventas.copy()
df_diff['cantidad_anterior'] = df_diff['Cantidad'].shift(1)
df_diff = df_diff.dropna()
df_diff['diferencia'] = (df_diff['Cantidad'] - df_diff['cantidad_anterior'])
df_supervised = df_diff.drop(['cantidad_anterior'],axis=1)
#adding lags
for inc in range(1,31):
nombre_columna = 'retraso_' + str(inc)
df_supervised[nombre_columna] = df_supervised['diferencia'].shift(inc)
df_supervised = df_supervised.dropna()
df_supervisedNumpy = df_supervised.to_numpy()
train = df_supervisedNumpy
scaler = MinMaxScaler(feature_range=(0, 1))
X_train = scaler.fit(train)
train = train.reshape(train.shape[0], train.shape[1])
train_scaled = scaler.transform(train)
X_train, y_train = train_scaled[:, 1:], train_scaled[:, 0:1]
X_train = X_train.reshape(X_train.shape[0], 1, X_train.shape[1])
#LSTM MODEL
model = Sequential()
act = 'tanh'
actF = 'relu'
model.add(LSTM(200, activation = act, input_dim=34, return_sequences=True ))
model.add(Dropout(0.15))
#model.add(Flatten())
model.add(LSTM(200, activation= act))
model.add(Dropout(0.2))
#model.add(Flatten())
model.add(Dense(200, activation= act))
model.add(Dropout(0.3))
model.add(Dense(1, activation= actF))
optimizer = keras.optimizers.Adam(lr=0.00001)
model.compile(optimizer=optimizer, loss=keras.losses.binary_crossentropy, metrics=['accuracy'])
history = model.fit(X_train, y_train, batch_size = 100,
epochs = 50, verbose = 1)
hist = pd.DataFrame(history.history)
hist['Epoch'] = history.epoch
hist
History plot:
loss acc Epoch
0 0.847146 0.344070 0
1 0.769400 0.344070 1
2 0.703548 0.344070 2
3 0.698137 0.344070 3
4 0.653952 0.344070 4
As you can see the only value that change its loss, but what is going on with Acc?. I'm starting with machine learning, and i have no more knowledge to can see my errors. Thanks!
A Dense(1, activation='softmax') will always freeze and not learn anything A Dense(1, activation='relu') will very probably freeze and not learn anything A Dense(1, activation='sigmoid') is ideal for classification (binary) problems and somewhat good for regression with values between 0 and 1. A Dense(1, activation='tanh') is somewhat good for regression with values between -1 and 1 A Dense(1, activation='softplus') is somewhat good for regression with values between 0 and +infinite A Dense(1, actiavation='linear') is good for regression in general with no limits (but it's highly recommended that the data be normalized before) For regression, you can't use accuracy, but the metrics 'mae' and 'mse' don't provide "relative" difference, they provide "absolute" mean difference, one linear, the other squared.
Your output activation should be linear for continuous prediction or softmax for classification. Also multiply your learning rate by 100. Your loss should be mean_absolute_error. You could also easily divide your lstm neurons by a factor of 10. The tanh should be replaced by relu or the likes. For your accuracy problem, it makes no sense to use accuracy, since you're not trying to classify. For metrics, you can use mae. You're trying to know how far the prediction is from the actual target, on a continuous scale. Accuracy is for categories, not continuous data.
accuracy loss when running inference on edge tpu with keras neural network
I'm a student at the University of Bologna ( Italy) and I'm using the Google Coral USB accelerator for my thesis.
I realized a keras neural network that classifies my data in four classes and the accuracy I get is roughly 97%.
I performed a full integer post- training quantization since keras networks don't support quantization-aware training. I followed the guide on TensorFlow site but I had problems running inference on the edge tpu .
In particular my network undergoes an accuracy loss when the model is converted to a tensorflowlite one ( the accuracy drops to roughly the 25%) . This is due to quantization since the tensorflowlite model without quantization that runs on my pc is not affected by the conversion.
I tried to scale my input data in a range [0, 255] with MinMaxScaler but in this case , even if the accuracy of the tensorflowlite quantized model matches the one of the not converted one , the results are not satisfactory since the accuracy of the network itself is low.
I wonder if you could help me solve this problem. Perhaps the values on my dataset are too low and the quantization fails to convert float32 into uint8 without information loss.
Below you'll find my python code.
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from numpy import loadtxt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.model_selection import train_test_split
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.optimizers import RMSprop
from sklearn.preprocessing import StandardScaler,MinMaxScaler,Normalizer
from sklearn.metrics import confusion_matrix, accuracy_score
from tensorflow.keras.callbacks import EarlyStopping
import os
import time
import tflite_runtime.interpreter as tflite
import collections
import operator
"""Functions to work with classification models."""
Class = collections.namedtuple('Class', ['id', 'score'])
def input_tensor(interpreter):
"""Returns input tensor view as numpy array of shape (height, width, 3)."""
tensor_index = interpreter.get_input_details()[0]['index']
return interpreter.tensor(tensor_index)()[0]
def output_tensor(interpreter):
"""Returns dequantized output tensor."""
output_details = interpreter.get_output_details()[0]
output_data = np.squeeze(interpreter.tensor(output_details['index'])()) #Remove single-dimensional entries from the shape of an array.
scale, zero_point = output_details['quantization']
return scale * (output_data - zero_point)
def set_input(interpreter, data):
"""Copies data to input tensor."""
input_tensor(interpreter)[:] = data
return data
def get_output(interpreter, top_k=1, score_threshold=0.0):
"""Returns no more than top_k classes with score >= score_threshold."""
scores = output_tensor(interpreter)
classes = [
Class(i, scores[i])
for i in np.argpartition(scores, -top_k)[-top_k:]
if scores[i] >= score_threshold
]
return sorted(classes, key=operator.itemgetter(1), reverse=True)
#load the dataset
Modelli_Prova01_Nom01_Acc1L = loadtxt(r'/home/utente/Scrivania/csvtesi/Modelli_Prova01_Nom01_Acc1L.csv',delimiter=',')
Modelli_Prova02_Nom01_Acc1L = loadtxt(r'/home/utente/Scrivania/csvtesi/Modelli_Prova02_Nom01_Acc1L.csv',delimiter=',')
Modelli_Prova03_Nom01_Acc1L = loadtxt(r'/home/utente/Scrivania/csvtesi/Modelli_Prova03_Nom01_Acc1L.csv',delimiter=',')
Modelli_Prova04_Nom01_Acc1L = loadtxt(r'/home/utente/Scrivania/csvtesi/Modelli_Prova04_Nom01_Acc1L.csv',delimiter=',')
Modelli_Prova05_Nom01_Acc1L = loadtxt(r'/home/utente/Scrivania/csvtesi/Modelli_Prova05_Nom01_Acc1L.csv',delimiter=',')
time_start = time.perf_counter()
#split x and y data (train and test)
Acc1L01_train,Acc1L01_test = train_test_split(Modelli_Prova01_Nom01_Acc1L ,test_size=0.015,random_state=42)
Acc1L02_train,Acc1L02_test = train_test_split(Modelli_Prova02_Nom01_Acc1L,test_size=0.3,random_state=42)
Acc1L03_train,Acc1L03_test = train_test_split(Modelli_Prova03_Nom01_Acc1L,test_size=0.3,random_state=42)
Acc1L04_train,Acc1L04_test = train_test_split(Modelli_Prova04_Nom01_Acc1L,test_size=0.3,random_state=42)
Acc1L05_train,Acc1L05_test = train_test_split(Modelli_Prova05_Nom01_Acc1L,test_size=0.15,random_state=42)
Y1_train= np.zeros([len(Acc1L01_train)+len(Acc1L05_train),1])
Y2_train= np.ones([len(Acc1L02_train),1])
Y3_train= np.ones([len(Acc1L03_train),1]) +1
Y4_train= np.ones([len(Acc1L04_train),1]) +2
Y1_test= np.zeros([len(Acc1L01_test)+len(Acc1L05_test),1])
Y2_test= np.ones([len(Acc1L02_test),1])
Y3_test= np.ones([len(Acc1L03_test),1]) +1
Y4_test= np.ones([len(Acc1L04_test),1]) +2
xAcc1L_train = np.concatenate((Acc1L01_train,Acc1L05_train,Acc1L02_train,Acc1L03_train,Acc1L04_train),axis=0)
xAcc1L_train=MinMaxScaler([0,255]).fit_transform(xAcc1L_train)
#xAcc1L_train=StandardScaler().fit_transform(xAcc1L_train)
#xAcc1L_train=Normalizer().fit_transform(xAcc1L_train)
#xAcc1L_train=np.transpose(xAcc1L_train)
yAcc1L_train = np.concatenate((Y1_train,Y2_train,Y3_train,Y4_train),axis=0)
xAcc1L_test = np.concatenate((Acc1L01_test,Acc1L05_test,Acc1L02_test,Acc1L03_test,Acc1L04_test),axis=0)
xAcc1L_test=Normalizer().fit_transform(xAcc1L_test)
#xAcc1L_test=MinMaxScaler([0,255]).fit_transform(xAcc1L_test)
#xAcc1L_test=StandardScaler().fit_transform(xAcc1L_test)
#xAcc1L_test=np.transpose(xAcc1L_test)
yAcc1L_test = np.concatenate((Y1_test,Y2_test,Y3_test,Y4_test),axis=0)
#1 hot encode y
one_hot_labelsAcc1L =to_categorical(yAcc1L_train, num_classes=4)
one_hot_labelsAcc1L_test = to_categorical(yAcc1L_test, num_classes=4)
#fit the model
model = Sequential()
model.add(Dense(300, activation='relu', input_dim=30))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
es1 = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=100)
es2 = EarlyStopping(monitor='val_accuracy', mode='max', verbose=1, patience=100)
history=model.fit(xAcc1L_train, one_hot_labelsAcc1L,validation_data=(xAcc1L_test,one_hot_labelsAcc1L_test),epochs=500, batch_size=30, verbose=1, callbacks=[es1,es2])
#history=model.fit(tf.cast(xAcc1L_train, tf.float32), one_hot_labelsAcc1L,validation_data=(tf.cast(xAcc1L_test, tf.float32),one_hot_labelsAcc1L_test),epochs=500, batch_size=30, verbose=1, callbacks=[es1,es2])
time_elapsed = (time.perf_counter() - time_start)
print ("%5.1f secs " % (time_elapsed))
start=time.monotonic()
_, accuracy = model.evaluate(xAcc1L_test, one_hot_labelsAcc1L_test, batch_size=30, verbose=1)
#_, accuracy = model.evaluate(tf.cast(xAcc1L_test, tf.float32), one_hot_labelsAcc1L_test, batch_size=30, verbose=1)
print(accuracy)
inference_time = time.monotonic() - start
print('%.1fms ' % (inference_time * 1000))
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper right')
plt.show()
#predicted labels
predictions = model.predict(xAcc1L_test)
y_pred = (predictions > 0.5)
matrix = confusion_matrix(one_hot_labelsAcc1L_test.argmax(axis=1), y_pred.argmax(axis=1))
print('confusion matrix = \n',matrix)
print("Accuracy:",accuracy_score(one_hot_labelsAcc1L_test.argmax(axis=1), y_pred.argmax(axis=1)))
mod01=model.save('/home/utente/Scrivania/csvtesi/rete_Nom01.h5')
#convert the model
#representative dataset
train_ds = tf.data.Dataset.from_tensor_slices(
(tf.cast(xAcc1L_train, tf.float32))).batch(1)
print(train_ds)
def representative_dataset_gen():
for input_value in train_ds:
yield [input_value]
print(model.layers[0].input_shape)
#integer post-training quantization
converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file('/home/utente/Scrivania/csvtesi/rete_Nom01.h5') #all operations mapped on edge tpu
#converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
print(converter.representative_dataset)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_quant_model = converter.convert()
open('/home/utente/Scrivania/csvtesi/rete_Nom01_quant.tflite', "wb").write(tflite_quant_model)
#compiler compila il modello quantizzato tflite per edge tpu
os.system("edgetpu_compiler \'/home/utente/Scrivania/csvtesi/rete_Nom01_quant.tflite'")
#interpret the model
interpreter = tf.lite.Interpreter('/home/utente/Scrivania/csvtesi/rete_Nom01_quant_edgetpu.tflite',experimental_delegates=[tflite.load_delegate('libedgetpu.so.1')])
interpreter.allocate_tensors()
idt=print(interpreter.get_input_details())
odt=print(interpreter.get_output_details())
for j in range(5):
start = time.monotonic()
o_test=np.arange(len(xAcc1L_test[:,0]))
o_test=o_test[:,np.newaxis]
for i in range (len(xAcc1L_test[:,0])):
input=set_input(interpreter, xAcc1L_test[i,:])
#print("inference input %s" % input)
interpreter.invoke()
classes = get_output(interpreter,4)
output = interpreter.get_tensor(interpreter.get_output_details()[0]['index'])#/255 con edgetpu
#print("inference output %s" % output)
#print("inference classes %s" % classes)
a=np.array([one_hot_labelsAcc1L_test[i,:].argmax(axis=0)])
b=np.array(output.argmax(axis=1))
o_test[i]=b
#if a==b:
#print('good classification')
#else:
#print('bad classification')
inference_time = time.monotonic() - start
print('%.1fms ' % (inference_time * 1000))
#print(o_test)
print("Accuracy:",accuracy_score(yAcc1L_test,o_test))
My input train dataset is part of csv files and it's a matrix of dimensions = (1756,30) and my input test one is a matrix of (183,30).
This is how the data looks like (first two rows):
[[-0.283589 -0.0831421 -0.199936 -0.144523 -0.215593 -0.199029 0.0300179 -0.0299262 -0.0759612 -0.0349733 0.102882 -0.00470235 -0.14267 -0.116636 -0.0842867 -0.124638 -0.107917 -0.0995006 -0.222817 -0.256093 -0.121859 -0.130829 -0.186091 -0.174511 -0.0715493 -0.0595195 -0.054914 -0.0362971 -0.0286576 -0.0409128],
[-0.226151 -0.0386177 -0.16834 -0.0768908 -0.166611 -0.161028 0.0493133 -0.00515959 -0.0362308 -0.00723895 0.105943 -0.010825 -0.142335 -0.10863 -0.0634201 -0.112928 -0.0927994 -0.0556194 -0.180721 -0.218341 -0.0934449 -0.100047 -0.134569 -0.119806 -0.0265749 -0.044841 -0.0538225 -0.017408 -0.00528171 -0.0248457]]
GAN doesn't proceed training very well
I have programmed a GAN model using keras but the training didn't go well. The generator model always returns a bare noise image (28x28 size) instead of something similar to mnist dataset. This doesn't give me any error though, when it comes to training discriminator model will become trainable=False, which is not what I want to do.
If this implementation is bad, please let me know. Can anyone help?
import os
import numpy as np
import matplotlib.pyplot as plt
import keras
from keras.models import Sequential
from keras.layers import Dense, Activation, BatchNormalization
from keras.optimizers import SGD, Adam, RMSprop
from keras.datasets import mnist
from keras.regularizers import l1_l2
def plot_generated(noise, Generator):
image_fake = Generator.predict(noise)
plt.figure(figsize=(10,8))
plt.show()
plt.close()
def plot_metircs(metrics, epoch=None):
plt.figure(figsize=(10,8))
plt.plot(metrics['d'], label='discriminative loss', color='b')
plt.legend()
plt.show()
plt.close()
plt.figure(figsize=(10,8))
plt.plot(metrics['g'], label='generative loss', color='r')
plt.legend()
plt.show()
plt.close()
def Generator():
model = Sequential()
LeakyReLU = keras.layers.advanced_activations.LeakyReLU(alpha=0.2)
model.add(Dense(input_dim=100, units=128, activation=LeakyReLU, name='g_input'))
model.add(Dense(input_dim=128, units=784, activation='tanh', name='g_output'))
return model
def Discriminator():
model = Sequential()
LeakyReLU = keras.layers.advanced_activations.LeakyReLU(alpha=0.2)
model.add(Dense(input_dim=784, units=128, activation=LeakyReLU, name='d_input'))
model.add(Dense(input_dim=128, units=1, activation='sigmoid', name='d_output'))
model.compile(loss='binary_crossentropy', optimizer='Adam')
return model
def Generative_Adversarial_Network(Generator, Discriminator):
model = Sequential()
model.add(Generator)
model.add(Discriminator)
# train only generator in the entire GAN architecture
Discriminator.trainable = False
model.compile(loss='binary_crossentropy', optimizer='Adam')
return model
def Training(z_input_size, Generator, Discriminator, GAN, loss_dict, X_train, epoch, batch, smooth):
for e in range(epoch):
# z: noise, used for input of G to generate fake image based on this noise! it's like a seed
noise = np.random.uniform(-1, 1, size=[batch, z_input_size])
image_fake = Generator.predict_on_batch(noise)
# sampled real_image from dataset
rand_train_index = np.random.randint(0, X_train.shape[0], size=batch)
image_real = X_train[rand_train_index, :]
# concatenate real and fake images
"""
X = [
image_real => label : 1 (we can multiply a smoothing factor)
image_fake => label : 0
]
"""
X = np.vstack((image_real, image_fake))
y = np.zeros(len(X))
# putting label "1" to image_real
y[len(image_real):] = 1*(1 - smooth)
y = y.astype(int)
# train only discriminator
d_loss = Discriminator.train_on_batch(x=X, y=y)
# NOTE: remember?? we set discriminator OFF during the training of GAN!
# So, we can safely train only generator, weight of discriminator set fixed!
g_loss = GAN.train_on_batch(x=noise, y=y[len(noise):])
loss_dict['d'].append(d_loss)
loss_dict['g'].append(g_loss)
if e%1000 == 0:
plt.imshow(image_fake)
plt.show()
plot_generated(noise, Generator)
plot_metircs(loss_dict)
return "done!"
Gen = Generator()
Dis = Discriminator()
GAN = Generative_Adversarial_Network(Gen, Dis)
GAN.summary()
Gen.summary()
Dis.summary()
gan_losses = {"d":[], "g":[], "f":[]}
epoch = 30000
batch = 1000
smooth = 0.9
z_input_size = 100
row, col = 28, 28
z_group_matrix = np.random.uniform(0, 1, examples*z_input_size)
z_group_matrix = z_group_matrix.reshape([9, z_input_size])
print(z_group_matrix.shape)
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train, X_test = X_train.reshape(X_train.shape[0], row*col), X_test.reshape(X_test.shape[0], row*col)
X_train.astype('float32')
X_test.astype('float32')
X_train, X_test = X_train/255, X_test/255
print('X_train shape: ', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
Training(z_input_size, Gen, Dis, GAN, loss_dict=gan_losses, X_train=X_train, epoch=epoch, batch=batch, smooth=smooth)
The model itself is correct. I would suggest a few minor changes: smooth 0.9 is too much. Make it close to 0.1. Leak Factor you have is 0.2, usually its a very small decimal close to 0; take around 0.01/0.02. Batchsize around 400 Epochs around 2000 And finally early stopping with a bit large threshold.
Keras - How to actually use an autoencoder
I have developed the following autoencoder in Keras for the purpose of dimension reduction.
from keras.layers import Input, Dense, BatchNormalization
from keras.models import Model
from keras import regularizers
from keras.callbacks import TensorBoard, EarlyStopping
import keras.backend as K
from sklearn.metrics import r2_score
input_size = len(spot_dat.columns)
coder_size = 32
inner_size = 64
betwe_size = 96
outer_size = 128
batch_size = 25
def r2(y_true, y_pred):
SS_res = K.sum(K.square(y_true - y_pred))
SS_tot = K.sum(K.square(y_true - K.mean(y_true)))
return (1 - SS_res / (SS_tot + K.epsilon()))
def rmse(y_true, y_pred):
return K.sqrt(K.mean(K.square(y_pred - y_true), axis=-1))
input_ = Input(shape=(input_size,))
encoded = Dense(outer_size, activation='selu')(input_) #hidden 1
#encoded = Dense(betwe_size, activation='elu')(input_) #hidden 2
encoded = BatchNormalization()(encoded)
encoded = Dense(inner_size, activation='selu')(encoded) #hidden 3
code = Dense(coder_size, activation='selu')(encoded) #code
decoded = Dense(inner_size, activation='selu')(code) #hidden 2
decoded = BatchNormalization()(decoded)
#decoded = Dense(betwe_size, activation='elu')(decoded) #hidden 2
decoded = Dense(outer_size, activation='selu')(decoded) #hidden 1
output = Dense(input_size, activation='sigmoid')(decoded) #output
autoencoder = Model(input_, output)
autoencoder.compile(optimizer = 'adam', loss = 'mean_squared_error', metrics = [r2, rmse])
val = autoencoder.fit(x_train, x_train,
epochs=1000,
batch_size = 75,
shuffle=True,
validation_data=(x_test, x_test),
callbacks=[TensorBoard(log_dir='/tmp/test'), EarlyStopping(monitor = 'val_loss', min_delta = 0, patience = 30, verbose = True, mode = 'auto')])
plt.plot(val.history['loss'])
plt.plot(val.history['val_loss'])
plt.title('Model loss (MSR)')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()
plt.plot(val.history['r2'])
plt.plot(val.history['val_r2'])
plt.title('Model R2')
plt.ylabel('R2')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()
plt.plot(val.history['rmse'])
plt.plot(val.history['val_rmse'])
plt.title('Model RMSE')
plt.ylabel('RMSE')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc = 'best')
plt.show()
The model works fine, the issue is I cannot find anyway to extract part of the model (from input to code) after it has been trained on the full autoencoder. It seems to me, if you cant pull half the model the autoencoder cannot be used for any practical dimension reduction. Thus, there should be a way to freeze and pull the trained models first several layers. The end goal is to apply the layer with 32 dimensions to a t-SNE and compare results against other dimension reduction techniques.
After training just do the following: encoder = Model(input_, code) Then use encoder.predict to get the code from an input sample.