I am basically doing this for a school project and followed some guides to make a neuron network using CNN. Libraries I am using are cv2, NumPy, TensorFlow, and matplotlib. The problem currently I am facing is that my network has high accuracy but very bad predictions. I made sure the pictures are inverted and 28x28. I also expand the number of images to predict from 5 to 10. I also tried adding more layers but didn't help either. If anyone can help me out would be awesome! I am also very new to this so please explain the best you can!
Example of the output:As you can see, the hand-writing isn't bad or anything but it still can't predict that it's a 6 but a 1.
Here is the Epoch with an accuracy of 99% basically
Here is the code:
import cv2
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(a_train, b_train), (a_test, b_test) = mnist.load_data()
a_train = tf.keras.utils.normalize(a_train, axis=1)
a_test = tf.keras.utils.normalize(a_test, axis=1)
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=(28,28)))
model.add(tf.keras.layers.Dense(units=255, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(units=255, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(units=20, activation=tf.nn.softmax))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(a_train, b_train, epochs=50)
lost, accuracy = model.evaluate(a_train, b_train)
print(lost)
print(accuracy)
model.save('test.model')
for x in range(1,11):
img = cv2.imread(fr'C:\Users\Eric\PycharmProjects\pythonProject2\test.model\{x}.png')[:,:,0]
img = np.invert(np.array([img]))
prediction = model.predict(img)
print(f'My Guess is: {np.argmax(prediction)}')
plt.imshow(img[0], cmap=plt.cm.binary)
plt.show()
Somethings I tried doing:
I tried adding more layers assuming that it will train and have a better prediction.
I added more samples numbers to see if I can have a higher prediction. I went from 5 to 10 but still a 20% right prediction.
I have tried changing about of Epoch and tried more batch size but also didn't work.
I am pretty much stuck at this point trying my best to understand it but not able to improve it at all. If anyone has any tips, please let me know!
You need to normalize your images when predicting. cv2.imread creates an array from 0 to 255. You can normalize it by dividing img by 255.
Your image you use to predict should also have white text on a black background.
Lastly, you do not need the np.invert.
So your code should be
for x in range(1, 11):
img = np.expand_dims(cv2.imread(f'C:\Users\Eric\PycharmProjects\pythonProject2\test.model\{x}.png')[:, :, 0], 0) / 255.
prediction = model.predict(img)
print(f'My Guess is: {np.argmax(prediction)}')
plt.imshow(img[0], cmap=plt.cm.binary)
plt.show()
Related
I'm writing here, hoping to solve a problem, that i had with a neural network, developed in python by using keras.
I'm newer with the deep learning world, and I'm studying the theory and trying to implement some code.
Goal: develop a net that allows me to recognize 2 different words (commands) that i say [in the future them will be used to drive a small robot-car]. Actually, I want only to achieve the identification of "yes/no"
Implementation: i'm trying to implement a binary classification network.
Here is my code:
i used librosa to convert the audio training and test set in a matrix input with 193 features
to overcame the possibility of batch normalization problem, i scaled the data, by using the preprocessing package (I saw that effectively that improves and affects performance): i notice that if I don't normalize training, test and data_to_be_anlyzed by using the same normalization, it doesn't work
I read that keras accept as input that can be both numpy array, so i convert the target y into numpy
i proceed with the construction of the model, training and test (i know that i'm using methods that actually are deprecated)
I use one audio to perform another text, because for the future i assume that the net will receive (and judge) one audio at-a-time
import support.myutilities as utilNP
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from sklearn.preprocessing import StandardScaler
#READ AND BUILD INPUT
X_si = utilNP.np_array_features_dir('DIRECTORY PATH')
X_no = utilNP.np_array_features_dir('DIRECTORY PATH')
X_tot = np.concatenate((X_si, X_no), axis=0)
# Scale the train set
scaler = StandardScaler().fit(X_tot)
X_train = scaler.transform(X_tot)
#0=si 1=no
y=[]
for i in range(len(X_si)):
y.append(0)
for i in range(len(X_no)):
y.append(1)
Y=np.array(y)
#READ AND BUIL TEST TRAINING SET
X_si_test = utilNP.np_array_features_dir('DIRECTORY PATH')
X_no_test = utilNP.np_array_features_dir('DIRECTORY PATH')
X_tot_test = np.concatenate((X_si_test, X_no_test), axis=0)
# Scale the test set
scaler2 = StandardScaler().fit(X_tot_test)
X_test = scaler2.transform(X_tot_test)
y_test=[]
for i in range(len(X_si_test)):
y_test.append(0)
for i in range(len(X_no_test)):
y_test.append(1)
Y_test=np.array(y_test)
###### BUILD MODEL
model = Sequential()
model.add(Dense(100, input_dim=len(X_tot[0]), activation='relu')) #193 features as input
model.add(Dense(50, activation='relu'))
model.add(Dense(1, activation='sigmoid')) #1 output
# compile the keras model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
model.fit(X_train, Y, epochs=300, verbose=1)
#test
scores = model.evaluate(X_test, Y_test, verbose=0)
print('Accuracy on training data: {}% \n Error on training data: {}'.format(scores[1], 1 - scores[1]))
predictions = model.predict(X_test)
for i in range(len(predictions)):
print('=> %d (expected %d)' % (predictions[i], y_test[i]))
#TEST WITH A PRACTICAL NEW SOUND: supposed acquired
file_name = 'PATH AUDIO'
X = utilNP.np_array_features(file_name)
#Normalize according to input data
X_analyze = scaler2.transform(X)
y_analysis=[]
y_analysis.append(1) # i supposed that the audio is the one that return 1
pred_test= model.predict(X_analyze)
scores2 = model.evaluate(X_analyze, np.array(y_analysis), verbose=0)
print('Accuracy on test data: {}% \n Error on test data: {}'.format(scores2[1], 1 - scores2[1]))
Problems:
accuracy go in 100% in very few epochs. Is real that the training set is not so big (a total of 300 samples, and 40 for test), but this result is clearly wrong. By the way, if I use a number of epochs > 100, the net works well and performs its work good (practically the result of the single case study, is recognized)
if the number of epochs is low (20 for example), accuracy still reach 100% after few iterations, but the training is affect by the error in the results (why are not recognized?) and the final prediction wrong. It is not normal: i would expect a low accuracy to justify the wrong answer, but it remains at 100%
I test a lot of solution, passing from setting 'training=True/False', and read a lot of answer here and in stack exchange, but I don't solved nothing.
There is something wrong in my code?
Thanks in advance.
I've been working on a Keras network to classify images as to whether they contain traffic lights or not, but so far I've had 0 success. I have a dataset of 11000+ images, and for my first test I used 240 images (or rather, text files for each image with the grayscale pixel values). There is only one output - a 0 or 1 saying whether the image contains traffic lights.
However, when I ran the test, it only predicted one class. Given that 53/240 images had traffic lights, it was achieving about a 79% accuracy rate because it was just predicting 0 all the time. I read that this might be down to inbalanced data, so I downscaled to just 4 images - 2 with traffic lights, 2 without.
Even with this test, it still stuck at 50% accuracy after 5 epochs; it's just predicting one class! Similar questions have been answered but I haven't found anything that is working for me :(
Here is the code I am using:
from keras.datasets import mnist
from keras import models
from keras import layers
from keras.utils import to_categorical
import numpy as np
import os
train_images = []
train_labels = []
#The following is just admin tasks - extracting the grayscale pixel values
#from the text files, adding them to the input array. Same with the labels, which
#are extracted from text files and added to output array. Not important to performance.
for fileName in os.listdir('pixels1/'):
newRead = open(os.path.join('pixels1/', fileName))
currentList = []
for pixel in newRead:
rePixel = int(pixel.replace('\n', ''))/255
currentList.append(rePixel)
train_images.append(currentList)
for fileName in os.listdir('labels1/'):
newRead = open(os.path.join('labels1/', fileName))
line = newRead.readline()
train_labels.append(int(line))
train_images = np.array(train_images)
train_labels = np.array(train_labels)
train_images = train_images.reshape((4,13689))
#model
model = models.Sequential()
model.add(layers.Dense(13689, input_dim=13689, activation='relu'))
model.add(layers.Dense(13689, activation='relu'))
model.add(layers.Dense(1, activation='softmax'))
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=5, batch_size=1)
I was hoping at the very least it would be able to recognise the images at the end. I really want to move onto running a training session on my full 11,000 examples, but at this point I can't get it to work with 4.
Rough points:
You seem to believe that the number of units in your dense layers should be equal to your data dimension (13869); this is not the case. Change both of them to something smaller (in the range of 100-200) - they do not even have to be equal. A model that big is not recommended with your relatively small number of data samples (images).
Since you are in a binary classification setting with a single node in your last layer, you should use activation=sigmoid for this (last) layer, and compile your model with loss='binary_crossentropy'.
In imaging applications, normally the first couple of layers are convolutional ones.
I am working on an already implemented project in Kaggle which has to do with Image Classification. I have 6 classes to predict on in total, which are Angry, Happy, Sad etc. I have implemented a CNN model and I am currently using only 4 classes(the ones with highest number of images), but my model is overfitting, my validation accuracy is going 53% at maximum, therefore I have tried several things but not seemingly improving my accuracy. Now I saw people mentioning something called Data Augmentation and thought to give it a go as it seems a potential to increase the accuracy. However I am stuck with an error which I cannot figure out.
Distribution of dataset:
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from matplotlib.pyplot import imread, imshow, subplots, show
def plot(data_generator):
"""
Plots 4 images generated by an object of the ImageDataGenerator class.
"""
data_generator.fit(df_training)
image_iterator = data_generator.flow(df_training)
# Plot the images given by the iterator
fig, rows = subplots(nrows=1, ncols=4, figsize=(18,18))
for row in rows:
row.imshow(image_iterator.next()[0].astype('int'))
row.axis('off')
show()
x_train = df_training.drop("emotion",axis=1)
image = x_train[1:2].values.reshape(48, 48)
x_train = x_train.values.reshape(x_train.shape[0], 48, 48,1)
x_train = x_train.astype("float32")
image = image.astype("float32")
image = x_train[1:2].reshape(48, 48)
# Creating a dataset which contains just one image.
images = image.reshape((1, image.shape[0], image.shape[1]))
imshow(images[0])
show()
print(x_train.shape)
data_generator = ImageDataGenerator(rotation_range=90)
plot(data_generator)
Error:
ValueError: Input to .fit() should have rank 4. Got array with
shape: (28709, 2305)
I have already reshaped my data into a 4d array but for some reason in the error it appears as my data is 2d.
This is the shape of print(x_train.shape) => (28709, 48, 48, 1)
x_train is where the dataset is, x_train[1:2] accessing one image.
P.s Is there any other approach that you would recommend to improve my accuracy according to this dataset. For further questions about my dataset please let me know if you don't understand something in this partial code.
You use your data_generator on df_training and not on x_train.
As for more ideas about how to avoid overfitting:
Tensorflow has an official tutorial on that with some good suggestions:
https://www.tensorflow.org/tutorials/keras/overfit_and_underfit
I am working with Keras' sample denoising autoencoder;
https://keras.io/examples/mnist_denoising_autoencoder/
As I compile it, I use the following options:
autoencoder.compile(loss='mse', optimizer= Adadelta, metrics=['accuracy'])
Followed by training. I did training deliberately WITHOUT using noisy training data(x_train_noisy), but merely tried to recover x_train.
autoencoder.fit(x_train, x_train, epochs=30, batch_size=128)
After training 60,000 inputs of MNIST digits, it gives me an accuracy of 81.25%. Does it mean there are 60000*81.25% images are PERFECTLY recovered (equaling to the original input pixel by pixel), that is, 81.25% output images from the autoencoder are IDENTICAL to their input counterparts, or something else?
Furthermore, I also conducted a manual check by comparing output and the original data (60000 28X28 matrices) pixel by pixel--counting non-zeros elements from their differences:
x_decoded = autoencoder.predict(x_train)
temp = x_train*255
x_train_uint8 = temp.astype('uint8')
temp = x_decoded*255
x_decoded_uint8 = temp.astype('uint8')
c = np.count_nonzero(x_train_uint8 - x_decoded_uint8)
cp = 1-c /60000/28/28
Yet cp is only about 71%. Could any tell me why there is a difference?
Accuracy doesn't make sense for a regression problem, hence the keras sample doesn't use that metric during autoencoder.compile.
In this case, keras calculates the accuracy as per this metric.
binary_accuracy
def binary_accuracy(y_true, y_pred):
return K.mean(K.equal(y_true, K.round(y_pred)), axis=-1)
Using this numpy implementation, you should get the same value as output by Keras for validation accuracy at the end of training.
x_decoded = autoencoder.predict(x_test_noisy)
acc = np.mean(np.equal(x_test, np.round(x_decoded)))
print(acc)
Refer this answer for more details:
What function defines accuracy in Keras when the loss is mean squared error (MSE)?
I am very new to Keras, neural networks and machine learning having just started to learn yesterday. I decided to try predicting the experience over an hour (0 to 23) (for a game and my own generated data-set) that a user would earn. Currently running what I have the predictions seem to be very low and very poor. I have tried a relu activation, which produced predictions all to be zero and from a bit of research, LeakyReLU.
This is the code I have for the prediction model so far:
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LeakyReLU
import numpy
numpy.random.seed(7)
dataset = numpy.loadtxt("experience.csv", delimiter=",")
X = dataset[: ,0]
Y = dataset[: ,1]
model = Sequential()
model.add(Dense(12, input_dim = 1, activation=LeakyReLU(0.3)))
model.add(Dense(8, activation=LeakyReLU(0.3)))
model.add(Dense(1, activation=LeakyReLU(0.3)))
model.compile(loss = 'mean_absolute_error', optimizer='adam', metrics = ['accuracy'])
model.fit(X, Y, epochs=120, batch_size=10, verbose = 0)
predictions = model.predict(X)
rounded = [round(x[0]) for x in predictions]
print(rounded)
I have also tried playing around with the hidden levels of the network, but honestly have no idea how many there should be or a good way to justify an amount.
If it helps here is the data-set I have been using:
https://raw.githubusercontent.com/NightShadeII/xpPredictor/master/experience.csv
Thankyou for any help
Looking at your data it does not seem like a classification problem.
You have two options:
-> Look at the second column and bucket them depending on the ranges and make classes that can be predicted, for instance: 0, 1, 2 etc. Now it tries to train but does not have enough examples for millions of classes that it thinks you are trying to predict.
-> If you want real valued output and not classes, try using linear regression.