We Keep Coding

ValueError: Input arrays should have the same number of samples as target arrays. Found 60000 input samples and 10000 target samples

I'm trying to train my Deep Neural Network to recognize handwritten
numbers but I keep getting the error stated previously in the title It
gives me an error saying: ValueError: Input arrays should have the
same number of samples as target arrays. Found 60000 input samples and
10000 target samples. How can i fix this? (i already tried
train_test_split and transport but nothing worked)
# Imports
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical
# Configuration options
feature_vector_length = 784
num_classes = 60000
# Load the data
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
# Reshape the data - MLPs do not understand such things as '2D'.
# Reshape to 28 x 28 pixels = 784 features
X_train = X_train.reshape(X_train.shape[0], feature_vector_length)
X_test = X_test.reshape(X_test.shape[0], feature_vector_length)
# Convert into greyscale
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
# Convert target classes to categorical ones
Y_train = to_categorical(Y_train, num_classes)
Y_test = to_categorical(Y_test, num_classes)
# Load the data
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
# Visualize one sample
import matplotlib.pyplot as plt
plt.imshow(X_train[0], cmap='Greys')
plt.show()
# Set the input shape
input_shape = (feature_vector_length,)
print(f'Feature shape: {input_shape}')
# Create the model
# Using sigmoid instead of relu function
model = Sequential()
model.add(Flatten())
model.add(Dense(350, input_shape=input_shape, activation="sigmoid",
kernel_initializer=init))
model.add(Dense(50, activation="sigmoid", kernel_initializer=init))
model.add(Dense(num_classes, activation="sigmoid",
kernel_initializer=init))
# Configure the model and start training
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=
['accuracy'])
model.fit(X_train, Y_train, epochs=10, batch_size=250, verbose=1,
validation_split=0.2)
# Test the model after training
test_results = model.evaluate(X_test, Y_test, verbose=1)
print(f'Test results - Loss: {test_results[0]} - Accuracy:
{test_results[1]}%')

If you want a solution to your problem, here it is:
# Imports
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Flatten
from keras.utils import to_categorical
# Configuration options
feature_vector_length = 784
num_classes = 10
# Load the data
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
# Reshape the data - MLPs do not understand such things as '2D'.
# Reshape to 28 x 28 pixels = 784 features
# X_train = X_train.reshape(X_train.shape[0], feature_vector_length)
# X_test = X_test.reshape(X_test.shape[0], feature_vector_length)
# Convert into greyscale
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
# Convert target classes to categorical ones
Y_train = to_categorical(Y_train, num_classes)
Y_test = to_categorical(Y_test, num_classes)
# Create the model
# Using sigmoid instead of relu function
model = Sequential()
model.add(Flatten())
model.add(Dense(350, input_shape=input_shape, activation="relu"))
model.add(Dense(50, activation="relu"))
model.add(Dense(num_classes, activation="softmax"))
# Configure the model and start training
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=
['accuracy'])
model.fit(X_train, Y_train, epochs=10, batch_size=250, verbose=1,
validation_split=0.2)
# Test the model after training
test_results = model.evaluate(X_test, Y_test, verbose=1)
But you should really do some research and understand what every line in your code is supposed to do and what every parameter means. For example, the choice of sigmoid activation function being wrong, especially in the final layer is the very first thing to understand. That's one of the many things that you should do research about. Then there is:
understanding why and when to reshape your data,
what is the purpose of flatten layer
and most importantly, understand what is num_classes and why it is 10 and not 1000 or 60000

See all correctly and incorrectly identified images when training on the mnist dataset

I'm trying to find a way to visualize which numbers in the mnist dataset a model was able to correctly identify and which ones it wasn't.
What I can't seem to find is if such a visualization is possible in tensorboard or if I would need to use/create something else to achieve it.
I'm currently working from the basic tutorial provided for tensorflow 2.0 with tensorboard added.
import datetime
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit(x_train,
y_train,
epochs=5,
validation_data=(x_test, y_test),
callbacks=[tensorboard_callback])
model.evaluate(x_test, y_test)

It appears the what-if tool is what I was looking for, it allows you to visually sort testing data depending on whether it was correctly or incorrectly identified by the model.
If you want to test it out here is their demo that I used to get the above image and they have multiple other demos on the tools site.

Tensorflow——keras model.save() raise NotImplementedError

import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = tf.keras.utils.normalize(x_train, axis=1)
x_test = tf.keras.utils.normalize(x_test, axis=1)
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128,activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128,activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(10,activation=tf.nn.softmax))
model.compile(optimizer ='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3)
When I tried to save the model
model.save('epic_num_reader.model')
I get a NotImplementedError:
NotImplementedError Traceback (most recent call last)
<ipython-input-4-99efa4bdc06e> in <module>()
1
----> 2 model.save('epic_num_reader.model')
NotImplementedError: Currently `save` requires model to be a graph network. Consider using `save_weights`, in order to save the weights of the model.
So how can I save the model defined in the code?

You forgot the input_shape argument in the definition of the first layer, which makes the model undefined, and saving undefined models has not been implemented yet, which triggers the error.
model.add(tf.keras.layers.Flatten(input_shape = (my, input, shape)))
Just add the input_shape to the first layer and it should work fine.

For those who still have not solved the problem even did as Matias suggested, you can consider using tf.keras.models.save_model() and load_model(). In my case, it worked.

tf.keras.models.save_model
Works here (tensorflow 1.12.0) (even when the input_shape is unspecified)

Reason For the Error:
I was getting the same error and tried the above answers but got errors. But I find a solution to the problem that I will share below:
Check whether you passed input_shape at the time of defining the input layer of the model if not you will get an error at the time of saving and loading the model.
How to define input_shape?
Lets consider the one example If you use minst dataset:
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
It consists of images of handwritten digits 0-9 of size 28 x 28 resolution each.
For this, we can define input shape as (28,28) without mentioning batch size as follows:
model.add(tf.keras.layers.Flatten(input_shape=(28,28)))
In this way, you can give input shape by looking at your input training dataset.
Save your trained model:
Now after training and testing the model we can save our model. Following code worked for me it did not change the accuracy as well after reloading the model:
by using save_model()
import tensorflow as tf
tf.keras.models.save_model(
model,
"your_trained_model.model",
overwrite=True,
include_optimizer=True
)
by using .save()
your_trained_model.save('your_trained_model.model')
del model # deletes the existing model
Now load the model which we saved :
model2 = tf.keras.models.load_model("your_trained_model.model")
For more details refer to this link: Keras input explanation: input_shape, units, batch_size, dim, etc

<!-- Success, please check -->
import tensorflow as tf
import matplotlib.pyplot as plt
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
plt.imshow(x_train[0], cmap=plt.cm.binary)
x_train = tf.keras.utils.normalize(x_train, axis=1)
x_test = tf.keras.utils.normalize(x_test, axis=1)
plt.imshow(x_train[0], cmap=plt.cm.binary)
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=x_train[0].shape))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3)
val_loss, val_acc = model.evaluate(x_test, y_test)
print(val_loss)
print(val_acc)
model.save('epic_num_reader.model')

Invalid array shape with neural network using Keras?

Currently studying the 'Deep Learning with Python' book by Francios Chollet. I am very new to this and I am getting this error code despite following his code verbatim. Can anyone interpret the error message or what needs to be done to solve it? Any help would be greatly appreciated!
from keras.datasets import imdb
import numpy as np
from keras import models
from keras import layers
(train_data, train_labels), (test_data, test_labels) =
imdb.load_data(num_words=10000)
def vectorize_sequences(sequences, dimension=10000):
results = np.zeros((len(sequences), dimension))
for i, sequence in enumerate(sequences):
results[i, sequence] = 1.
return results
x_train = vectorize_sequences(train_data)
y_train = vectorize_sequences(test_data)
x_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')
model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=4, batch_size=512)
results = model.evaluate(x_test, y_test)
Edit: Here is an image of the error code that I am getting:

I tested your Code and found, that x_test was not defined. I think you meant to vectorize it as follows. With this Code it worked:
x_train = vectorize_sequences(train_data)
x_test = vectorize_sequences(test_data)
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')

Loading a trained Keras model and continue training

I was wondering if it was possible to save a partly trained Keras model and continue the training after loading the model again.
The reason for this is that I will have more training data in the future and I do not want to retrain the whole model again.
The functions which I am using are:
#Partly train model
model.fit(first_training, first_classes, batch_size=32, nb_epoch=20)
#Save partly trained model
model.save('partly_trained.h5')
#Load partly trained model
from keras.models import load_model
model = load_model('partly_trained.h5')
#Continue training
model.fit(second_training, second_classes, batch_size=32, nb_epoch=20)
Edit 1: added fully working example
With the first dataset after 10 epochs the loss of the last epoch will be 0.0748 and the accuracy 0.9863.
After saving, deleting and reloading the model the loss and accuracy of the model trained on the second dataset will be 0.1711 and 0.9504 respectively.
Is this caused by the new training data or by a completely re-trained model?
"""
Model by: http://machinelearningmastery.com/
"""
# load (downloaded if needed) the MNIST dataset
import numpy
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import np_utils
from keras.models import load_model
numpy.random.seed(7)
def baseline_model():
model = Sequential()
model.add(Dense(num_pixels, input_dim=num_pixels, init='normal', activation='relu'))
model.add(Dense(num_classes, init='normal', activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
if __name__ == '__main__':
# load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()
# flatten 28*28 images to a 784 vector for each image
num_pixels = X_train.shape[1] * X_train.shape[2]
X_train = X_train.reshape(X_train.shape[0], num_pixels).astype('float32')
X_test = X_test.reshape(X_test.shape[0], num_pixels).astype('float32')
# normalize inputs from 0-255 to 0-1
X_train = X_train / 255
X_test = X_test / 255
# one hot encode outputs
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]
# build the model
model = baseline_model()
#Partly train model
dataset1_x = X_train[:3000]
dataset1_y = y_train[:3000]
model.fit(dataset1_x, dataset1_y, nb_epoch=10, batch_size=200, verbose=2)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Baseline Error: %.2f%%" % (100-scores[1]*100))
#Save partly trained model
model.save('partly_trained.h5')
del model
#Reload model
model = load_model('partly_trained.h5')
#Continue training
dataset2_x = X_train[3000:]
dataset2_y = y_train[3000:]
model.fit(dataset2_x, dataset2_y, nb_epoch=10, batch_size=200, verbose=2)
scores = model.evaluate(X_test, y_test, verbose=0)
print("Baseline Error: %.2f%%" % (100-scores[1]*100))
Edit 2: tensorflow.keras remarks
For tensorflow.keras change the parameter nb_epochs to epochs in the model fit. The imports and basemodel function are:
import numpy
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import load_model
numpy.random.seed(7)
def baseline_model():
model = Sequential()
model.add(Dense(num_pixels, input_dim=num_pixels, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model

Actually - model.save saves all information need for restarting training in your case. The only thing which could be spoiled by reloading model is your optimizer state. To check that - try to save and reload model and train it on training data.

Most of the above answers covered important points. If you are using recent Tensorflow (TF2.1 or above), Then the following example will help you. The model part of the code is from Tensorflow website.
import tensorflow as tf
from tensorflow import keras
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
def create_model():
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy',metrics=['accuracy'])
return model
# Create a basic model instance
model=create_model()
model.fit(x_train, y_train, epochs = 10, validation_data = (x_test,y_test),verbose=1)
Please save the model in *.tf format. From my experience, if you have any custom_loss defined, *.h5 format will not save optimizer status and hence will not serve your purpose if you want to retrain the model from where we left.
# saving the model in tensorflow format
model.save('./MyModel_tf',save_format='tf')
# loading the saved model
loaded_model = tf.keras.models.load_model('./MyModel_tf')
# retraining the model
loaded_model.fit(x_train, y_train, epochs = 10, validation_data = (x_test,y_test),verbose=1)
This approach will restart the training where we left before saving the model. As mentioned by others, if you want to save weights of best model or you want to save weights of model every epoch you need to use keras callbacks function (ModelCheckpoint) with options such as save_weights_only=True, save_freq='epoch', and save_best_only.
For more details, please check here and another example here.

The problem might be that you use a different optimizer - or different arguments to your optimizer. I just had the same issue with a custom pretrained model, using
reduce_lr = ReduceLROnPlateau(monitor='loss', factor=lr_reduction_factor,
patience=patience, min_lr=min_lr, verbose=1)
for the pretrained model, whereby the original learning rate starts at 0.0003 and during pre-training it is reduced to the min_learning rate, which is 0.000003
I just copied that line over to the script which uses the pre-trained model and got really bad accuracies. Until I noticed that the last learning rate of the pretrained model was the min learning rate, i.e. 0.000003. And if I start with that learning rate, I get exactly the same accuracies to start with as the output of the pretrained model - which makes sense, as starting with a learning rate that is 100 times bigger than the last learning rate used in the pretrained model will result in a huge overshoot of GD and hence in heavily decreased accuracies.

Notice that Keras sometimes has issues with loaded models, as in here.
This might explain cases in which you don't start from the same trained accuracy.

You might also be hitting Concept Drift, see Should you retrain a model when new observations are available. There's also the concept of catastrophic forgetting which a bunch of academic papers discuss. Here's one with MNIST Empirical investigation of catastrophic forgetting

All above helps, you must resume from same learning rate() as the LR when the model and weights were saved. Set it directly on the optimizer.
Note that improvement from there is not guaranteed, because the model may have reached the local minimum, which may be global. There is no point to resume a model in order to search for another local minimum, unless you intent to increase the learning rate in a controlled fashion and nudge the model into a possibly better minimum not far away.

If you are using TF2, use the new saved_model method(format pb). More information available here and here.
model.fit(x=X_train, y=y_train, epochs=10,callbacks=[model_callback])#your first training
tf.saved_model.save(model, save_to_dir_path) #save the model
del model #to delete the model
model = tf.keras.models.load_model(save_to_dir_path)
model.fit(x=X_train, y=y_train, epochs=10,callbacks=[model_callback])#your second training

It is completely okay to train a model with a saved model. I trained the saved model with the same data and found it was giving good accuracy. Moreover, the time taken was quite less in each epoch.
Here is the code have a look:
from keras.models import load_model
model = load_model('/content/drive/MyDrive/CustomResNet/saved_models/model_1.h5')
history=model.fit(train_gen,validation_data=valid_gen,epochs=5)