I am following the steps outlined in the tutorial here
I am attempting to run the following code from the tutorial in a cell inside of a Google Colaboratory notebook:
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) =
tf.keras.datasets.fashion_mnist.load_data()
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255
model = tf.keras.Sequential()
# Must define the input shape in the first layer of the neural network
model.add(tf.keras.layers.Conv2D(filters=64, kernel_size=2, padding='same', activation='relu', input_shape=(28,28,1)))
model.add(tf.keras.layers.MaxPooling2D(pool_size=2))
model.add(tf.keras.layers.Dropout(0.3))
model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'))
model.add(tf.keras.layers.MaxPooling2D(pool_size=2))
model.add(tf.keras.layers.Dropout(0.3))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(256, activation='relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(10, activation='softmax'))
# Take a look at the model summary
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.fit(x_train,
y_train,
batch_size=64,
epochs=10)
# Evaluate the model on test set
score = model.evaluate(x_test, y_test, verbose=0)
# Print test accuracy
print('\n', 'Test accuracy:', score[1])
When I run the cell , I get the following error:
Error when checking input: expected conv2d_5_input to have 4 dimensions, but got array with shape (60000, 28, 28)
I feel like I am missing something that is fundamental to the usage of a convolutional layer, nonetheless it appears as though this should have worked. I found some similar questions on SO where people recommended manipulating the "input_shape" argument. I've tried changing it to (60000, 28, 28) and also added additional dimensions with values of 1, but nothing has worked so far. Can anyone point out where I might be missing something here?
It looks like you skipped reshaping part from tutorial:
# Reshape input data from (28, 28) to (28, 28, 1)
w, h = 28, 28
x_train = x_train.reshape(x_train.shape[0], w, h, 1)
x_valid = x_valid.reshape(x_valid.shape[0], w, h, 1)
x_test = x_test.reshape(x_test.shape[0], w, h, 1)
The idea here is that your samples are 28x28x1 (one color, 28x28 pixels), and the first dimension - the number of the sample (60000 in your case).
Related
This code was given to us by a teacher, so it should work right off the bat. However, I can't get it to run.
K.image_data_format()
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# load data
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
# normalize inputs from 0-255 to 0.0-1.0
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train = X_train / 255.0
X_test = X_test / 255.0
# one hot encode outputs
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]
# Create the model
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(3, 32, 32), padding='same', activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.2))
model.add(Conv2D(32, (3, 3), activation='relu', padding='same', kernel_constraint=maxnorm(3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(512, activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
# Compile model
epochs = 25
lrate = 0.01
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
print(model.summary())
# Fit the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=epochs, batch_size=32)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Accuracy: %.2f%%" % (scores[1]*100))
I receive the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 32 but received input with shape [None, 32, 32, 3]
Just thrown off since no one else had this issue with the given code. I did have to change the first line from
K.set_image_dim_ordering('th')
to
K.image_data_format
as I was being told that set_image_dim_ordering was not a known function of Keras.backend
Any ideas here? Could my change has introduced this error?
Update: If data is in channel_last format, then change input shape from input_shape=(3, 32, 32) to input_shape=(32, 32, 3) in,
model.add(Conv2D(32, (3, 3), input_shape=(3, 32, 32), padding='same', activation='relu', kernel_constraint=maxnorm(3)))
I found reference to set_image_dim_ordering below for keras 1.2.2,
https://faroit.com/keras-docs/1.2.2/backend/
Here, it mentions,
For 2D data (e.g. image), "tf" assumes (rows, cols, channels) while
"th" assumes (channels, rows, cols).
Since you are receving input shape [None, 32, 32, 3] that means it is tf format of (rows, cols, channels). So providing th means the input shapes mismatch. You were using theano backend with th instead of tensorflow.
I could not find any reference to set_image_dim_ordering or image_dim_ordering in latest tf.keras backend. Instead these below provide method for getting and setting 'channels_first' or 'channels_last' format.
https://www.tensorflow.org/api_docs/python/tf/keras/backend/image_data_format
https://www.tensorflow.org/api_docs/python/tf/keras/backend/set_image_data_format
I have a pandas dataframe X_train with 321 samples and 43 features. Also, there are 18 different classes in y_train.
strong textI want to train a CNN over my data, but I am having trouble to give the input shape in case of pandas dataframe.
X.shape, y.shape
((321, 43), (321,))
X = np.array(X)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0, stratify = y)
X_train.shape, X_test.shape
((256, 43), (65, 43))
inputs = np.concatenate((X_train, X_test), axis=0)
targets = np.concatenate((y_train, y_test), axis=0)
inputs.shape, targets.shape
((321, 43), (321,))
In the first layer of my model, I am having trouble with input_shape.
I am new to CNN and all the tutorials have used image and they are just passing in the height, width and channel as the parameter of input_shape.
fold_no = 1
for train, test in kfold.split(inputs, targets):
model = Sequential()
**model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(???)))**
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dense(18, activation='softmax'))
model.compile(optimizer=Adam(learning_rate = 0.001),
loss = 'sparse_categorical_crossentropy',
metrics = ['accuracy'])
history = model.fit(inputs[train], targets[train], batch_size=5, epochs=50, validation_split=0.2, verbose=1)
scores = model.evaluate(inputs[test], targets[test], verbose=0)
fold_no = fold_no + 1
I am having trouble with input_shape in the first layer:
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(???)))
I tried to set the input shape like the following format:
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(None, train.shape[1])))
But I got the following error:
I also tried in this way:
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(321, 43)))
Then I got the following error:
I also tried the following format:
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(None, 43)))
Then I got the following error:
Conv1D takes a 3D shape as an input, but the 1st dimension is the batch size, so you can ignore it for input_shape. The other 2 dimensions are (steps, input_dim).
When dealing with numeric or text data, the two dimensions are usually (a) how many sequential rows you want your CNN layer to process at once, (b) how many features are in the row. If your data is naturally segmented into specific lengths (maybe 24, for hours in a day, or 3 words in a trigram), you'll want to specifically set the steps dimension. It will also affect your output shape, which will be (steps-kernel_size+1, filters). Try using some different shapes and look at the model summary to see how they change.
But as the documentation says, you can also use None as your steps, e.g. (None, 128) for variable-length sequences of 128-dimensional vectors.
So basically, I'd suggest this, where inputs[train].shape[1] should be 43 for you:
input_shape=(None, inputs[train].shape[1])
You could also try the full length of your dataset, e.g. (321, 43):
input_shape=inputs[train].shape
Take a look at this excellent answer and also this article for a good visual intuition of how Conv1D works on numeric/text input.
tensor =1D sample or data point how many numbers or features = and number of samples.
In this case
sample1 or vector=[x0,x1,x2,...,ystando], total amount of samples are:321 and total features=31,it will have input_shape=(321,43) of the tensor
I'm building a model to classify text into one of 9 layers, and am having this error when running it. Activation 1 seems to refer to the Convolutional layer's input, but I'm unsure about what's wrong with the input.
num_classes=9
Y_train = keras.utils.to_categorical(Y_train, num_classes)
#Reshape data to add new dimension
X_train = X_train.reshape((100, 150, 1))
Y_train = Y_train.reshape((100, 9, 1))
model = Sequential()
model.add(Conv1d(1, kernel_size=3, activation='relu', input_shape=(None, 1)))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x=X_train,y=Y_train, epochs=200, batch_size=20)
Running this results in the following error:
"ValueError: Error when checking target: expected activation_1 to have shape (None, 9) but got array with shape (9,1)
There are several typos and bugs in your code.
Y_train = Y_train.reshape((100,9))
Since you reshape X_train to (100,150,1), I guess your input step is 150, and channel is 1. So for the Conv1D, (there is a typo in your code), input_shape=(150,1).
You need to flatten your output of conv1d before feeding into Dense layer.
import keras
from keras import Sequential
from keras.layers import Conv1D, Dense, Flatten
X_train = np.random.normal(size=(100,150))
Y_train = np.random.randint(0,9,size=100)
num_classes=9
Y_train = keras.utils.to_categorical(Y_train, num_classes)
#Reshape data to add new dimension
X_train = X_train.reshape((100, 150, 1))
Y_train = Y_train.reshape((100, 9))
model = Sequential()
model.add(Conv1D(2, kernel_size=3, activation='relu', input_shape=(150,1)))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x=X_train,y=Y_train, epochs=200, batch_size=20)
An error occurred while creating the convolutional layer of the CNN model.
I do not know the cause of the error and how to handle it. I'm in the process of implementing CNN using keras and cifar10 for machine learning with Python 3.5.
I get the following error in the learning phase:
Error when checking target: expected dense_46 to have shape (10,) but got array with shape (1,)
It does not work well with reshaping the array of labels (y_train).
#import
import tensorflow as tf
import keras
import numpy as np
import matplotlib.pyplot as plt
#get data
from keras.datasets import cifar10
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
#Image normalization
x_train = x_train / 255.0
x_test = x_test / 255.0
#Build a CNN mode
def CNN_model():
model = keras.models.Sequential()
#1st convolutional layer
model.add(keras.layers.Conv2D(96, 11, strides=(4, 4), padding='valid', data_format='channels_last', activation='relu', bias_initializer='ones', input_shape=(32,32,3)))
model.add(keras.layers.normalization.BatchNormalization(axis=1))
model.add(keras.layers.MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid', data_format='channels_last'))
#2nd convolutional layer
model.add(keras.layers.Conv2D(256, 2, strides=(4, 4), padding='valid', data_format='channels_last', activation='relu', bias_initializer='zeros'))
model.add(keras.layers.normalization.BatchNormalization(axis=1))
model.add(keras.layers.MaxPooling2D(pool_size=(1, 1), strides=(2, 2), padding='valid', data_format='channels_last'))
#3rd layer
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(4096))
model.add(keras.layers.Dropout(0.5))
model.add(keras.layers.Dense(4096))
model.add(keras.layers.Dropout(0.5))
#4th Softmax layer
model.add(keras.layers.Dense(10, activation='softmax'))
#compile
model.compile(optimizer=keras.optimizers.SGD(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy'])
return model
#Summary
model = CNN_model()
model.summary()
#Fitting
early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, verbose=1, mode='auto')
history = model.fit(x_train, y_train, epochs=15, batch_size=128, shuffle=True, validation_split=0.25, callbacks=[early_stopping])
The problem here lies in your target aka y_train. Yours is probably a 1 x n_samples long vector with 10 different lables in. The way Keras needs it is in a One-Hot-Encoded way. You can do this by:
y_train = to_categorical(y_train)
Hope it helps!
I'm having problems in trying Keras with MNIST. I have a saved model that has more than 99% accuracy but when I use it to predict some images it always predics a 1. I think it's due to me reshaping the image data input in the wrong way in the test.py file.
I got the error:
ValueError: Error when checking : expected conv2d_1_input to have 4 dimensions, but got array with shape (28, 28)
Or if I try a random reshape (1, 1, 28, 28) I get this error:
ValueError: Error when checking : expected conv2d_1_input to have shape (None, 28, 28, 1) but got array with shape (1, 1, 28, 28)
So I tried adding the following in my image_to_data function:
image_data = image_data.reshape((1, 28, 28, 1))
Now the code runs but always predicts the same values. How can I reshape the image data 28 x 28 pixels to that it fits the first layer in the model in the correct way for predicting the class of one image?
train.py
from __future__ import print_function
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
batch_size = 128
num_classes = 10
epochs = 20
# input image dimensions
img_rows, img_cols = 28, 28
# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
if K.image_data_format() == 'channels_first':
x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
input_shape = (1, img_rows, img_cols)
else:
x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
input_shape = (img_rows, img_cols, 1)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
# serialize model to YAML
model_yaml = model.to_yaml()
with open("model-new.yaml", "w") as yaml_file:
yaml_file.write(model_yaml)
# serialize weights to HDF5
model.save_weights("model-new.h5")
print("Saved model to disk")
test.py
from PIL import Image
from keras.models import model_from_yaml
import numpy as np
def load_model():
# load YAML and create model
yaml_file = open('model.yaml', 'r')
model_yaml = yaml_file.read()
yaml_file.close()
model = model_from_yaml(model_yaml)
# load weights into new model
model.load_weights("model.h5")
print("Loaded model from disk")
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
return model
def image_to_data(image):
image_data = np.array(image) / 255
image_data = image_data.reshape((1, 28, 28, 1))
return image_data
def predict(model, image):
data = image_to_data(image)
prediction = model.predict_classes(data)
return prediction
def predict_image(model, filename):
image = Image.open(filename)
data = image_to_data(image)
prediction = predict(model, data)
return prediction
model = load_model()
print(predict_image(model, '3.png'))
print(predict_image(model, '6.png'))
print(predict_image(model, '8.png'))
Possible problems:
(not your case) MNIST data is normalized between 0 and 1, and your image may be from 0 to 255 as usual (compare image_data.max() with x_train.max())
MNIST data may have black and white colors inverted in relation to your images. After assuring everything is normalized between 0 and 1, use a tool to plot an image from x_train and to plot image_data. See if the colors are inverted. Or try predicing with image_data = 1 - image_data.
Depending on the way you're loading your images, you may have it transposed. After checking the two previous items, you may try to image_data = numpy.swapaxes(image_data,1,2)
Overfitting, as mentioned by #hi_im_vinzent. If all the three previous items are ok, try predicting with the training images to see if the model is doing it right.
If none of the previous worked, then you've probably got a problem when saving/loading the model.