Data dimensionality problem with tensorflow sequential model - python

Being new to Keras sequential models is causing me a few troubles!
I have an x_train of shape : 17755 x 500 x 12
and y_train of shape: 17755 x 15 (labels are already one-hot encoded)
And I made the next model to be trained on this data:
model = Sequential()
model.add(Conv2D(32,3,padding="same", activation="relu", input_shape=(17755,500,12)))
model.add(MaxPool2D())
model.add(Conv2D(32, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(128,activation="relu"))
model.add(Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)
1- when I don’t use np.expand_dims to add an axis for batch, I get this error:
ValueError: Input 0 of layer "sequential" is incompatible with the
layer: expected shape=(None, 17755, 500, 12), found shape=(None, 500,
12)
2- when I do use np.expand_dims and the shape of x_train became: 1x17755x500x12
I get this error:
Data cardinality is ambiguous:
x sizes: 1
y sizes: 17755
Make sure all arrays contain the same number of samples.
3- when I use np.expand_dims for y_train too and its shape became: 1x17755x15
I get this error:
ValueError: Shapes (None, 17755, 15) and (None, 15) are incompatible
I know I’m doing something fundamentally wrong, but what what is that? Can anyone please help me out with the shape of data please?

Regarding x_train try adding a new dimension at the end to represent the channel dimension needed for Conv2D layers. Note also that you do not provide the number of samples to your input shape. Here is a working example:
import tensorflow as tf
import numpy as np
x_train = np.random.random((17755,500,12))
x_train = np.expand_dims(x_train, axis=-1)
y_train = np.random.random((17755,15))
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32,3,padding="same", activation="relu", input_shape=(500, 12, 1)))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(32, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Dropout(0.4))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128,activation="relu"))
model.add(tf.keras.layers.Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)

Related

binary signal data: keras ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=3, found ndim=2

I'm attempting to train models for RF fingerprinting, and have captured samples from a number of devices at a length of 1 million each. I've converted the samples into a variety of images, and have successfully trained models using that form of data by means of:
imageSize = 224
x_train = np.array(x_train) / 255
x_train.reshape(-1, imageSize, imageSize, 1)
x_val = np.array(x_val) / 255
x_val.reshape(-1, imageSize, imageSize, 1)
y_train = np.array(y_train)
y_val = np.array(y_val)
model = Sequential()
model.add(Conv2D(96, 7, padding="same", activation="relu", input_shape = (224, 224, 3)))
model.add(MaxPool2D())
model.add(Conv2D(96, 7, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Conv2D(192, 7, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(384, activation="relu"))
model.add(Dense(6, activation="softmax"))
opt = Adam(learning_rate=0.000001)
model.compile(optimizer = opt, loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=["accuracy"])
model.summary()
history = model.fit(x_train, y_train, epochs = 500, validation_data = (x_val, y_val))
However, attempting to do the same to the array data (shape (60, 4000)) which was used to create the images yields the "ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=3, found ndim=2" issue listed in the title. My code for that is:
x_train = np.array(x_train)
x_train.reshape(-1, 4000, 1)
x_val = np.array(x_val)
x_val.reshape(-1, 4000, 1)
y_train = np.array(y_train)
y_val = np.array(y_val)
model = Sequential()
model.add(Conv1D(96, 7, padding="same", activation="relu", input_shape=(4000, 1)))
model.add(MaxPooling1D())
model.add(Conv1D(96, 7, padding="same", activation="relu"))
model.add(MaxPooling1D())
model.add(Conv1D(192, 7, padding="same", activation="relu"))
model.add(MaxPooling1D())
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(384, activation="relu"))
model.add(Dense(6, activation="softmax"))
opt = Adam(learning_rate=0.000001)
model.compile(optimizer = opt, loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=["accuracy"])
model.summary()
history = model.fit(x_train, y_train, epochs = 500, validation_data = (x_val, y_val))
Like many it seems, I'm unable to figure out why this input shape isn't working for the array data. Any clarifications will be helpful.
The error: expected min_ndim=3, found ndim=2 clearly explains all. The first problem, you used input shape is in three dimension (224, 224, 3), while for the second one the input shape changed to 1 dimensional array of shape (4000, 1). You should reshape the dimension of your input to the sequential model.

Keras model ValueError: Can not squeeze dim[1], expected a dimension of 1, got 90

My current model is:
# from tensorflow.keras.layers import InputLayer
model_training = Sequential()
# input_layer = keras.Input(shape=(300,1))
model_training.add(InputLayer(input_shape=(300,1)))
model_training.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='tanh'))
model_training.add(Dropout(0.2))
model_training.add(MaxPooling1D(pool_size=3))
model_training.add(Dropout(0.2))
model_training.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='tanh'))
model_training.add(Dropout(0.2))
model_training.add(MaxPooling1D(pool_size=3))
# model_training.add(Dropout(0.2))
# model_training.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='tanh'))
# model_training.add(Dropout(0.2))
# model_training.add(MaxPooling1D(pool_size=3))
# model_training.add(Dropout(0.2))
# model_training.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='tanh'))
# model_training.add(Dropout(0.2))
# model_training.add(MaxPooling1D(pool_size=3))
# model_training.add(Dropout(0.2))
#model.add(Dropout(0.2))
model_training.add(Flatten())
model_training.add(Dense(90))
model_training.add(Activation('sigmoid'))
model_training.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model_training.summary())
My fit function:
model_training.fit(train_data, train_labels, validation_data=(test_data, test_labels), batch_size=32, epochs=15)
I get this error when I run this:
ValueError: Can not squeeze dim[1], expected a dimension of 1, got 90 for '{{node Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](remove_squeezable_dimensions/Squeeze)' with input shapes: [?,90].
Any idea?
my output layer has 90 as there are 90 classes in total to give a prediction on.
The train and labels are shaped as follows:
(7769, 300, 1)
(7769, 90, 1)
I can't figure out this issue. Any help is appreciated!
Partial model summary:
Squeeze your labels before training:
train_labels = tf.squeeze(train_labels, axis=-1)
It seems like the shape of your labels is the problem. The model will output a shape of (batch, 90), but you are providing (batch, 90, 1). Keras is unable to squeeze dimension 1 because it has a length of 90 and not 1.

Input shape error with Keras - not sure what's going on?

This code was given to us by a teacher, so it should work right off the bat. However, I can't get it to run.
K.image_data_format()
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# load data
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
# normalize inputs from 0-255 to 0.0-1.0
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train = X_train / 255.0
X_test = X_test / 255.0
# one hot encode outputs
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]
# Create the model
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(3, 32, 32), padding='same', activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.2))
model.add(Conv2D(32, (3, 3), activation='relu', padding='same', kernel_constraint=maxnorm(3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(512, activation='relu', kernel_constraint=maxnorm(3)))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
# Compile model
epochs = 25
lrate = 0.01
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
print(model.summary())
# Fit the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=epochs, batch_size=32)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Accuracy: %.2f%%" % (scores[1]*100))
I receive the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 32 but received input with shape [None, 32, 32, 3]
Just thrown off since no one else had this issue with the given code. I did have to change the first line from
K.set_image_dim_ordering('th')
to
K.image_data_format
as I was being told that set_image_dim_ordering was not a known function of Keras.backend
Any ideas here? Could my change has introduced this error?
Update: If data is in channel_last format, then change input shape from input_shape=(3, 32, 32) to input_shape=(32, 32, 3) in,
model.add(Conv2D(32, (3, 3), input_shape=(3, 32, 32), padding='same', activation='relu', kernel_constraint=maxnorm(3)))
I found reference to set_image_dim_ordering below for keras 1.2.2,
https://faroit.com/keras-docs/1.2.2/backend/
Here, it mentions,
For 2D data (e.g. image), "tf" assumes (rows, cols, channels) while
"th" assumes (channels, rows, cols).
Since you are receving input shape [None, 32, 32, 3] that means it is tf format of (rows, cols, channels). So providing th means the input shapes mismatch. You were using theano backend with th instead of tensorflow.
I could not find any reference to set_image_dim_ordering or image_dim_ordering in latest tf.keras backend. Instead these below provide method for getting and setting 'channels_first' or 'channels_last' format.
https://www.tensorflow.org/api_docs/python/tf/keras/backend/image_data_format
https://www.tensorflow.org/api_docs/python/tf/keras/backend/set_image_data_format

Keras - "ValueError: Error when checking target: expected activation_1 to have shape (None, 9) but got array with shape (9,1)

I'm building a model to classify text into one of 9 layers, and am having this error when running it. Activation 1 seems to refer to the Convolutional layer's input, but I'm unsure about what's wrong with the input.
num_classes=9
Y_train = keras.utils.to_categorical(Y_train, num_classes)
#Reshape data to add new dimension
X_train = X_train.reshape((100, 150, 1))
Y_train = Y_train.reshape((100, 9, 1))
model = Sequential()
model.add(Conv1d(1, kernel_size=3, activation='relu', input_shape=(None, 1)))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x=X_train,y=Y_train, epochs=200, batch_size=20)
Running this results in the following error:
"ValueError: Error when checking target: expected activation_1 to have shape (None, 9) but got array with shape (9,1)
There are several typos and bugs in your code.
Y_train = Y_train.reshape((100,9))
Since you reshape X_train to (100,150,1), I guess your input step is 150, and channel is 1. So for the Conv1D, (there is a typo in your code), input_shape=(150,1).
You need to flatten your output of conv1d before feeding into Dense layer.
import keras
from keras import Sequential
from keras.layers import Conv1D, Dense, Flatten
X_train = np.random.normal(size=(100,150))
Y_train = np.random.randint(0,9,size=100)
num_classes=9
Y_train = keras.utils.to_categorical(Y_train, num_classes)
#Reshape data to add new dimension
X_train = X_train.reshape((100, 150, 1))
Y_train = Y_train.reshape((100, 9))
model = Sequential()
model.add(Conv1D(2, kernel_size=3, activation='relu', input_shape=(150,1)))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x=X_train,y=Y_train, epochs=200, batch_size=20)

Python - Tensorflow - LSTM- ValueError: Error when checking model target: expected dense_16 to have shape (None, 100) but got array with shape (16, 2)

Could someone help me to understand what this error is all about?
model = Sequential()
model.add(Embedding(82, 100, weights=[embedding_matrix], input_length=1000))
model.add(LSTM(100))
model.add(Dense(100, activation = 'sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())
model.fit(x_train, y_train, epochs = 5, batch_size=64)
When i run this LSTM model, I am getting an error as
ValueError: Error when checking model target: expected dense_16 to have shape (None, 100) but got array with shape (16, 2)
I am not sure how much the below information would be useful:
x_train.shape
Out[959]: (16, 1000)
y_train.shape
Out[962]: (16, 2)
If you need any other information, I am ready to provide
you have defined dense layer input shape is 100.
model.add(Dense(100, activation = 'sigmoid'))
so you need to make sure your input should always same shape.
here in your case make x_train and and y_train same shape.
try with :
model = Sequential()
# here the batch dimension is None,
# which means any batch size will be accepted by the model.
model.add(Dense(32, batch_input_shape=(None, 500)))
model.add(Dense(32))
Your last layer has an output shape of None,100
model.add(Dense(100, activation = 'sigmoid'))
But your data (y_train) has the shape (16,2). It should be
model.add(Dense(2, activation = 'sigmoid'))

Categories