I'm working on a first model in Keras, just to get started. I downloaded the MNIST data from Kaggle, which has preprocessed a subset of the data into a csv with a label column and 784 (28x28) greyscale pixel values.
I'm getting a ValueError: Shapes (None, 1) and (None, 10) are incompatible. I can't understand why.
Here's my code:
## load csv and partition into train/dev sets
dataframe = pd.read_csv('data/train.csv') ##load as pandas
dev_df = dataframe.sample(n=3000, random_state=1)
train_df = dataframe.drop(dev_df.index)
assert train_df.shape[1] == 785 #make sure it's not a problem with the data shape.
##Build the model
inputs = keras.Input(shape=(784))
x = layers.experimental.preprocessing.Rescaling(1./255)(inputs)
x = layers.Dense(100, activation='relu')(x)
x = layers.Dense(100, activation='relu')(x)
outputs = layers.Dense(10, activation='softmax')(x)
model = keras.Model(inputs=inputs,outputs=outputs)
model.summary()
At this point, I get the following model summary, which looks right to me (i.e., it looks a lot like what I'm seeing in the guides that I'm following.)
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 784)] 0
_________________________________________________________________
rescaling_6 (Rescaling) (None, 784) 0
_________________________________________________________________
dense_3 (Dense) (None, 100) 78500
_________________________________________________________________
dense_4 (Dense) (None, 100) 10100
_________________________________________________________________
dense_5 (Dense) (None, 10) 1010
=================================================================
Total params: 89,610
Trainable params: 89,610
Non-trainable params: 0
Then
## compile and fit model.
model.compile(optimizer="adam", loss="categorical_crossentropy")
(X, y) = (train_df.drop('label', axis=1).to_numpy(), train_df['label'].to_numpy())
assert X.shape[1] == 784 ## we're really sure this is the right size.
assert X.shape[0] == y.shape[0] ## the number of labels matches the number of samples
history = model.fit(X, y, epochs=1, batch_size=64)
The last line raises the error. I would guess it emerges with the final dense layer, since it has the expected (None,10) shape, but I can't figure out where the (None, 1) shaped entity (whatever it is) comes from.
Related
I am working on a gesture recognition problem. For that I have a train set. Train set consists of multiple folders and each folder consists of a series of 30 images. From those images the model is trained. Also I have a csv file that contains the class label of each folder. The class labels are : "Left Swipe", "Right Swipe", "Stop", "Thumbs Down" and "Thumbs Up". Those labels are present in one np.array variable train_class. Now, I have created a CNN model then feeding that in a Sequential model.
The code is available in below GIT location
https://github.com/subhrajyoti-ghosh/ML-and-Deep-Learning/blob/main/Gesture_Recognition.ipynb
But when I am trying to fit the model, I am receiving error. Can you please help me understanding the error and how to solve that?
You are trying to use a TimeDistributed layer on a 2D input (batch_size, 256), which will not work, because the layer needs at least a 3D tensor. You should try using tf.keras.layers.RepeatVector:
import tensorflow as tf
resnet = tf.keras.applications.ResNet50(include_top=False,weights='imagenet',input_shape=(224,224,3))
cnn = tf.keras.Sequential([resnet])
cnn.add(tf.keras.layers.Conv2D(64,(2,2),strides=(1,1)))
cnn.add(tf.keras.layers.Conv2D(16,(3,3),strides=(1,1)))
cnn.add(tf.keras.layers.Flatten())
inputs = tf.keras.layers.Input(shape=(224,224,3))
x = cnn(inputs)
x = tf.keras.layers.RepeatVector(n=30)(x)
x = tf.keras.layers.GRU(16,return_sequences=True)(x)
x = tf.keras.layers.GRU(8)(x)
outputs = tf.keras.layers.Dense(5,activation='softmax')(x)
model = tf.keras.Model(inputs, outputs)
dummy_x = tf.random.normal((1, 224,224,3))
print(model.summary())
print(model(dummy_x))
Model: "model_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_14 (InputLayer) [(None, 224, 224, 3)] 0
sequential_6 (Sequential) (None, 256) 24121296
repeat_vector_2 (RepeatVect (None, 30, 256) 0
or)
gru_5 (GRU) (None, 30, 16) 13152
gru_6 (GRU) (None, 8) 624
dense_7 (Dense) (None, 5) 45
=================================================================
Total params: 24,135,117
Trainable params: 24,081,997
Non-trainable params: 53,120
_________________________________________________________________
None
I have some data I'm trying to make a simple TensorFlow neural network for. I've preprocessed the data so its shape is (18631, 147), so around 18000 rows of 147 values each. each row is its own training data, and I have another dataframe with shape (18631, 3) that has the ground truths for three different variables I might try predicting, labeled 1, 2 and 3. When I try to run model.fit, I get a weird error I don't understand:
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 147 but received input with shape (147, 1)
The way I try to prepare my data for input into TensorFlow is with the from_tensor_slices() function, specifically with the code:
#data is the (18631, 147) dataframe, target is the (18631, 3).
lat_target = target.pop(1)
dataset = tensorflow.data.Dataset.from_tensor_slices((data.values,lat_target))
I then make my model with:
input_layer = tensorflow.keras.Input((147,))
first_layer = tensorflow.keras.layers.Dense(32)(input_layer)
second_layer = tensorflow.keras.layers.Dense(32)(first_layer)
last_layer = tensorflow.keras.layers.Dropout(.3)(second_layer)
pred_layer = tensorflow.keras.layers.Dense(1)(last_layer)
model = tensorflow.keras.Model(inputs=input_layer,outputs=pred_layer)
model.compile(optimizer="adam",loss=tensorflow.keras.losses.CategoricalCrossentropy(from_logits=True),metrics=['accuracy'])
and try to train with:
model.fit(dataset,epochs=20)
which is when I get the error message. I think the main problem is I don't understand what the difference between a shape with value 147 and an input with shape (147,1) is, but I'm also not sure I didn't make an obvious mistake somewhere else in the code.
Your input shape is (147,1) but you are feeding input= 147.
This should work
import tensorflow as tf
input_layer = tf.keras.Input((147,1))
first_layer = tf.keras.layers.Dense(32)(input_layer)
second_layer = tf.keras.layers.Dense(32)(first_layer)
last_layer = tf.keras.layers.Dropout(.3)(second_layer)
pred_layer = tf.keras.layers.Dense(1)(last_layer)
model = tf.keras.Model(inputs=input_layer,outputs=pred_layer)
model.summary()
Output
Model: "model_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_4 (InputLayer) [(None, 147, 1)] 0
dense_9 (Dense) (None, 147, 32) 64
dense_10 (Dense) (None, 147, 32) 1056
dropout_3 (Dropout) (None, 147, 32) 0
dense_11 (Dense) (None, 147, 1) 33
=================================================================
Total params: 1,153
Trainable params: 1,153
Non-trainable params: 0
i'm fairly new to tensorflow and would appreciate answers a lot.
i'm trying to use a transformer model as an embedding layer and feed the data to a custom model.
from transformers import TFAutoModel
from tensorflow.keras import layers
def build_model():
transformer_model = TFAutoModel.from_pretrained(MODEL_NAME, config=config)
input_ids_in = layers.Input(shape=(MAX_LEN,), name='input_ids', dtype='int32')
input_masks_in = layers.Input(shape=(MAX_LEN,), name='attention_mask', dtype='int32')
embedding_layer = transformer_model(input_ids_in, attention_mask=input_masks_in)[0]
X = layers.Bidirectional(tf.keras.layers.LSTM(50, return_sequences=True, dropout=0.1, recurrent_dropout=0.1))(embedding_layer)
X = layers.GlobalMaxPool1D()(X)
X = layers.Dense(64, activation='relu')(X)
X = layers.Dropout(0.2)(X)
X = layers.Dense(30, activation='softmax')(X)
model = tf.keras.Model(inputs=[input_ids_in, input_masks_in], outputs = X)
for layer in model.layers[:3]:
layer.trainable = False
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
return model
model = build_model()
model.summary()
r = model.fit(
train_ds,
steps_per_epoch=train_steps,
epochs=EPOCHS,
verbose=3)
I have 30 classes and the labels are not one-hot encoded so im using sparse_categorical_crossentropy as my loss function but i keep getting the following error
ValueError: Shape mismatch: The shape of labels (received (1,)) should equal the shape of logits except for the last dimension (received (10, 30)).
how can i solve this?
and why is the (10, 30) shape required? i know 30 is because of the last Dense layer with 30 units but why the 10? is it because of the MAX_LENGTH which is 10?
my model summary:
Model: "model_16"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_ids (InputLayer) [(None, 10)] 0
__________________________________________________________________________________________________
attention_mask (InputLayer) [(None, 10)] 0
__________________________________________________________________________________________________
tf_bert_model_21 (TFBertModel) TFBaseModelOutputWit 162841344 input_ids[0][0]
attention_mask[0][0]
__________________________________________________________________________________________________
bidirectional_17 (Bidirectional (None, 10, 100) 327600 tf_bert_model_21[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_15 (Global (None, 100) 0 bidirectional_17[0][0]
__________________________________________________________________________________________________
dense_32 (Dense) (None, 64) 6464 global_max_pooling1d_15[0][0]
__________________________________________________________________________________________________
dropout_867 (Dropout) (None, 64) 0 dense_32[0][0]
__________________________________________________________________________________________________
dense_33 (Dense) (None, 30) 1950 dropout_867[0][0]
==================================================================================================
Total params: 163,177,358
Trainable params: 336,014
Non-trainable params: 162,841,344
10 is a number of sequences in one batch. I suspect that it is a number of sequences in your dataset.
Your model acting as a sequence classifier. So you should have one label for every sequence.
I'm trying to do an image recognition using Keras on Flask. While doing prediction, I encounter this error
ValueError: Input 0 of layer sequential_16 is incompatible with the layer: expected axis -1 of input shape to have value 24 but received input with shape [None, 150, 150, 3]
I kinda understand the problem but I'm not sure how to specify the shape. This is on Flask server & I don't do any training here. I use a model that I have trained before on Jupyter notebook.
This is the code
def predict(img):
# Preprocess input image
img_width, img_height = 150, 150
x = load_img(img, target_size=(img_width, img_height))
x = img_to_array(x)
x = np.expand_dims(x, axis=0)
# Load model
dependencies = {
'precision': Precision,
'recall': Recall
}
model = load_model('model.h5', custom_objects=dependencies)
# Predict
result = model.predict(x)[0]
label = np.argmax(result)
return label
The traceback says it happens on result = model.predict(x)[0]. Anyone knows how to approach this error? Tried googling but I don't find any similar error.
EDIT - Model Summary
Model: "sequential_16"
Layer (type) Output Shape Param #
=================================================================
dense_96 (Dense) (None, 32) 800
_________________________________________________________________
dense_97 (Dense) (None, 1024) 33792
_________________________________________________________________
dense_98 (Dense) (None, 512) 524800
_________________________________________________________________
dense_99 (Dense) (None, 256) 131328
_________________________________________________________________
dense_100 (Dense) (None, 128) 32896
_________________________________________________________________
dense_101 (Dense) (None, 3) 387
_________________________________________________________________
activation_16 (Activation) (None, 3) 0
=================================================================
Total params: 724,003
Trainable params: 724,003
Non-trainable params: 0
Ok, so your input shape is wrong. The input layer isn't shown in the model summary but it says that the first layer has 800 parameters.
This tells me that your input layer has dimensions [None, 24] because 24 * 32 (weights) + 32 (biases) = 800.
When you add the first dense layer it should be
model.add(Dense(32, input_shape=(150,150,3))
I am using keras with TF backend to build a simple Conv1d net. The data has the following shape:
train feature shape: (33960, 3053, 1)
train label shape: (33960, 686, 1)
I build my model with:
def create_conv_model():
inp = Input(shape=(3053, 1))
conv = Conv1D(filters=2, kernel_size=2)(inp)
pool = MaxPool1D(pool_size=2)(conv)
flat = Flatten()(pool)
dense = Dense(686)(flat)
model = Model(inp, dense)
model.compile(loss='mse', optimizer='adam')
return model
Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 3053, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 3052, 2) 6
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 1526, 2) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 3052) 0
_________________________________________________________________
dense_1 (Dense) (None, 686) 2094358
=================================================================
Total params: 2,094,364
Trainable params: 2,094,364
Non-trainable params: 0
Upon running
model.fit(x=train_feature,
y=train_label_categorical,
epochs=100,
batch_size=64,
validation_split=0.2,
validation_data=(test_feature,test_label_categorical),
callbacks=[tensorboard,reduce_lr,early_stopping])
i get the following VERY USUAL ERROR:
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (8491, 3053)
I've checked pretty much all the posts regarding this very common problem, but I've been unable to find a solution. What am i doing wrong? I don't understand what's going on. Where is the shape (8491, 3053) coming from?
Any help will be much appreciated, I am not able to make this go away.
Change validation_data=(test_feature,test_label_categorical) in model.fit function to
validation_data=(np.expand_dims(test_feature, -1),test_label_categorical)
The model is expecting validation feature of shape (8491, 3053, 1), but in above code you are providing it (8491, 3053).