We Keep Coding

ValueError: Error when checking target: expected embedding_37 to have 3 dimensions, but got array with shape (90, 1)

I have trained a model for CNN and I am getting the error on the Embedding layer and I use my pre trained embedding to classification
my code
embed_size = 300
max_features = 100000
maxlen = 254
num_filters = 100
nb_words = min(max_features , len(word_index)) + 1
model = Sequential()
model.add(Embedding(nb_words, embed_size, trainable=False))
model.add(Conv1D(num_filters, 7, activation='relu', padding='same'))
model.add(MaxPooling1D(2))
model.add(Conv1D(num_filters, 7, activation='relu', padding='same'))
Model Summary
Model: "sequential_47"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_38 (Embedding) (None, None, 300) 537900
_________________________________________________________________
conv1d_39 (Conv1D) (None, None, 100) 210100
_________________________________________________________________
max_pooling1d_19 (MaxPooling (None, None, 100) 0
_________________________________________________________________
conv1d_40 (Conv1D) (None, None, 100) 70100
=================================================================
Total params: 818,100
Trainable params: 280,200
Non-trainable params: 537,900
_________________________________________________________________
I tried to train model
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0.01, patience=4, verbose=1)
callbacks_list = [early_stopping]
model.fit(train_X, y_train, batch_size=1000, epochs=10, verbose=1,
callbacks=callbacks_list)
ValueError: Error when checking target: expected embedding_37 to have 3 dimensions, but got array with shape (90, 1)
train_X.shape
(90, 352)
Any help will be greatly appreciated.
Thank you

Keras Conv2D CNN - Error when checking target - expected smaller output

I am stacking 6 layers of 2D satellite imagery (x data) and attempting to run a CNN over them to classify the landcover (using 8 land cover classes taken from a reformatted USDA Crop Data Layer - y data).
The x data is shaped (2004, 2753, 6) and the y is shaped (2004, 2753, 8) originally and I have used data_x.reshape(-1,2004,2752,6) (same for y) to add an extra dimension as the model.
The 8 categories in the y data-set represent 8 possible land-cover categories in numerical format in 8 bands (i.e. 1st band is corn and represented by 1's for positive and 0 for not corn).
However, when i try to run the model the expected shape does not match what is being passed through to it. I am unsure if I am using the correct model structure or data structure - one idea would be to take the 8 bands of the y dataset
Based on some serious googling i have been learning how to get the data into the correct format with the right number of dimensions etc but feel I am falling at the last hurdle with regards to dimensions (and most likely correct preparation of the x & y data sets).
Below is the CNN model
input_shape=([2004, 2753, 6])
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),strides=(1, 1),activation='relu',input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2,2), strides=(2, 2), padding="same"))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2), padding="same"))
model.add(Dropout(0.25))
#model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(8, activation='softmax'))
#model.add(Flatten())
model.summary()
Model Summary - expecting 500, 687, 8 out at the end
Layer (type) Output Shape Param #
=================================================================
conv2d_54 (Conv2D) (None, 2002, 2751, 32) 1760
_________________________________________________________________
max_pooling2d_52 (MaxPooling (None, 1001, 1376, 32) 0
_________________________________________________________________
conv2d_55 (Conv2D) (None, 999, 1374, 32) 9248
_________________________________________________________________
max_pooling2d_53 (MaxPooling (None, 500, 687, 32) 0
_________________________________________________________________
dropout_57 (Dropout) (None, 500, 687, 32) 0
_________________________________________________________________
dense_59 (Dense) (None, 500, 687, 128) 4224
_________________________________________________________________
dropout_58 (Dropout) (None, 500, 687, 128) 0
_________________________________________________________________
dense_60 (Dense) (None, 500, 687, 8) 1032
=================================================================
Total params: 16,264
Trainable params: 16,264
Non-trainable params: 0
_________________________________________________________________
compile
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
optimizer='sgd',
metrics=['accuracy'])
fit - and where i get the error message
history = model.fit(x_train3d, y_train3d,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_split=0.2, validation_data=None)
shape of x_train3D = (1, 2004, 2753, 6)
shape of y_train3D = (1, 2004, 2753, 8)
error message
ValueError: Error when checking target: expected dense_58 to have shape (500, 687, 8) but got array with shape (2004, 2753, 8)
Again, I suspect this is down to needing to get the data in the right format both for the input and output but also likely something wrong in the specification of the model. Would appreciate some guidance as i'm new to Keras.

Can you please explain what are you trying to classifiy and what is your expected y_train3D (is it an image or some value for classificaton e.g. 1/2/3.. or x/y/z..etc)

Just for an update on this - I have managed to clear the error (and now onto a memory error but that's another question).
Solved the issue in 2 ways.
1. Added upsampling to the end of the model to get the data back into the original size - new code in below
model = Sequential()
model.add(Conv2D(32, (3, 3), padding="same", activation="relu",input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2, 2),strides=(2, 2)))
model.add(Conv2D(64, (3, 3), padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(128, (3, 3), padding="same", activation="relu"))
#Upsampling
model.add(UpSampling2D(size=(2,2),interpolation='nearest'))
model.add(UpSampling2D(size=(2,2),interpolation='nearest'))
model.add(Dense(8, activation='relu'))
model.summary()
Give me the below summary
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 2004, 2752, 32) 1760
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 1002, 1376, 32) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 1002, 1376, 64) 18496
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 501, 688, 64) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 501, 688, 128) 73856
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 1002, 1376, 128) 0
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 2004, 2752, 128) 0
_________________________________________________________________
dense_1 (Dense) (None, 2004, 2752, 8) 1032
=================================================================
Total params: 95,144
Trainable params: 95,144
Non-trainable params: 0
Part 2 - was ensuring the x and y data arrays were dividable by 4, otherwise this meant as I was losing some of the data through the model through rounding. The below is specific to my code and not robust but worked
if x_train3d.shape[2] % 2:
x_train3d_adj = x_train3d_adj[:,:,:-1,:]
y_train3d_adj = y_train3d_adj[:,:,:-1,:]
Not a complete solution yet but does get me closer to the end goal

1D Convolutional Neural Network Input Shape `ValueError`

I am generating a 1D Convolution and have some trouble with the input shape of my data. I had a look at some posts and it seems the error was that the data has to be 3D but my data is already 3D.
# shape
# x_train shape: (1228, 1452, 20)
# y_train shape: (1228, 1452, 8)
# x_val shape: (223, 680, 20)
# x_val shape: (223, 680, 8)
###
n_outputs = 8
n_timesteps = 1452
n_features = 20
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(x_train.shape[1:]))) # ie 1452, 20
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(n_outputs, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=9,
batch_size=64,
shuffle=True)
But I keep on getting this error message:
ValueError: A target array with shape (1228, 1452, 8) was passed for an output of shape (None, 8) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
What I gather from this is that target shape which is 3 dimensional is not the same as the 2 dimensional output and so it can't work out the loss, but I just need to find a way to reshape so that they are equal.
EDIT
model.summary() is shown below
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d (Conv1D) (None, 1450, 64) 3904
_________________________________________________________________
conv1d_1 (Conv1D) (None, 1448, 64) 12352
_________________________________________________________________
dropout (Dropout) (None, 1448, 64) 0
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 724, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 46336) 0
_________________________________________________________________
dense (Dense) (None, 100) 4633700
_________________________________________________________________
dense_1 (Dense) (None, 8) 808
=================================================================
Total params: 4,650,764
Traceback (most recent call last):
Trainable params: 4,650,764
Non-trainable params: 0

The issue in my case was that the target vector was 3D whilst the output vector was 2D and so there is an obvious mismatch. To fix the issue change the shape of the y_train to (batch, 8) or use return_sequences=True to return the same shape from the previous LSTM layer.

Keras: How to connect a CNN model with a decision tree

I want to train a model to predict one's emotion from the physical signals. I have a physical signal and using it as input feature;
ecg(Electrocardiography)
I want to use the CNN architecture to extract features from the data, and then use these extracted features to feed a classical "Decision Tree Classifier". Below, you can see my CNN aproach without the decision tree;
model = Sequential()
model.add(Conv1D(15,60,padding='valid', activation='relu',input_shape=(18000,1), strides = 1, kernel_regularizer=regularizers.l1_l2(l1=0.1, l2=0.1)))
model.add(MaxPooling1D(2,data_format='channels_last'))
model.add(Dropout(0.6))
model.add(BatchNormalization())
model.add(Conv1D(30, 60, padding='valid', activation='relu',kernel_regularizer = regularizers.l1_l2(l1=0.1, l2=0.1), strides=1))
model.add(MaxPooling1D(4,data_format='channels_last'))
model.add(Dropout(0.6))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(3, activation = 'softmax'))
I want to edit this code so that, in the output layer there will be working decision tree instead of model.add(Dense(3, activation = 'softmax')). I have tried to save the outputs of the last convolutional layer like this;
output = model.layers[-6].output
And when I printed out the output variable, result was this;
THE OUTPUT: Tensor("conv1d_56/Relu:0", shape=(?, 8971, 30),
dtype=float32)
I guess, the output variable holds the extracted features. Now, how can I feed my decision tree classifier model with this data which is stored in the output variable? Here is the decision tree from scikit learn;
from sklearn.tree import DecisionTreeClassifier
dtc = DecisionTreeClassifier(criterion = 'entropy')
dtc.fit()
How should I feed the fit() method? Thanks in advance.

To extract a vector of features that you can pass on to another algorithm, you need a fully connected layer before your softmax layer. Something like this will add in a 128 dimensional layer just before your softmax layer:
model = Sequential()
model.add(Conv1D(15,60,padding='valid', activation='relu',input_shape=(18000,1), strides = 1, kernel_regularizer=regularizers.l1_l2(l1=0.1, l2=0.1)))
model.add(MaxPooling1D(2,data_format='channels_last'))
model.add(Dropout(0.6))
model.add(BatchNormalization())
model.add(Conv1D(30, 60, padding='valid', activation='relu',kernel_regularizer = regularizers.l1_l2(l1=0.1, l2=0.1), strides=1))
model.add(MaxPooling1D(4,data_format='channels_last'))
model.add(Dropout(0.6))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation = 'softmax'))
If you then run model.summary() you can see the name of the layers:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_9 (Conv1D) (None, 17941, 15) 915
_________________________________________________________________
max_pooling1d_9 (MaxPooling1 (None, 8970, 15) 0
_________________________________________________________________
dropout_10 (Dropout) (None, 8970, 15) 0
_________________________________________________________________
batch_normalization_9 (Batch (None, 8970, 15) 60
_________________________________________________________________
conv1d_10 (Conv1D) (None, 8911, 30) 27030
_________________________________________________________________
max_pooling1d_10 (MaxPooling (None, 2227, 30) 0
_________________________________________________________________
dropout_11 (Dropout) (None, 2227, 30) 0
_________________________________________________________________
batch_normalization_10 (Batc (None, 2227, 30) 120
_________________________________________________________________
flatten_6 (Flatten) (None, 66810) 0
_________________________________________________________________
dense_7 (Dense) (None, 128) 8551808
_________________________________________________________________
dropout_12 (Dropout) (None, 128) 0
_________________________________________________________________
dense_8 (Dense) (None, 3) 387
=================================================================
Total params: 8,580,320
Trainable params: 8,580,230
Non-trainable params: 90
_________________________________________________________________
Once your network has been trained you can create a new model where the output layer becomes 'dense_7' and it'll generate 128 dimensional feature vectors:
feature_vectors_model = Model(model.input, model.get_layer('dense_7').output)
dtc_features = feature_vectors_model.predict(your_X_data) # fit your decision tree on this data

Keras 2D input to 2D output

First, I have read this and this questions with similar names to mine and still do not have an answer.
I want to build a feedforward network for sequence prediction. (I realize that RNNs are more suitable for this task, but I have my reasons). The sequences are of length 128 and each element is a vector with 2 entries, so each batch should be of shape (batch_size, 128, 2) and the target is the next step in the sequence, so the target tensor should be of shape (batch_size, 1, 2).
The network architecture is something like this:
model = Sequential()
model.add(Dense(50, batch_input_shape=(None, 128, 2), kernel_initializer="he_normal" ,activation="relu"))
model.add(Dense(20, kernel_initializer="he_normal", activation="relu"))
model.add(Dense(5, kernel_initializer="he_normal", activation="relu"))
model.add(Dense(2))
But trying to train I get the following error:
ValueError: Error when checking target: expected dense_4 to have shape (128, 2) but got array with shape (1, 2)
I've tried variations like:
model.add(Dense(50, input_shape=(128, 2), kernel_initializer="he_normal" ,activation="relu"))
but get the same error.

If you take a look at the model.summary() output you will see that what the issue is:
Layer (type) Output Shape Param #
=================================================================
dense_13 (Dense) (None, 128, 50) 150
_________________________________________________________________
dense_14 (Dense) (None, 128, 20) 1020
_________________________________________________________________
dense_15 (Dense) (None, 128, 5) 105
_________________________________________________________________
dense_16 (Dense) (None, 128, 2) 12
=================================================================
Total params: 1,287
Trainable params: 1,287
Non-trainable params: 0
_________________________________________________________________
As you can see, the output of the model is (None, 128,2) and not (None, 1, 2) (or (None, 2)) as you expected. So, you may or may not know that Dense layer is applied on the last axis of its input array and as a result, as you see above, the time axis and dimension is preserved until the end.
How to resolve this? You mentioned you don't want to use a RNN layer, therefore you have two options: you need to either use Flatten layer somewhere in the model or you can also use some Conv1D + Pooling1D layers or even a GlobalPooling layer. For example (these are just for demonstration, you may do it differently):
using Flatten layer
model = models.Sequential()
model.add(Dense(50, batch_input_shape=(None, 128, 2), kernel_initializer="he_normal" ,activation="relu"))
model.add(Dense(20, kernel_initializer="he_normal", activation="relu"))
model.add(Dense(5, kernel_initializer="he_normal", activation="relu"))
model.add(Flatten())
model.add(Dense(2))
model.summary()
Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_17 (Dense) (None, 128, 50) 150
_________________________________________________________________
dense_18 (Dense) (None, 128, 20) 1020
_________________________________________________________________
dense_19 (Dense) (None, 128, 5) 105
_________________________________________________________________
flatten_1 (Flatten) (None, 640) 0
_________________________________________________________________
dense_20 (Dense) (None, 2) 1282
=================================================================
Total params: 2,557
Trainable params: 2,557
Non-trainable params: 0
_________________________________________________________________
using GlobalAveragePooling1D layer
model = models.Sequential()
model.add(Dense(50, batch_input_shape=(None, 128, 2), kernel_initializer="he_normal" ,activation="relu"))
model.add(Dense(20, kernel_initializer="he_normal", activation="relu"))
model.add(GlobalAveragePooling1D())
model.add(Dense(5, kernel_initializer="he_normal", activation="relu"))
model.add(Dense(2))
model.summary()
​Model summary:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_21 (Dense) (None, 128, 50) 150
_________________________________________________________________
dense_22 (Dense) (None, 128, 20) 1020
_________________________________________________________________
global_average_pooling1d_2 ( (None, 20) 0
_________________________________________________________________
dense_23 (Dense) (None, 5) 105
_________________________________________________________________
dense_24 (Dense) (None, 2) 12
=================================================================
Total params: 1,287
Trainable params: 1,287
Non-trainable params: 0
_________________________________________________________________
Note that in both cases above you need to reshape the labels (i.e. targets) array to (n_samples, 2) (or you may want to use a Reshape layer at the end).