I'm making a dense ML model with Keras
but I get this error
ValueError: Error when checking target: expected dense_3 to have shape (1,) but got array with shape (9,)
This is how my model is set up
get_custom_objects().update({'swish': Swish(swish)})
model = Sequential()
model.add(Dense(33, activation='swish', input_shape=(trainX.shape[1],)))
model.add(Dense(33, activation='swish'))
model.add(Dense(9, activation='softmax'))
#Train Network
model.compile(optimizer="adam", loss=keras.losses.sparse_categorical_crossentropy, metrics=["accuracy"])
model.fit(trainX, trainY, validation_split=0.2, epochs=3)
trainX and trainY are pandas DataFrames
trainX has 2 columns and trainY has 9.
I'm not sure why it says it should be (1,) since I specified the output layer to have 9 neurons.
Any help is greatly appreciated.
From Keras docs:
When using the sparse_categorical_crossentropy loss, your targets should be integer targets. If you have categorical targets, you should use categorical_crossentropy.
So you should replace
model.compile(optimizer="adam", loss=keras.losses.sparse_categorical_crossentropy, metrics=["accuracy"])
With
model.compile(optimizer="adam", loss=keras.losses.categorical_crossentropy, metrics=["accuracy"])
That is needed because trainY is a categorical target, since it has 9 columns instead of just one integer.
Related
I am trying to train a basic LSTM network, but I am running into an error with model.fit. I have two datasets, each with 3145 sequences of length 7. I want to package these two datasets into the same time step. As such, I have reshaped my x_train and y_train into the following shapes:
x_train.shape = (3145, 70, 2)
y_train.shape = (3145, 70)
As you can see, I should have 3145 samples, each with 70 timesteps, and each with 2 features and a target. I then define the following model:
model = Sequential()
model.add(LSTM(4, input_shape=(x_train.shape[1], x_train.shape[2])))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
To train, I run the line
model.fit(x_train, y_train, epochs=100, batch_size=1, verbose=2)
But this gives me the error
ValueError: Error when checking target: expected dense_23 to have shape (1,) but got array with shape (70,)
I'm confused as to why this error is occurring. With 70 time steps, I should have 70 targets right?
I would greatly appreciate any help explaining this error!
You need the same units in input layer dense than the output dim of the LSTM layer. In your case the output of the LSTM layer is 70 like you can see in the shape.
try change by
model.add(Dense(70))
or let the model infer the amount of units of the layer like this.
model.add(Dense())
I cannot find a hands on tutorial on how to structure the data for use with keras LSTM.
Data
x_train = 7300 rows where each vector is length 64.
y_train = array of 7300 items either 0's or 1's (the class).
Model
model = Sequential()
model.add(LSTM(200, dropout=0.2, recurrent_dropout=0.2, input_shape = (1, 64)))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train,
epochs = 5,
batch_size = 32,
validation_split = 0.1,
callbacks=[EarlyStopping(monitor='val_loss', patience=3, min_delta=0.0001)])
My question is simply, why doesn't this work? Why isn't is as simple as giving an 2d array of vectors and similar length y values to fit.
Keras LSTM expects input of shape [batch_size, timesteps, features]. Your data is of shape [batch_size, features].
To add the timestep dimension (where number of timesteps is 1), do the following:
x_train = np.expand_dims(x_train, axis=1)
I have received multiple diffrent ValueErrors when trying to solve following by changing many parameters.
It is a time series problem, I have data from 60 shops, 215 items, 1034 days. I have splitted 973 days for train and 61 for test.:
train_x = train_x.reshape((60, 973, 215))
test_x = test_x.reshape((60, 61, 215))
train_y = train_y.reshape((60, 973, 215))
test_y = test_y.reshape((60, 61, 215))
My model:
model = Sequential()
model.add(LSTM(100, input_shape=(train_x.shape[1], train_x.shape[2]),
return_sequences='true'))
model.add(Dense(215))
model.compile(loss='mean_squared_error', optimizer='adam', metrics=
['accuracy'])
history = model.fit(train_x, train_y, epochs=10,
validation_data=(test_x, test_y), verbose=2, shuffle=False)
ValueError: Error when checking input: expected lstm_1_input to have
shape (973, 215) but got array with shape (61, 215)
You've split your data with respect to timesteps as opposed to the samples. You need to decide on what are your samples in the first instance. For the sake of the answer I will assume these are along the first axis (assuming the data has been framed as a supervised time-series problem).
The input_size in LSTM expects the shape of (timesteps, data_dim) as explained here, and these dimensions must remain the same for each batch. In your example, samples from training and testing have different dimensions. The batch size can differ (unless specified with batch_size parameter).
Your data should be split between training and testing along the first axis. Here is an analogous example from Keras tutorials:
from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np
data_dim = 16
timesteps = 8
num_classes = 10
# expected input data shape: (batch_size, timesteps, data_dim)
model = Sequential()
model.add(LSTM(32, return_sequences=True,
input_shape=(timesteps, data_dim))) # returns a sequence of vectors of dimension 32
model.add(LSTM(32, return_sequences=True)) # returns a sequence of vectors of dimension 32
model.add(LSTM(32)) # return a single vector of dimension 32
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# Generate dummy training data
x_train = np.random.random((1000, timesteps, data_dim))
y_train = np.random.random((1000, num_classes))
# Generate dummy validation data
x_val = np.random.random((100, timesteps, data_dim))
y_val = np.random.random((100, num_classes))
model.fit(x_train, y_train,
batch_size=64, epochs=5,
validation_data=(x_val, y_val))
You will notice that timesteps is the same for training and testing data and x_train.shape[1] == x_val.shape[1]. It is the number of samples that differs along the first axis x_train.shape[0] is 1000 and x_val.shape[0] is 100.
I have an existing LSTM model that looks as follows:
model_glove1 = Sequential()
model_glove1.add(Embedding(vocabulary_size, 25, input_length=50, weights=[embedding_matrix25],trainable=False))
model_glove1.add(LSTM(32))
model_glove1.add(Dense(128, activation='relu'))
model_glove1.add(Dense(64, activation='relu'))
model_glove1.add(Dense(1, activation='softmax'))
model_glove1.compile(loss='binary_crossentropy',optimizer='adam',metrics['accuracy',auc_roc])
model_glove1.fit(data, np.array(train_y), batch_size=32,
epochs=4,
verbose=1,
validation_split=0.1,
shuffle=True)
I want to add an additional auxiliary input layer which is present in a dataframe of 27 columns . I want that layer to be concatenated with the output of the LSTM layer. Is it possible ? If so how can I achieve it?
Before using the code, please check the secondary input has the same dimension like output of LSTM layer.
Moreover, in model1_glove.fit() function, you need to provide two inputs
def NNStructure():
initial_input= Embedding(vocabulary_size, 25, input_length=50, weights=
[embedding_matrix25],trainable=False)
lstm = LSTM(32)(initial_input)
secondary_input = Input(shape=(Number_of_row,27))
merge = concatenate([lstm, secondary_input])
first_dense = Dense(128, activation='relu')(merge)
second_dense=Dense(64, activation='relu')(first_dense)
output=Dense(1, activation='softmax')(second_dense)
model_glove1 = Model(inputs=[initial_input, secondary_input], outputs=output)
return model_glove1
model_glove1=NNStructure()
model_glove1.compile(loss='binary_crossentropy',optimizer='adam',metrics['accuracy',auc_roc])
model_glove1.fit(x=[data1,data2], y=np.array(train_y), batch_size=32,
epochs=4,
verbose=1,
validation_split=0.1,
shuffle=True)
I have features in shape of (size,2) and labels in shape of (size,1) i.e. for [x,y] in feature the label will be z. I want to build an LSTM in keras that can do such job since the feature is linked somehow with the previous inputs i.e. 1 or multiple(I believe its a hyperparameter).
Sample dataset values are:-
features labels
[1,2] [5]
[3,4] [84]
Here is what I have done so far:-
print(labels.shape) #prints (1414,2)
print(features.shape) #prints(1414,1)
look_back=2
# reshape input to be [samples, time steps, features]
features = np.reshape(features, (features.shape[0], 1, features.shape[1]))
labels = np.reshape(labels, (labels.shape[0], 1, 1))
X_train, X_test, y_train, y_test = train_test_split(features,labels,test_size=0.2)
model = Sequential()
model.add(LSTM(4, input_shape=(1, look_back))) #executing correctly
model.add(Dense(1)) #error here is "ValueError: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (1131, 1, 1)"
model.summary()
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, epochs=100, batch_size=1, verbose=2)
So can anyone please help me build a minimal LSTM example to run my code? Thank you. I don't know how can dense layer have 2 dimensions I mean it is an integer telling how many units to use in the dense layer.
You must not reshape your labels.
Try this:
features = np.reshape(features, (features.shape[0], 1, features.shape[1]))
model = Sequential()
model.add(LSTM(4, input_shape=(1, features.shape[1])))
model.add(Dense(1))
model.summary()
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, epochs=100, batch_size=1, verbose=2)