I have features in shape of (size,2) and labels in shape of (size,1) i.e. for [x,y] in feature the label will be z. I want to build an LSTM in keras that can do such job since the feature is linked somehow with the previous inputs i.e. 1 or multiple(I believe its a hyperparameter).
Sample dataset values are:-
features labels
[1,2] [5]
[3,4] [84]
Here is what I have done so far:-
print(labels.shape) #prints (1414,2)
print(features.shape) #prints(1414,1)
look_back=2
# reshape input to be [samples, time steps, features]
features = np.reshape(features, (features.shape[0], 1, features.shape[1]))
labels = np.reshape(labels, (labels.shape[0], 1, 1))
X_train, X_test, y_train, y_test = train_test_split(features,labels,test_size=0.2)
model = Sequential()
model.add(LSTM(4, input_shape=(1, look_back))) #executing correctly
model.add(Dense(1)) #error here is "ValueError: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (1131, 1, 1)"
model.summary()
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, epochs=100, batch_size=1, verbose=2)
So can anyone please help me build a minimal LSTM example to run my code? Thank you. I don't know how can dense layer have 2 dimensions I mean it is an integer telling how many units to use in the dense layer.
You must not reshape your labels.
Try this:
features = np.reshape(features, (features.shape[0], 1, features.shape[1]))
model = Sequential()
model.add(LSTM(4, input_shape=(1, features.shape[1])))
model.add(Dense(1))
model.summary()
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, epochs=100, batch_size=1, verbose=2)
Related
I'm trying to fit a LSTM classifier using Keras but don't understand how to prepare the data for training.
I currently have two dataframes for the training data. X_train contains 48 hand-crafted temporal features from IMU data, and y_train contains corresponding labels (4 kinds) representing terrain. The shape of these dataframes is given below:
X_train = X_train.values.reshape(X_train.shape[0],X_train.shape[1],1)
print(X_train.shape, y_train.shape)
**(268320, 48, 1) (268320,)**
Model using batch_size = (32,5,48):
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 5, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
LSTM_history = model_LSTM.fit(X_train, y_train, epochs=15, validation_data=(X_valid, y_valid), verbose=1)
The error that I am getting:
ValueError: Shapes (32, 1) and (32, 48, 4) are incompatible
Any insight into how to fix this particular error and any intuition into what Keras is expecting?
What is the 5 in your batch size ? The batch_size argument in the LSTM layer indicates that your data should be in the form (batch_size, time_steps, feature_per_time_step). If I am understanding correctly, your data has time_steps = 1 and feature_per_time_step = 48.
Here is a sample of working code and the shape of each of them.
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 1, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
X_train = np.random.random((10000,1,48))
y_train = np.random.random((10000,4))
y_train = y_train.reshape(-1,1,4)
data = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(32)
model_LSTM.fit(data, epochs=15, verbose=1)
Passing data instead of x_train and y_train in your fit function will fit the model properly.
If you want to have 5 timesteps in your data, you will have to create your X_train in such a way to have it have a shape (n_samples,5,48).
I am trying to train a LSTM Autoencoder with multivariate time series data. The shape of data is:
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
(160573, 4, 4)
(160573, 4)
(17838, 4, 4)
(17838, 4)
I wanted to transfer my model for univariate time series into a model for multivariate time series but somehow I do not know what to change:
model = keras.Sequential()
model.add(keras.layers.LSTM(
units=64,
input_shape=(X_train.shape[1], X_train.shape[2])
))
model.add(keras.layers.Dropout(rate=0.2))
model.add(keras.layers.RepeatVector(n=X_train.shape[1]))
model.add(keras.layers.LSTM(units=64, return_sequences=True))
model.add(keras.layers.Dropout(rate=0.2))
model.add(
keras.layers.TimeDistributed(
keras.layers.Dense(units=X_train.shape[2])
)
)
model.compile(loss='mse', optimizer='adam')
training = model.fit(
X_train, y_train,
epochs=10,
batch_size=64,
validation_split=0.1,
shuffle=False
)
The error is:
InvalidArgumentError: Incompatible shapes: [64,4,4] vs. [64,4]
[[node gradient_tape/mean_squared_error/BroadcastGradientArgs (defined at <ipython-input-83-6205cceab3d0>:1) ]] [Op:__inference_train_function_96522]
Function call stack:
train_function
Is there any solution for multivariate time series?
Thank you and best,
Can
Assuming you're not trying to predict a sequence, you need to remove return_sequences=True to remove the time step. Same goes for the TimeDistributed layer, remove it.
Corrected, minimal example:
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
from tensorflow import keras
import numpy as np
X_train = np.random.rand(160, 4, 4)
y_train = np.random.rand(160, 4)
X_test = np.random.rand(17, 4, 4)
y_test = np.random.rand(17, 4)
model = keras.Sequential()
model.add(keras.layers.LSTM(
units=4,
input_shape=(X_train.shape[1], X_train.shape[2])
))
model.add(keras.layers.Dropout(rate=0.2))
model.add(keras.layers.RepeatVector(n=X_train.shape[1]))
model.add(keras.layers.LSTM(units=4))
model.add(keras.layers.Dropout(rate=0.2))
model.add(keras.layers.Dense(units=X_train.shape[2]))
model.compile(loss='mse', optimizer='adam')
training = model.fit(
X_train, y_train,
epochs=1,
batch_size=8,
validation_split=0.1,
shuffle=False)
I want to make prediction with Keras Neural Network. My output data has 3 different values -1,0,1.
When I run my NN I get the error:
ValueError: Error when checking target: expected dense_35 to have shape (3,) but got array with shape (1,)
Then I tried to do:
from tensorflow.python.keras.utils import to_categorical
results = to_categorical(results)
But again I get the same error:
ValueError: Error when checking target: expected dense_35 to have shape (3,) but got array with shape (2,)
What am I doing wrong?
This is my code:
features = df.iloc[:,-8:]
results = df.iloc[:,-9]
x_train, x_test, y_train, y_test = train_test_split(features, results, test_size=0.3, random_state=42)
model = Sequential()
model.add(Dense(64, input_dim = x_train.shape[1], activation = 'relu')) # input layer requires input_dim param
model.add(Dense(32, activation = 'relu'))
model.add(Dense(16, activation = 'relu'))
model.add(Dense(3, activation = 'softmax'))
model.compile(loss="categorical_crossentropy", optimizer= "adam", metrics=['accuracy'])
# call the function to fit to the data training the network)
es = EarlyStopping(monitor='val_loss', min_delta=0.001, patience=0, verbose=1, mode='auto')
model.fit(x_train, y_train, epochs = 10, shuffle = True, batch_size=128, validation_data=(x_test, y_test), verbose=2, callbacks=[es])
results = df.iloc[:,-9] you're choosing 1-d output (shape: (rows,1)), but your last layer has 3 units model.add(Dense(3, activation = 'softmax')).
So, your result must have shape: (rows, 3) not (rows, 1).
I see your result has values -1, 0, 1. Just add one so that they are 0, 1, 2. That's why you're getting error with to_categorical; according to the docs, it expects
y: class vector to be converted into a matrix (integers from 0 to num_classes).
So go for
results = results + 1
Then, apply to_categorical.
After that fit should work fine.
I'm making a dense ML model with Keras
but I get this error
ValueError: Error when checking target: expected dense_3 to have shape (1,) but got array with shape (9,)
This is how my model is set up
get_custom_objects().update({'swish': Swish(swish)})
model = Sequential()
model.add(Dense(33, activation='swish', input_shape=(trainX.shape[1],)))
model.add(Dense(33, activation='swish'))
model.add(Dense(9, activation='softmax'))
#Train Network
model.compile(optimizer="adam", loss=keras.losses.sparse_categorical_crossentropy, metrics=["accuracy"])
model.fit(trainX, trainY, validation_split=0.2, epochs=3)
trainX and trainY are pandas DataFrames
trainX has 2 columns and trainY has 9.
I'm not sure why it says it should be (1,) since I specified the output layer to have 9 neurons.
Any help is greatly appreciated.
From Keras docs:
When using the sparse_categorical_crossentropy loss, your targets should be integer targets. If you have categorical targets, you should use categorical_crossentropy.
So you should replace
model.compile(optimizer="adam", loss=keras.losses.sparse_categorical_crossentropy, metrics=["accuracy"])
With
model.compile(optimizer="adam", loss=keras.losses.categorical_crossentropy, metrics=["accuracy"])
That is needed because trainY is a categorical target, since it has 9 columns instead of just one integer.
I cannot find a hands on tutorial on how to structure the data for use with keras LSTM.
Data
x_train = 7300 rows where each vector is length 64.
y_train = array of 7300 items either 0's or 1's (the class).
Model
model = Sequential()
model.add(LSTM(200, dropout=0.2, recurrent_dropout=0.2, input_shape = (1, 64)))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train,
epochs = 5,
batch_size = 32,
validation_split = 0.1,
callbacks=[EarlyStopping(monitor='val_loss', patience=3, min_delta=0.0001)])
My question is simply, why doesn't this work? Why isn't is as simple as giving an 2d array of vectors and similar length y values to fit.
Keras LSTM expects input of shape [batch_size, timesteps, features]. Your data is of shape [batch_size, features].
To add the timestep dimension (where number of timesteps is 1), do the following:
x_train = np.expand_dims(x_train, axis=1)