I am trying to fit an lstm model to my data read as csv file. (320,6) is the shape of x_train, and the model is given as
def build_modelLSTMlite(input_shape):
model = keras.Sequential()
model.add(keras.layers.LSTM(64, input_shape=input_shape))
model.add(keras.layers.Dense(64, activation='relu'))
model.add(keras.layers.Dropout(0.3))
model.add(keras.layers.Dense(10, activation='softmax'))
return model
model = build_modelLSTMlite(input_shape)
optimiser = keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=optimiser,
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.summary()
history = model.fit(x_train, y_train, batch_size=32, epochs=100)
This model.fit() is showing value error
ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 320, 6), found shape=(32, 6)
You have to create a sliding version of your x_train data. Something like:
from numpy.lib.stride_tricks import sliding_window_view
x_train_lstm = sliding_window_view(x_train, (input_shape[0], x_train.shape[1])).squeeze(axis=1)
history = model.fit(X_train_lstm, y_train[:-input_shape[0]+1], batch_size=32, epochs=100)
Related
I am trying to implement a Bidirectional LSTM for a sequence-to-sequence model. I have already one-hot-encoded my sequences with 12 total features. The input is 11 steps while the output is 23 steps. First, I coded this LSTM implementation that works with the first LSTM as the encoder and the second as the decoder.
model = Sequential()
model.add(LSTM(75, input_shape=(11, 12)))
model.add(RepeatVector(23))
model.add(LSTM(50, return_sequences=True))
model.add(TimeDistributed(Dense(12, activation='softmax')))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
X, y = generate_data(1, taskset, trainset)
model.fit(X, y, epochs=1, batch_size=32, verbose=1)
I then tried to turn this into a bidirectional LSTM as follows:
model = Sequential()
model.add(Bidirectional(LSTM(75, return_sequences=True), input_shape=(11,12), merge_mode='concat'))
model.add(Bidirectional(LSTM(50, return_sequences=True)))
model.add(TimeDistributed(Dense(12, activation='softmax')))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics= ['accuracy'])
model.summary()
X, y = generate_data(1, taskset, trainset)
model.fit(X, y, epochs=1, batch_size=32, verbose=1)
The goal is to use the first bidirectional LSTM as the encoder and the second bidirectional LSTM as the decoder. I removed the RepeatVector in the bidirectional implementation because it gave me a dimension error (needed dim=2, received dim=3). With the current bidirectional LSTM I am getting this error:
ValueError: Shapes (None, 23, 12) and (None, 11, 12) are incompatible
Any help with fixing the bidirectional LSTM implementation?
Simply setting return_sequences=False in your first bidirectional LSTM and adding as before RepeatVector(23) works fine
n_sample = 10
X = np.random.uniform(0,1, (n_sample, 11, 12))
y = np.random.randint(0,2, (n_sample, 23, 12))
model = Sequential()
model.add(Bidirectional(LSTM(75), input_shape=(11,12), merge_mode='concat'))
model.add(RepeatVector(23))
model.add(Bidirectional(LSTM(50, return_sequences=True)))
model.add(TimeDistributed(Dense(12, activation='softmax')))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics= ['accuracy'])
model.fit(X, y, epochs=3, batch_size=32, verbose=1)
I'm trying to fit a LSTM classifier using Keras but don't understand how to prepare the data for training.
I currently have two dataframes for the training data. X_train contains 48 hand-crafted temporal features from IMU data, and y_train contains corresponding labels (4 kinds) representing terrain. The shape of these dataframes is given below:
X_train = X_train.values.reshape(X_train.shape[0],X_train.shape[1],1)
print(X_train.shape, y_train.shape)
**(268320, 48, 1) (268320,)**
Model using batch_size = (32,5,48):
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 5, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
LSTM_history = model_LSTM.fit(X_train, y_train, epochs=15, validation_data=(X_valid, y_valid), verbose=1)
The error that I am getting:
ValueError: Shapes (32, 1) and (32, 48, 4) are incompatible
Any insight into how to fix this particular error and any intuition into what Keras is expecting?
What is the 5 in your batch size ? The batch_size argument in the LSTM layer indicates that your data should be in the form (batch_size, time_steps, feature_per_time_step). If I am understanding correctly, your data has time_steps = 1 and feature_per_time_step = 48.
Here is a sample of working code and the shape of each of them.
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 1, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
X_train = np.random.random((10000,1,48))
y_train = np.random.random((10000,4))
y_train = y_train.reshape(-1,1,4)
data = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(32)
model_LSTM.fit(data, epochs=15, verbose=1)
Passing data instead of x_train and y_train in your fit function will fit the model properly.
If you want to have 5 timesteps in your data, you will have to create your X_train in such a way to have it have a shape (n_samples,5,48).
I am learning Keras using audio classification, Actually, I am implementing the code with modification from https://github.com/deepsound-project/genre-recognition/blob/master/train_model.py using Keras.
The shape of the dataset is
X_train shape = (800, 32, 1)
y_train shape = (800, 10)
X_test shape = (200, 32, 1)
y_test shape = (200, 10)
The model
model = Sequential()
model.add(Conv1D(filters=256, kernel_size=5, input_shape=(32,1), activation="relu"))
model.add(BatchNormalization(momentum=0.9))
model.add(MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(Conv1D(filters=256, kernel_size=5, activation="relu"))
model.add(BatchNormalization(momentum=0.9))
model.add(MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation="relu", ))
model.add(Dense(10, activation='softmax'))
model.compile(
loss='categorical_crossentropy',
optimizer = Adam(lr=0.001),
metrics = ['accuracy'],
)
model.summary()
red_lr= ReduceLROnPlateau(monitor='val_loss',patience=2,verbose=2,factor=0.5,min_delta=0.01)
check=ModelCheckpoint(filepath=r'/content/drive/My Drive/Colab Notebooks/gen/cnn.hdf5', verbose=1, save_best_only = True)
History = model.fit(X_train,
y_train,
epochs=100,
#batch_size=512,
validation_data = (X_test, y_test),
verbose = 2,
callbacks=[check, red_lr],
shuffle=True )
The accuracy graph
Loss graph
I do not understand, Why the val_acc is in the range of 70%. I tried to modify the model architecture including optimizer, but no improvement.
And, Is it good to have a lot of difference between loss and val_loss.
how to improve the accuracy above 80... any help...
Thank you
I found it, I use concatenate function from Keras to concatenate all convolution layers and, it gives the best performance.
How do you deal with this error?
Error when checking target: expected dense_3 to have shape (1,) but got array with shape (398,)
I Tried changing the input_shape=(14,) which is the amount of columns in the train_samples, but i still get the error.
set = pd.read_csv('NHL_DATA.csv')
set.head()
train_labels = [set['Won/Lost']]
train_samples = [set['team'], set['blocked'],set['faceOffWinPercentage'],set['giveaways'],set['goals'],set['hits'],
set['pim'], set['powerPlayGoals'], set['powerPlayOpportunities'], set['powerPlayPercentage'],
set['shots'], set['takeaways'], set['homeaway_away'],set['homeaway_home']]
train_labels = np.array(train_labels)
train_samples = np.array(train_samples)
scaler = MinMaxScaler(feature_range=(0,1))
scaled_train_samples = scaler.fit_transform(train_samples).reshape(-1,1)
model = Sequential()
model.add(Dense(16, input_shape=(14,), activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(Adam(lr=.0001), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(scaled_train_samples, train_labels, batch_size=1, epochs=20, shuffle=True, verbose=2)
1) You reshape your training example with .reshape(-1,1) which means all training samples have 1 dimension. However, you define the input shape of the network as input_shape=(14,) that tells the input dimension is 14. I guess this is one problem with your model.
2) You used sparse_categorical_crossentropy which means the ground truth labels are sparse (train_labels should be sparse) but I guess it is not.
Here is an example of how your input should be:
import numpy as np
from tensorflow.python.keras.engine.sequential import Sequential
from tensorflow.python.keras.layers import Dense
x = np.zeros([1000, 14])
y = np.zeros([1000, 2])
model = Sequential()
model.add(Dense(16, input_shape=(14,), activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile('adam', 'categorical_crossentropy')
model.fit(x, y, batch_size=1, epochs=1)
My data is a matrix of 400k observations, where each observation is a single row of data, 200 elements in length. My output data is 400k 1-hot vectors corresponding to 11 output classes. I would like to format my data for a keras sequential model. (Keras 2.0.4, Python 3)
train_values.shape
>>> (400000, 200)
data_labels.shape
>>> (400000, 11)
Attempt 1:
model = Sequential()
model.add(LSTM(100, input_shape = (200,), return_sequences=True))
model.add(Dense(output_dim=11, input_dim=100, activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer='rmsprop',metrics=['accuracy'])
model.fit(train_values, train_labels, batch_size=200, epochs=10)
score = model.evaluate(test_values, test_labels, batch_size=200)
Error 1:
----> 3 model.add(LSTM(100, input_shape = (200,), return_sequences=True))
ValueError: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=2
Attempt 2:
model = Sequential()
model.add(Embedding(400000,200))
model.add(LSTM(100, input_shape = (200,), return_sequences=True))
model.add(Dense(output_dim=11, input_dim=100, activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer='rmsprop',metrics=['accuracy'])
model.fit(train_values, train_labels, batch_size=200, epochs=10)
score = model.evaluate(test_values, test_labels, batch_size=200)
Error 2:
----> 7 model.fit(train_values, train_labels, batch_size=200, epochs=10)
ValueError: Error when checking target: expected dense_1 to have 3 dimensions, but got array with shape (400000, 11)
Conclusion: Is my error with the input formatting, the calling of my layers, or something else entirely that I'm unaware of?