I have:
Multiple time series as INPUT
Forecast time series point in OUTPUT
How can be sure that model predict data by using dependencies between all time series in input?
Edit 1
My current model:
model = Sequential()
model.add(keras.layers.LSTM(hidden_nodes, input_dim=num_features, input_length=window, consume_less="mem"))
model.add(keras.layers.Dense(num_features, activation='sigmoid'))
optimizer = keras.optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9, nesterov=True)
By default LSTM layer in keras (and any other type of recurrent layer) is not stateful, and hence the states are reset every time a new input is fed into the network. Your code uses this default version. If you want, you can make it stateful by specifying stateful=True inside the LSTM layer, and then the states will not be reset. You can read more about the relevant syntax here, and this blog post provides more information regarding the stateful mode.
Here is an example of the corresponding syntax, taken from here:
trainX = numpy.reshape(trainX, (trainX.shape[0], trainX.shape[1], 1))
testX = numpy.reshape(testX, (testX.shape[0], testX.shape[1], 1))
# create and fit the LSTM network
batch_size = 1
model = Sequential()
model.add(LSTM(4, batch_input_shape=(batch_size, look_back, 1), stateful=True))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
for i in range(100):
model.fit(trainX, trainY, epochs=1, batch_size=batch_size, verbose=2, shuffle=False)
model.reset_states()
# make predictions
trainPredict = model.predict(trainX, batch_size=batch_size)
model.reset_states()
testPredict = model.predict(testX, batch_size=batch_size)
Related
I am having a time series prediction problem and building an LSTM like below :
def create_model():
model = Sequential()
model.add(LSTM(50,kernel_regularizer=l2(0.01), recurrent_regularizer=l2(0.01), bias_regularizer=l2(0.01), input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dropout(0.591))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
return model
When I train the model on 5 splits like below :
tss = TimeSeriesSplit(n_splits = 5)
X = data.drop(labels=['target_prediction'], axis=1)
y = data['target_prediction']
for train_index, test_index in tss.split(X):
train_X, test_X = X.iloc[train_index, :].values, X.iloc[test_index,:].values
train_y, test_y = y.iloc[train_index].values, y.iloc[test_index].values
model=create_model()
history = model.fit(train_X, train_y, epochs=10, batch_size=64,validation_data=(test_X, test_y), verbose=0, shuffle=False)
I get an overfitting problem. The graph of loss is attached
I am not sure why there is overfitting when I use regularizers in my Keras model. Any help is appreciated .
EDIT:
Tried the architectures
def create_model():
model = Sequential()
model.add(LSTM(20, input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
return model
def create_model(x,y):
# define LSTM
model = Sequential()
model.add(Bidirectional(LSTM(20, return_sequences=True), input_shape=(x,y)))
model.add(TimeDistributed(Dense(1, activation='sigmoid')))
model.compile(loss='mean_squared_error', optimizer='adam')
return model
but still it is overfitting.
First of all remove all your regularizers and dropout. You are literally spamming with all the tricks out there and 0.5 dropout is too high.
Reduce the number of units in your LSTM. Start from there. Reach a point where your model stops overfitting.
Then, add dropout if required.
After that, the next step is to add the tf.keras.Bidirectional. If still, you are not satfisfied then, increase number of layers. Remember to keep return_sequences True for every LSTM layer except the last one.
It is seldom I come across networks using layer regularization despite the availability because dropout and layer regularization have a same effect and people usually go with dropout (at maximum, I have seen 0.3 being used).
I'm working on a multivariate timeseries prediction using LSTM. I'm trying to get a better match between my actual and predicted values, but no matter what my hyperparamters are, the accuracy won't change. I was wondering if you can give me few insight on how to increase my model accuracy...
I have 3 inputs (time, two rates) and one output (pressure).
enter image description here
This is the LSTM section of my code:
model = Sequential()
model.add(LSTM(units=4,
activation='tanh',
recurrent_activation='hard_sigmoid',
use_bias=True,
unit_forget_bias=True,
dropout=0,
recurrent_dropout=0.3,
input_shape=(look_back, 3)))
model.add(Dense(units=1,
activation='linear',
use_bias=True))
model.compile(loss='mean_squared_error', optimizer='Adam', metrics=['mae','accuracy'])
hist = model.fit(x_train, y_train,
epochs=50,
batch_size=20,
validation_split=0.0,
verbose=2,
shuffle=False)
1.
model = Model(inPut, outputs=outPut)
model.compile(loss="mse", optimizer="adam")
for i in range(10):
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
2.
model = Model(inPut, outputs=outPut)
for i in range(10):
model.compile(loss="mse", optimizer="adam")
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
3.
for i in range(10):
model = Model(inPut, outputs=outPut)
model.compile(loss="mse", optimizer="adam")
model.fit(dataX, dataY, epochs=EPOCH, batch_size=BATCHSIZE, verbose=0, shuffle=False)
#save model
I studied neural network in keras.
i want that each iteration is each learning.
but second(third.. fourth..) iteration learning(model.fit) is going continually fisrt learning(model.fit)
loss is going continually in three code.
could you tell me some method that each iteration is each learning?
each iteration model is another model.
If it is your intent to create differently trained models but maintain the same architecture, then your code snippet for #3 is the one you're after. You create a new model using the same architecture, configure the neural network then train it. After each iteration, make sure you save the model.
I am creating a LSTM model in Keras with Python and I have the next context:
I have a regression problem, and I want a LSTM model of X different layers that inputs a sequence of 40 elements and outputs the next Y values. Those Y values should be dynamic. This is: maybe I want to predict the next 10 elements for a sequence of 40 elements in one case, but in other case I want to predict the next 100 values.
My question is: is it possible?
I have the next code:
model = Sequential()
model.add(LSTM(200, activation='relu', input_shape=(trainx.shape[1], trainx.shape[2])))
model.add(RepeatVector(outputs))
model.add(LSTM(200, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(100, activation='relu')))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mse', optimizer='adam')
# fit network
history = model.fit(trainx, trainy, epochs=100, batch_size=300, verbose=1)
# make predictions
trainPredict = model.predict(trainx)
After two epochs the validation accuracy of my model shows .30, but when I return the predicted classes using model.predict_generator, and measure the accuracy myself - the accuracy is much lower at about .18.
Why are these methods returning different accuracies? I believe it may be related to my implementation or understanding of timeseriesgenerator.
data_gen_train = sequence.TimeseriesGenerator(X, y_ct, timesteps, sampling_rate=1, stride=1, start_index=0, end_index=len(y), batch_size=batch_size)
data_gen_test = sequence.TimeseriesGenerator(X_ho, y_ho_ct, timesteps, sampling_rate=1, stride=1, start_index=0, end_index=len(y), batch_size=batch_size)
model = Sequential()
model.add(LSTM(20, stateful=True, batch_input_shape=(batch_size, timesteps, data_dim)))
model.add(Dense(9, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer= 'Nadam', metrics=['accuracy'])
model.fit_generator(data_gen_train, validation_data=data_gen_test, epochs=epochs, shuffle=False, validation_steps= len(y_ho) //batch_size)
y_pred = model.predict_generator(data_gen_test, steps= len(y_ho)//batch_size)
enter image description here
I found the answer to my question above. The difference in accuracy measurements was due to the pred and ground truth dataset being shifted relative to each other. Keras' pad_sequences should have been used here to avoid this issue.