I would appreciate your help with creating an LSTM model in Keras. My input is a 2D numpy array with rows that are numeric time-series of equal length.
I've written the following lines to create an LSTM model that operates on the raw data without an embedding layer:
lstm_clf = Sequential()
lstm_clf.add(LSTM(100, input_shape=(X_train[0], X_train[1]))
lstm_clf.add(Dense(7, activation='softmax'))
lstm_clf.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])
When I reach the fitting phase, I get the following error: "ValueError: Error when checking input: expected lstm_11_input to have 3 dimensions, but got array with shape (100, 289)".
Could anyone please explain to me what I'm doing wrong and how to fix the code. It must be related to the input shape, but I've no idea what it is.
Thanks a lot for your help,
Alexander.
I was able to find the answer. Here's the correct code:
lstm_clf = Sequential()
lstm_clf.add(LSTM(100, input_shape=(1, max_seq_len)))
lstm_clf.add(Dense(7, activation='softmax'))
lstm_clf.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])
My data also had to be reshaped, as following:
X_seqs_train = reshape(X_seqs_train, (X_seqs_train.shape[0], 1, X_seqs_train.shape[1]))
X_seqs_test = reshape(X_seqs_test, (X_seqs_test.shape[0], 1, X_seqs_test.shape1))
These changes solved the problem. A clue that led to the answer was found here.
Related
I am trying to build a Tensorflow simple AI and I don't understand where my problem is when I compare it to various articles I read about this topic.
I have two numpy arrays made for training of sizes (N,1) (what should be predicted) and (N,3) (what are the inputs that should be predicted), and I want my algorithm to predict with Input[0,i] Input[1,i] Input[2,i] the value of ToPredict[i] (the i-th sample).
I thought I had to use tf.data.Dataset.from_tensor_slices and then
modele = tf.keras.Sequential([ tf.keras.layers.Flatten(), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(1) ])
modele.compile(optimizer=tf.keras.optimizers.RMSprop(), loss=tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['sparse_categorical_accuracy'])
modele.fit(A, epochs=10)
from an example I got online. (here A represents the tf.data.Dataset.from_tensor_slices(Inputs_Learning,ToPredict_Learning))
When I do that, I get the following error:
logits and labels must have the same first dimension, got logits shape [3,10] and labels shape [1]
[[node loss_9/output_1_loss/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at :11) ]] [Op:__inference_keras_scratch_graph_2640]
So I know that there is a shape error, but I don't know how to tell the algorithm what I want as shapes.
Could anyone help me? Also yes I have seen related stackoverflow posts but they don't seem to solve my issue.
Add batch dimension to your data:
A = A.batch(10)
When I Run this code:
model = Sequential([
LSTM(300, activation='tanh', input_shape=data.shape, recurrent_activation='sigmoid'),
Dropout(0.2),
Dense(4, activation='linear')
])
model.compile(optimizer='adam', loss='mse')
history = model.fit_generator(training, epochs=22, steps_per_epoch=1)
This error pops up:
ValueError: cannot copy sequence with size 7 to array axis with dimension 5
I don't know how to handle that. Yesterday, the code worked perfectly. In other threads I've found; the error might be related to np.vstack. This is the thread I'm writing about.
I've tried to change the data to a np.array but it did not work. Can anyone help?
This might be useful:
training = TimeseriesGenerator(data, data, length=7, batch_size=6)
TimeseriesGenerator is from keras.preprocessing.sequence
You need input numpy arrays into Keras' TimeSeries generator, not directly DataFrames, you can do data.values for pass numpy only.
Trying to set up a Conv1D layer to be the input layer in keras.
The dataset is 1000 timesteps, and each timestep has 1 feature.
After reading a bunch of answers I reshaped my dataset to be in the following format of (n_samples, timesteps, features), which corresponds to the following in my case:
train_data = (78968, 1000, 1)
test_data = (19742, 1000, 1)
train_target = (78968,)
test_target = (19742,)
I later create and compile the code using the following lines
model = Sequential()
model.add(Conv1D(64, (4), input_shape = (1000,1) ))
model.add(MaxPooling1D(pool_size=2))
model.add(Dense(1))
optimizer = opt = Adam(decay = 1.000-0.999)
model.compile(optimizer=optimizer,
loss='mean_squared_error',
metrics=['mean_absolute_error','mean_squared_error'])
Then I try to fit, note, train_target and test_target are pandas series so i'm calling DataFrame.values to convert to numpy array, i suspect there might be an issue there?
training = model.fit(train_data,
train_target.values,
validation_data=(test_data, test_target.values),
epochs=epochs,
verbose=1)
The model compiles but I get an error when I try to fit
Error when checking target: expected dense_4 to have 3 dimensions,
but got array with shape (78968, 1)
I've tried every combination of reshaping the data and can't get this to work.
I've used keras with dense layers only before for a different project where the input_dimension was specificied instead of the input_shape, so I'm not sure what I'm doing wrong here. I've read almost every stack overflow question about data shape issues and I'm afraid the problem is elsewhere, any help is appreciated, thank you.
Under the line model.add(MaxPooling1D(pool_size=2)), add one line model.add(Flatten()), your problem will be solved. Flatten function will help you convert your data into correct shape, please see this site for more information https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten
I'm trying to assign one of two classes (positive/nagative) to audio using CNN with Keras. My model should accept varied lengths of input (frames) in which each frame contains 41 features but I struggle with the input size. Bear in mind that I haven't acquired full dataset so I just mocked some meaningless data just to check if network works at all.
According to documentation https://keras.io/layers/convolutional/ and my best understanding Conv1D can tackle varied lengths if first element of input_shape tuple is None. Shape of variable containing input data X_train.shape is (4, 497, 41).
data = pd.read_csv('output_file.csv', sep=';')
featureCount = data.values.shape[1]
#mocks because full data is not available yet
Y_train = np.asarray([1, 0, 1, 0])
X_train = np.asarray(
[np.array(data.values, copy=True), np.array(data.values, copy=True), np.array(data.values, copy=True),
np.array(data.values, copy=True)])
# variable length with 41 features
model = keras.models.Sequential()
model.add(keras.layers.Conv1D(100, 5, activation='relu', input_shape=(None, featureCount)))
model.add(keras.layers.GlobalMaxPooling1D())
model.add(keras.layers.Dense(10, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
model.summary()
model.fit(X_train, Y_train, epochs=10, verbose=False, validation_data=(np.array(data.values, copy=True), [1]))
This code produces error
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (497, 41). So it appears like the first dimension was cut out as it contains training samples (it seems correct to me) what bothered me is the required dimensionality, why is it 3?
After searching for the answer I stumbled onto Dimension of shape in conv1D and followed it by adding last dimension (using X_train = np.expand_dims(X_train, axis=3)) that contains only single digit but I ended up with another, similar error:
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (4, 497, 41, 1) now it seems that first dimension that previously was treated as sample "list" is now part of actual data.
I also tried fiddling with input_shape parameter but to no avail and using Reshape layer but ended up fighting with size the
What should I do to satisfy required shape? How to prepare data for processing?
I am trying to use an LSTM model for binary classification on multivariate time series data. I have seven properties collected over the course of the day for about 100 days (i.e. 100 arrays of size [9000, 7]). Each of these arrays has a single classification status of either 1 or 0.
I've started out trying to build the simplest model possible given that I am new to Keras and Machine Learning in general, but I keep getting errors regarding input shape when I try to train them. For example, my first layers:
model = Sequential()
model.add(Conv2D(32, (3,3), input_shape=(9000,7,1), activation='relu'))
...
model.fit(x=X_train, y=Y_train, epochs=100)
With an X_train of type float64, and a size of (100L, 9000L, 7L), I get an error that reads:
ValueError: Error when checking input: expected conv2d_11_input to have 4 dimensions, but got array with shape (100L, 9000L, 7L)
I've tried changing the batch size and number of epochs with no sucess so can someone explain how to correctly reshape my input? Am I missing something simple?
I suspect you want to use a Conv1D (3D data), no?
You're using a Conv2D (4D data = images).
For either Conv1D and any of the RNN layers, such as LSTM, your input is 3D data and your input_shape should be input_shape=(9000,7).
The input data should be an array with shape (100,9000,7), which is already ok, by the content of the error message.
Assuming each day is an individual sequence and you don't want to connect days.