I know this question is asked many times, but I truly can't fix this input shape issue for my case.
My x_train shape == (5523000, 13) // (13 timeseries of length 5523000)
My y_train shape == (5523000, 1)
number of classes == 2
To reshape the x_train and y_train:
x_train= x_train.values.reshape(27615,200,13) # 5523000/200 = 27615
y_train= y_train.values.reshape((5523000,1)) # I know I have a problem here but I dont know how to fix it
Here is my lstm network :
def lstm_baseline(x_train, y_train):
batch_size=200
model = Sequential()
model.add(LSTM(batch_size, input_shape=(27615,200,13),
activation='relu', return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='softmax'))
model.compile(
loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train,y_train, epochs= 15)
return model
Whenever I run the code I get this error :
ValueError: Input 0 is incompatible with layer lstm_10: expected
ndim=3, found ndim=4
My question is what I am missing here?
PS: The idea of the project is that I have 13 signals coming from the 13 points of the human body, I want to use them to detect a certain type of diseases (an arousal). By using the LSTM, I want my model to locate the regions where I have that arousal based on these 13 signals.
.
The whole data is 993 patients, for each one I use 13 signals to detect the disorder regions.
if you want me to put the data in 3D dimensions:
(500000 ,13, 993) # (nb_recods, nb_signals, nb_patient)
for each patient I have 500000 observations of 13 signals.
nb_patient is 993
It worth noting that the 500000 size doesn't matter ! as i can have patients with more observations or less than that.
Update: here is a sample data of one patient.
Here is a chunk of my data first 2000 rows
Ok, I did some changes to your code. First, I still don't now what the "200" in your attempt to reshape your data means, so I'm gonna give you a working code and let's see if you can use it or you can modify it to make your code work. The size of your input data and your targets, have to match. You can not have an input x_train with 27615 rows (which is the meaning of x_train[0] = 27615) and a target set y_train with 5523000 values.
I took the first two rows from the data example that you provided for this example:
x_sample = [[-17, -7, -7, 0, -5, -18, 73, 9, -282, 28550, 67],
[-21, -16, -7, -6, -8, 15, 60, 6, -239, 28550, 94]]
y_sample = [0, 0]
Let's reshape x_sample:
x_train = np.array(example)
#Here x_train.shape = (2,11), we want to reshape it to (2,11,1) to
#fit the network's input dimension
x_train = x_train.reshape(x_train.shape[0], x_train.shape[1], 1)
You are using a categorical loss, so you have to change your targets to categorical (chek https://keras.io/utils/)
y_train = np.array(target)
y_train = to_categorical(y_train, 2)
Now you have two categories, I assumed two categories as in the data that you provided all the targets values are 0, so I don't know how many possible values your target can take. If your target can take 4 possible values, then the number of categories in the to_categorical function will be 4. Every output of your last dense layer will represent a category and the value of that output, the probability of your input to belong to that category.
Now, we just have to slightly modify your LSTM model:
def lstm_baseline(x_train, y_train):
batch_size = 200
model = Sequential()
#We are gonna change input shape for input_dim
model.add(LSTM(batch_size, input_dim=1,
activation='relu', return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
#We are gonna set the number of outputs to 2, to match with the
#number of categories
model.add(Dense(2, activation='softmax'))
model.compile(
loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=15)
return model
You may try some modifications like this below:
x_train = x_train.reshape(1999, 1, 13)
# double-check dimensions
x_train.shape
def lstm_baseline(x_train, y_train, batch_size):
model = Sequential()
model.add(LSTM(batch_size, input_shape=(None, 13),
activation='relu', return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='softmax'))
model.compile(
loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
return model
Related
I am working on a project where I have to classify genomic sequences as either positive or negative. So basically, I have sequences that are in the form 'accccttttttgggg...'
These sequences are 150 characters long consisting of a, c, t and g characters. I perform one hot encoding of the data and create a dataframe that contains 600 columns (150 x 4) for sequences plus one column for label (either 0 or 1). I then pass this data through CNN. The model performs well on the training and validation data but when I see the predictions on test data it always predicts a single label 0. Can anyone help me know why does this happen and what I am doing wrong here. This is the model that I have been using with epochs=50, learning_rate=0.00001 and batch_size=64
model = Sequential()
model.add(Conv1D(32, kernel_size=5, input_shape=(600, 1), activation='relu', padding='same'))
model.add(MaxPooling1D())
model.add(Conv1D(16, kernel_size=5, activation='relu', padding='same'))
model.add(MaxPooling1D())
model.add(Conv1D(8, kernel_size=5, activation='relu', padding='same'))
model.add(MaxPooling1D())
model.add(Flatten())
model.add(Dense(4, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
adam = Adam(learning_rate=learning_rate)
model.compile(loss='binary_crossentropy', optimizer=adam, metrics=['accuracy'])
I'm trying to fit a LSTM classifier using Keras but don't understand how to prepare the data for training.
I currently have two dataframes for the training data. X_train contains 48 hand-crafted temporal features from IMU data, and y_train contains corresponding labels (4 kinds) representing terrain. The shape of these dataframes is given below:
X_train = X_train.values.reshape(X_train.shape[0],X_train.shape[1],1)
print(X_train.shape, y_train.shape)
**(268320, 48, 1) (268320,)**
Model using batch_size = (32,5,48):
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 5, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
LSTM_history = model_LSTM.fit(X_train, y_train, epochs=15, validation_data=(X_valid, y_valid), verbose=1)
The error that I am getting:
ValueError: Shapes (32, 1) and (32, 48, 4) are incompatible
Any insight into how to fix this particular error and any intuition into what Keras is expecting?
What is the 5 in your batch size ? The batch_size argument in the LSTM layer indicates that your data should be in the form (batch_size, time_steps, feature_per_time_step). If I am understanding correctly, your data has time_steps = 1 and feature_per_time_step = 48.
Here is a sample of working code and the shape of each of them.
def def_model():
model = Sequential()
model.add(LSTM(units=144,batch_size=(32, 1, 48),return_sequences=True))
model.add(Dropout(0.5))
model.add(Dense(144, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
model_LSTM = def_model()
X_train = np.random.random((10000,1,48))
y_train = np.random.random((10000,4))
y_train = y_train.reshape(-1,1,4)
data = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(32)
model_LSTM.fit(data, epochs=15, verbose=1)
Passing data instead of x_train and y_train in your fit function will fit the model properly.
If you want to have 5 timesteps in your data, you will have to create your X_train in such a way to have it have a shape (n_samples,5,48).
I'm quite new to CNN.
I'm trying to create a the following model. but I get the following error: "ValueError: logits and labels must have the same shape ((1, 7, 7, 2) vs (1, 2))"
Below the code I'm trying to implement
#create the training data set
train_data=scaled_data[0:training_data_len,:]
#define the number of periods
n_periods=28
#split the data into x_train and y_train data set
x_train=[]
y_train=[]
for i in range(n_periods,len(train_data)):
x_train.append(train_data[i-n_periods:i,:28])
y_train.append(train_data[i,29])
x_train=np.array(x_train)
y_train=np.array(y_train)
#Reshape the train data
x_train=x_train.reshape(x_train.shape[0],x_train.shape[1],x_train.shape[2],1)
x_train.shape
y_train = keras.utils.to_categorical(y_train,2)
# x_train as the folllowing shape (3561, 28, 28, 1)
# y_train as the following shape (3561, 2, 2)
#Build the 2 D CNN model for regression
model= Sequential()
model.add(Conv2D(32,kernel_size=(3,3),padding='same',activation='relu',input_shape=(x_train.shape[1],x_train.shape[2],1)))
model.add(Conv2D(64,kernel_size=(3,3),padding='same',activation='relu'))
model.add(MaxPooling2D(pool_size=(4,4)))
model.add(Dropout(0.25))
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='sigmoid'))
model.add(Dense(2, activation='sigmoid'))
model.summary()
#compile the model
model.compile(optimizer='ADADELTA', loss='binary_crossentropy', metrics=['accuracy'])
#train the model
model.fit(x_train, y_train, batch_size=1, epochs=1, verbose=2)
There are two problems in your approach:
You're using Convolutional/MaxPooling layers in which the inputs/outputs are as matrices, i.e., with the shape of (Batch_Size, Height, Width, Depth). You then add some Dense layers which usually expect vectors, not matrices as inputs. Therefore, you have to first flatten the outputs of MaxPooling before giving it to Dense layer, i.e., add a model.add(Flatten()) after model.add(Dropout(0.25)) and before model.add(Dense(128,activation='relu')).
You are doing binary classification, i.e., you have two classes. You are using binary_crossentropy as the loss function, for this to work, you should keep your targets as they are (0 and 1) and not use y_train = keras.utils.to_categorical(y_train,2). Your final layer should have 1 neuron and not 2 (Change model.add(Dense(2, activation='sigmoid')) into model.add(Dense(1, activation='sigmoid')) )
I want to learn a convnet to classify > 240.000 docs in approx 2000 classes. For this I selected the first 60 words and converted them to indices.
I tried to implement a OneHot layer in Keras to avoid memory issues but the model performs much worse than the model with the data already prepared as OneHot. What is the real difference?
The models summary reports are similar in shape and parameters except for the additional One_hot Lambda layer. I used the One_Hot function described here: https://fdalvi.github.io/blog/2018-04-07-keras-sequential-onehot/
def OneHot(input_dim=None, input_length=None):
# input_dim refers to the eventual length of the one-hot vector (e.g.
vocab size)
# input_length refers to the length of the input sequence
# Check if inputs were supplied correctly
if input_dim is None or input_length is None:
raise TypeError("input_dim or input_length is not set")
# Helper method (not inlined for clarity)
def _one_hot(x, num_classes):
return K.one_hot(K.cast(x, 'uint8'),
num_classes=num_classes)
# Final layer representation as a Lambda layer
return Lambda(_one_hot,
arguments={'num_classes': input_dim},
input_shape=(input_length,))
# Model A : This is the Keras model I use with the OneHot function:
model = Sequential()
model.add(OneHot(input_dim=model_max,
input_length=input_length))
model.add(Conv1D(256, 6, activation='relu'))
model.add(Conv1D(64, 3, activation='relu'))
model.add(MaxPooling1D(3))
model.add(Conv1D(128, 3, activation='relu'))
model.add(Conv1D(128, 3, activation='relu'))
model.add(GlobalAveragePooling1D())
model.add(Dropout(0.5))
model.add(Dense(labels_max, activation='softmax'))
checkpoint = ModelCheckpoint('model-best.h5', verbose=1,
monitor='val_loss',save_best_only=True, mode='auto')
model.compile(optimizer=Adam(),
loss='categorical_crossentropy',
metrics=['accuracy'])
#Model B: And this model I use with the data already converted to OneHot:
model = Sequential()
model.add(Conv1D(256, 6, activation='relu', input_shape=(input_length,
model_max)))
model.add(Conv1D(64, 3, activation='relu'))
model.add(MaxPooling1D(3))
model.add(Conv1D(128, 3, activation='relu'))
model.add(Conv1D(128, 3, activation='relu'))
model.add(GlobalAveragePooling1D())
model.add(Dropout(0.5))
model.add(Dense(labels_max, activation='softmax'))
checkpoint = ModelCheckpoint('model-best.h5', verbose=1,
monitor='val_loss',save_best_only=True, mode='auto')
model.compile(optimizer=Adam(),
loss='categorical_crossentropy',
metrics=['accuracy'])
Model B is performing much better with validation accuracy up to 60% but it runs easily into memory errors.
Model A is much faster but only reaches a maximum validation accuracy of 25%.
I would expect them to perform similar. What am I missing here? Thanks!
I'm trying to create a keras LSTM to predict time series. My x_train is shaped like 3000,15,10 (Examples, Timesteps, Features), y_train like 3000,15,1 and I'm trying to build a many to many model (10 input features per sequence make 1 output / sequence).
The code I'm using is this:
model = Sequential()
model.add(LSTM(
10,
input_shape=(15, 10),
return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(
100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(1, activation='linear'))
model.compile(loss="mse", optimizer="rmsprop")
model.fit(
X_train, y_train,
batch_size=512, nb_epoch=1, validation_split=0.05)
However, I can't fit the model when using :
model.add(Dense(1, activation='linear'))
>> Error when checking model target: expected dense_1 to have 2 dimensions, but got array with shape (3000, 15, 1)
or when formatting it this way:
model.add(Dense(1))
model.add(Activation("linear"))
>> Error when checking model target: expected activation_1 to have 2 dimensions, but got array with shape (3000, 15, 1)
I already tried flattening the model ( model.add(Flatten()) ) before adding the dense layer but that just gives me ValueError: Input 0 is incompatible with layer flatten_1: expected ndim >= 3, found ndim=2. This confuses me because I think my data actually is 3 dimensional, isn't it?
The code originated from https://github.com/Vict0rSch/deep_learning/tree/master/keras/recurrent
In case of keras < 2.0: you need to use TimeDistributed wrapper in order to apply it element-wise to a sequence.
In case of keras >= 2.0: Dense layer is applied element-wise by default.
Since you updated your keras version and your error messages changed, here is what works on my machine (Keras 2.0.x)
This works:
model = Sequential()
model.add(LSTM(10,input_shape=(15, 10), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM( 100, return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(1, activation='linear'))
This also works:
model = Sequential()
model.add(LSTM(10,input_shape=(15, 10), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM( 100, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(1,return_sequences=True, activation='linear'))
Testing with:
x = np.ones((3000,15,10))
y = np.ones((3000,15,1))
Compiling and training with:
model.compile(optimizer='adam',loss='mse')
model.fit(x,y,epochs=4,verbose=2)