I created a CNN-LSTM for survival prediction of web sessions, my training data looks as follows:
print(x_train.shape)
(288, 3, 393)
with (samples, timesteps, features) and my model:
model = Sequential()
model.add(TimeDistributed(Conv1D(128, 5, activation='relu'),
input_shape=(x_train.shape[1], x_train.shape[2])))
model.add(TimeDistributed(MaxPooling1D()))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(64, stateful=True, return_sequences=True))
model.add(LSTM(16, stateful=True))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer=Adam(lr=0.001), loss='binary_crossentropy', metrics=['accuracy'])
However, the TimeDistributed Layer requires a minimum of 3 dimensions, how should I transform the data to get it work?
Thanks a lot!
your data are in 3d format and this is all you need to feed a conv1d or an LSTM. if your target is 2D remember to set return_sequences=False in your last LSTM cell.
using a flatten before an LSTM is a mistake because you are destroying the 3D dimensionality
pay attention also on the pooling operation in order to not have a negative time dimension to reduce (I use 'same' padding in the convolution above in order to avoid this)
below is an example in a binary classification task
n_sample, time_step, n_features = 288, 3, 393
X = np.random.uniform(0,1, (n_sample, time_step, n_features))
y = np.random.randint(0,2, n_sample)
model = Sequential()
model.add(Conv1D(128, 5, padding='same', activation='relu',
input_shape=(time_step, n_features)))
model.add(MaxPooling1D())
model.add(LSTM(64, return_sequences=True))
model.add(LSTM(16, return_sequences=False))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(X,y, epochs=3)
Related
Being new to Keras sequential models is causing me a few troubles!
I have an x_train of shape : 17755 x 500 x 12
and y_train of shape: 17755 x 15 (labels are already one-hot encoded)
And I made the next model to be trained on this data:
model = Sequential()
model.add(Conv2D(32,3,padding="same", activation="relu", input_shape=(17755,500,12)))
model.add(MaxPool2D())
model.add(Conv2D(32, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(MaxPool2D())
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(128,activation="relu"))
model.add(Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)
1- when I don’t use np.expand_dims to add an axis for batch, I get this error:
ValueError: Input 0 of layer "sequential" is incompatible with the
layer: expected shape=(None, 17755, 500, 12), found shape=(None, 500,
12)
2- when I do use np.expand_dims and the shape of x_train became: 1x17755x500x12
I get this error:
Data cardinality is ambiguous:
x sizes: 1
y sizes: 17755
Make sure all arrays contain the same number of samples.
3- when I use np.expand_dims for y_train too and its shape became: 1x17755x15
I get this error:
ValueError: Shapes (None, 17755, 15) and (None, 15) are incompatible
I know I’m doing something fundamentally wrong, but what what is that? Can anyone please help me out with the shape of data please?
Regarding x_train try adding a new dimension at the end to represent the channel dimension needed for Conv2D layers. Note also that you do not provide the number of samples to your input shape. Here is a working example:
import tensorflow as tf
import numpy as np
x_train = np.random.random((17755,500,12))
x_train = np.expand_dims(x_train, axis=-1)
y_train = np.random.random((17755,15))
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32,3,padding="same", activation="relu", input_shape=(500, 12, 1)))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(32, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Conv2D(64, 3, padding="same", activation="relu"))
model.add(tf.keras.layers.MaxPool2D())
model.add(tf.keras.layers.Dropout(0.4))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128,activation="relu"))
model.add(tf.keras.layers.Dense(15, activation="sigmoid"))
model.compile(optimizer ='adam', loss='categorical_crossentropy', metrics = ['Accuracy'])
history = model.fit(x_train, y_train, epochs=5)
I'm getting a list of images to train my CNN.
model = Sequential()
model.add(Dense(32, activation='tanh', input_dim=100))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
data, labels = ReadImages(TRAIN_DIR)
# Train the model, iterating on the data in batches of 32 samples
model.fit(np.array(data), np.array(labels), epochs=10, batch_size=32)
But I faced this error:
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected dense_1_input to have 2 dimensions, but got array with shape (391, 605, 700, 3)
You are feeding images to the Dense Layer. Either flatten the images using .flatten() or use a model with CNN Layers. The shape (391,605,700,3) means you have 391 images of size 605x700 having 3 dimensions(rgb).
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', input_shape=(605, 700, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(100, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
This link has good explanations for basic CNN.
This is no CNN. A Convolutional Neural Network is defined by having Conv Layer. Those Layers work with imput shapes in 4D (Batchsize, ImageDimX, ImageDimY, ColorChannels). The Dense Layers(aka. Fully connected) you are using 2D input (Batchsize, DataAsAVector)
You need to first flatten the image if you want to pass the image directly to dense layers as Dense layer takes input in 2 dimensions only and since you are passing whole image there is 4 dimensions in it i.e. Number of images X Height X Width X Number of channels (391, 605, 700, 3).
You are not actually doing any convolutions on the image. To do convolutions you need to add CNN layers after initialising the model as sequential.
To add dense layer :
model = Sequential()
model.add(Flatten())
model.add(Dense(32, activation='tanh', input_dim=100))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
To add CNN layer and then flatten it :
model = Sequential()
model.add(Conv2D(input_shape=(605,700,3), filters=64, kernel_size=(3,3),
padding="same",activation="relu"))
model.add(Flatten())
model.add(Dense(32, activation='tanh', input_dim=100))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
I'm trying to build a 2 layered neural network for MNIST dataset and I want to get weights from my model.
I found a similar question her on SO and I tried this,
model.get_weights()
But It returned 11 values when I check the len(model.get_weights()) Isn't it suppose to return 3 weights? I have even disabled bias.
model = Sequential()
model.add(Flatten(input_shape = (28, 28)))
model.add(Dense(512, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Dense(128, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.add(BatchNormalization())
model.add(Dropout(0.1))
model.add(Dense(10, activation='relu', kernel_initializer='he_normal', use_bias=False,))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
result = model.fit(x_train, y_train, validation_split=0.25, epochs=10,
batch_size=128, verbose=1)
To get the weights of a particular layer, you could retrieve this layer by using its name and call get_weights on it (as shubham-panchal said in its comment).
For example:
model.get_layer('dense').get_weights()
or
model.get_layer('dense_2').get_weights()
You could go though the layers of your model and retrieve its name and weights:
{layer.name: layer.get_weights() for layer in model.layers}
I am experimenting with the Merku molecular activity challenge and I have created the train and test dataset.
The shape of the data is the following:
x_train.shape=(1452, 4306)
y_train.shape=(1452, 1)
x_test.shape=(363, 4306)
y_test.shape=(363, 1)
I have used the Dense layer for defining the model as follows:
model = Sequential()
model.add(Dense(100, activation="relu", input_shape=(4306,)))
model.add(Dense(50, activation="relu"))
model.add(Dense(25, activation="relu"))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1))
# Compile the model
model.compile(
loss='categorical_crossentropy',
optimizer="adam",
)
model.summary()
# Train the model
model.fit(
x_train,
y_train,
batch_size=300,
epochs=900,
validation_data=(x_test, y_test),
shuffle=True
)
While trying the above code, the following error occurred:
ValueError: Input 0 is incompatible with layer flatten_23: expected min_ndim=3, found ndim=2
How can I resolve this error?
Just remove the flatten layer:
model = Sequential()
model.add(Dense(100, activation="relu", input_shape=(4306,)))
model.add(Dense(50, activation="relu"))
model.add(Dense(25, activation="relu"))
model.add(Dropout(0.25))
model.add(Dense(1))
The data sent to sequential layers is essentially 1-D (ignoring the batch column) so there's nothing to flatten. The data entering the flatten layer is already 1D.
EDIT -- for regression:
Categorical crossentropy is not an appropriate cost function for regression, you need to use mean-square error, which is commonly used for all regression tasks:
model.compile(
loss='mse',
optimizer="adam",
)
I'm trying to create a keras LSTM to predict time series. My x_train is shaped like 3000,15,10 (Examples, Timesteps, Features), y_train like 3000,15,1 and I'm trying to build a many to many model (10 input features per sequence make 1 output / sequence).
The code I'm using is this:
model = Sequential()
model.add(LSTM(
10,
input_shape=(15, 10),
return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(
100,
return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(1, activation='linear'))
model.compile(loss="mse", optimizer="rmsprop")
model.fit(
X_train, y_train,
batch_size=512, nb_epoch=1, validation_split=0.05)
However, I can't fit the model when using :
model.add(Dense(1, activation='linear'))
>> Error when checking model target: expected dense_1 to have 2 dimensions, but got array with shape (3000, 15, 1)
or when formatting it this way:
model.add(Dense(1))
model.add(Activation("linear"))
>> Error when checking model target: expected activation_1 to have 2 dimensions, but got array with shape (3000, 15, 1)
I already tried flattening the model ( model.add(Flatten()) ) before adding the dense layer but that just gives me ValueError: Input 0 is incompatible with layer flatten_1: expected ndim >= 3, found ndim=2. This confuses me because I think my data actually is 3 dimensional, isn't it?
The code originated from https://github.com/Vict0rSch/deep_learning/tree/master/keras/recurrent
In case of keras < 2.0: you need to use TimeDistributed wrapper in order to apply it element-wise to a sequence.
In case of keras >= 2.0: Dense layer is applied element-wise by default.
Since you updated your keras version and your error messages changed, here is what works on my machine (Keras 2.0.x)
This works:
model = Sequential()
model.add(LSTM(10,input_shape=(15, 10), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM( 100, return_sequences=True))
model.add(Dropout(0.2))
model.add(Dense(1, activation='linear'))
This also works:
model = Sequential()
model.add(LSTM(10,input_shape=(15, 10), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM( 100, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(1,return_sequences=True, activation='linear'))
Testing with:
x = np.ones((3000,15,10))
y = np.ones((3000,15,1))
Compiling and training with:
model.compile(optimizer='adam',loss='mse')
model.fit(x,y,epochs=4,verbose=2)