I am trying to create a CNN + LSTM network, but the LSTM layer is not accepting the input shape. Is there anything I can do?
model = Sequential()
model.add(Conv2D(128, (2,2), padding = 'same', input_shape=(30, 216, 1)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(Conv2D(256, (2,2), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(LSTM(512, input_shape = (7, 54, 256,)))
model.add(Flatten())
model.add(Dense(7, activation='softmax'))
ValueError: Input 0 of layer lstm_21 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 7, 54, 256]
The LSTM layer in Keras expects this format as input:
inputs: A 3D tensor with shape [batch, timesteps, feature].
For this reason, you can't pass non-recurrent layers directly. First, Flatten() the layer before, and wrap that layer into a TimeDistributed layer, like this:
model.add(TimeDistributed(Flatten()))
model.add(LSTM(8))
This TimeDistributed layer allows to apply a layer to every temporal slice of an input. Here's a fully working example:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, LSTM, \
Dense, Flatten, Dropout, MaxPooling2D, Activation, TimeDistributed
import numpy as np
X = np.random.rand(100, 30, 216, 1)
y = np.random.randint(0, 7, 100)
model = Sequential()
model.add(Conv2D(16, (2,2), padding = 'same', input_shape=(30, 216, 1)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(Conv2D(32, (2,2), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(8))
model.add(Dense(7, activation='softmax'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
history = model.fit(X, y)
Related
I'm relatively new to programming neural networks and have been following a few tutorials about it before deciding to try and learn to program neural networks on my own using what i'd learnt. I've been trying to program a basic neural network just so i can learn how it works, but it keeps giving me an error. I would really appreciate it if someone could help.
Here's my code:
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
import pickle
pickle_in = open("X.pickle","rb")
X = pickle.load(pickle_in)
pickle_in = open("y.pickle","rb")
y = pickle.load(pickle_in)
X = X/255.0
model = Sequential()
model.add(Conv2D(8,(5, 5),padding="same",activation='relu',input_shape=(784,)))
model.add(Conv2D(32, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Flatten())
model.add(Dense(1))
model.add(Activation('linear'))
model.add(Dense(y.shape[1]))
model.add(Activation('linear'))
model.summary()
model.compile(loss='mean_squared_error',optimizer='adam',metrics=['mae','mse', 'accuracy'])
model.fit(X, y, epochs=20, batch_size=10,verbose=2)
Here's the error message I'm getting:
str(x.shape.as_list()))
ValueError: Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=2. Full shape received: [None, 784]
Thanks in advance!
The error is cause Conv2D expects input shape to be of 4 dimensions i.e. [batch_size, height, width, channels]. You can do one thing i.e. reshape your input to the model.
X = X.reshape(-1, 28, 28, 1) # incase of single channel (grayscale)
# OR
X = X.reshape(-1, 28, 28, 3) # incase of RGB
# and then change the input shape of your `Conv2D` layer accordingly to
model = Sequential()
model.add(Conv2D(8,(5, 5),padding="same",activation='relu',input_shape=(28,28,1)))
# OR
model = Sequential()
model.add(Conv2D(8,(5, 5),padding="same",activation='relu',input_shape=(28,28,3)))
I used two different models for my case.
The case is classification for different type of surface defect.
The input shape is (200, 200, 1), and there are 6 classes.
The numbers of training data is 1440(240 for 1 class), and the number of validation data is 360(60 for 1 class).
The training process is very well with first model. Both training loss and validation loss are dropping very quickly.
After that, I want to use MobileNetV2 from keras for comparing the training result. The training loss and accuracy in MobileV2 are improve, but the validation accuracy stuck in 0.1667(the loss is bumpy).
I wonder know what causes this result, Can I call this situation 'Over-fitting'? Or just this model is too deep to my case?
First model:
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization
import parameter
input_shape = (parameter.IMAGE_SIZE_Y, parameter.IMAGE_SIZE_X, parameter.channel)
def MyModel():
model = Sequential()
model.add(Conv2D(16, (3, 3), input_shape = input_shape, padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides = (2, 2)))
model.add(Flatten())
model.add(Dense(256, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(6, activation = 'softmax'))
model.summary()
return model
Second model:
from keras.models import Sequential, Model
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, BatchNormalization, GlobalAveragePooling2D
from keras.optimizers import Adam
from keras.applications import MobileNetV2
import parameter
def MyMobileNetV2():
input_shape = (parameter.IMAGE_SIZE_X, parameter.IMAGE_SIZE_Y, parameter.channel)
model = MobileNetV2(input_shape = input_shape,
include_top = False,
weights = 'imagenet')
x = model.output
x = GlobalAveragePooling2D()(x)
x = BatchNormalization()(x)
x = Dense(1280, activation='relu')(x)
x = BatchNormalization()(x)
predictions = Dense(6, activation='softmax', kernel_initializer='random_uniform', bias_initializer='zeros')(x)
model = Model(inputs = model.input, outputs = predictions)
optimizer = Adam(lr=0.01)
loss = "categorical_crossentropy"
for layer in model.layers:
layer.trainable = True
model.compile(optimizer=optimizer,
loss=loss,
metrics=["accuracy"])
model.summary()
for i, layer in enumerate(model.layers):
print(i, layer.name, layer.trainable)
return model
I'm trying to combine LSTM with CNN but I got stuck because of an error.
Here is the model I'm trying to implement:
model=Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(28, 28,3), activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(LSTM(128, return_sequences=True,input_shape=(1,32), activation='relu'))
model.add(LSTM(256))
model.add(Dropout(0.25))
model.add(Dense(37))
model.compile(loss='categorical_crossentropy', optimizer='adam')
and error happens in the first LSTM layer:
ERROR: Input 0 is incompatible with layer lstm_12: expected ndim=3, found ndim=2
The input of LSTM layer should be a 3D array which represents a sequence or a timeseries (this is what the error is trying to say: expected ndim=3). However, in your model the input of LSTM layer, which is actually the output of the Dense layer before it, is a 2D array (i.e. found ndim=2). To make it into a 3D array of shape (n_samples, n_timesteps, n_features), one solution is to use a RepeatVector layer to repeat it as much as the number of timesteps (which you need to specify in your code):
model.add(Dense(32, activation='relu'))
model.add(RepeatVector(n_timesteps))
model.add(LSTM(128, return_sequences=True, input_shape=(n_timesteps,32), activation='relu'))
Here is my code :
from keras import optimizers from keras.layers import Convolution1D, Dense, MaxPooling1D, Flatten
model = Sequential()
# my CNN layers
model.add(Conv1D(101, 101, strides=1, padding='same', dilation_rate=1, input_shape=(None, 120)))
model.add(Activation('relu')) model.add(MaxPooling1D(pool_size=2, padding='same', strides=None))
model.add(Dense(2048)) model.add(Activation('relu'))
model.add(Dense(100)) model.add(Activation('sigmoid'))
model.compile(optimizer=optimizers.Adam(lr=1e-4), loss='binary_crossentropy', metrics=['accuracy'])
model.fit(training_trainX_train, training_trainY_train, epochs=2, batch_size=100, verbose=1)
But I get this error: ValueError: Error when checking model input: expected conv1d_8_input to have 3 dimensions, but got array with shape (27660, 120)
Here is my training-set's shape :
training_trainX_train.shape = (27660, 120)
training_trainY_train.shape = (27660, 101)
Adding model.add(Flatten()) will resolve this
model.add(Conv1D(101, 101, strides=1, padding='same', dilation_rate=1, input_shape=(None, 120)))
model.add(Activation('relu')) model.add(MaxPooling1D(pool_size=2, padding='same', strides=None))
model.add(Flatten())
model.add(Dense(2048)) model.add(Activation('relu'))
model.add(Dense(100)) model.add(Activation('sigmoid'))
For more detail check https://github.com/keras-team/keras/issues/6351
I have a CNN and like to change this to a LSTM, but when I modified my code I receive the same error: ValueError: Input 0 is incompatible with layer gru_1: expected ndim=3, found ndim=4
I already change ndim but didn't work.
follow my cnn
def build_model(X,Y,nb_classes):
nb_filters = 32 # number of convolutional filters to use
pool_size = (2, 2) # size of pooling area for max pooling
kernel_size = (3, 3) # convolution kernel size
nb_layers = 4
input_shape = (1, X.shape[2], X.shape[3])
model = Sequential()
model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],
border_mode='valid', input_shape=input_shape))
model.add(BatchNormalization(axis=1))
model.add(Activation('relu'))
for layer in range(nb_layers-1):
model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
model.add(BatchNormalization(axis=1))
model.add(ELU(alpha=1.0))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation("softmax"))
return model
and follow how i like to did my LSTM
data_dim = 41
timesteps = 20
num_classes = 10
model = Sequential()
model.add(LSTM(256, return_sequences=True, input_shape=(timesteps, data_dim)))
model.add(Dropout(0.5))
model.add(LSTM(128, return_sequences=True, input_shape=(timesteps, data_dim)))
model.add(Dropout(0.25))
model.add(LSTM(64))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))
What I was doing wrong?
Thanks
The LSTM code is fine, it executes with no errors for me.
The error you are seeing is related to internal incompatibility of the tensors within the model itself, not related to training data, in which case you'll get an "Exception: Invalid input shape"
What's confusing in your error is that it refers to a GRU layer, which isn't contained anywhere in your model definition. If your model only contains LSTM, you should get an error that calls out the LSTM layer that it conflicts with.
Perhaps check
model.get_config()
and make sure all the layers and configs are what you intended.
In particular, the first layer should say this:
batch_input_shape': (None, 20, 41)