I want to use part of a the tensor in the output of lstm layer, but don't know how to do it correctly.
My purpose is tell LSTM layer the "real" length of its input sequence.
Here is my attempt, but it fails.
Isthere anyone who can help solve this problem and explain the details, thanks a lot~
input_spectrogram = Input(shape=(64,500,1))
input_length = Input(shape=(1,))
cnn1 = Conv2D(filters = 64, kernel_size = (1,4),input_shape=(64,500, 1),padding = 'same',strides = 1,activation = 'relu',name='conv1')(input_spectrogram)
maxpooling1 = MaxPooling2D(pool_size = (1,4),name='maxpooling1')(cnn1)
bn1 = BatchNormalization(name='BN1')(maxpooling1)
cnn2 = Conv2D(filters = 128, kernel_size = (64,1),strides = 1,activation ='relu',name='conv2')(bn1)
maxpooling2 = MaxPooling2D(pool_size = (1,2),name='maxpooling2')(cnn2)
reshape = Reshape((62,128))(maxpooling2)
lstm1 = LSTM(128,return_sequences = True,recurrent_dropout=0.3,name='lstm1')(reshape) #output:(None,62,128)
softmax_in = Lambda(lambda x:x[0][x[1],:])([lstm1,input_length])
softmax_ = Dense(10,activation='softmax',name='softmax_')(softmax_in)
seq = Model(inputs=input_spectrogram, outputs=[softmax_])
seq.compile(loss='categorical_crossentropy', optimizer='adadelta',metrics=['accuracy'])
Seems to be indexing with tensor is not fully supported (see discussion here: https://github.com/tensorflow/tensorflow/issues/206#issuecomment-158435464).
Does it work for you to perform indexing with constant instead?
input_spectrogram = Input(shape=(64,500,1))
input_length = Input(shape=(1,))
cnn1 = Conv2D(filters = 64, kernel_size = (1,4),input_shape=(64,500, 1),padding = 'same',strides = 1,activation = 'relu',name='conv1')(input_spectrogram)
maxpooling1 = MaxPooling2D(pool_size = (1,4),name='maxpooling1')(cnn1)
bn1 = BatchNormalization(name='BN1')(maxpooling1)
cnn2 = Conv2D(filters = 128, kernel_size = (64,1),strides = 1,activation ='relu',name='conv2')(bn1)
maxpooling2 = MaxPooling2D(pool_size = (1,2),name='maxpooling2')(cnn2)
reshape = Reshape((62,128))(maxpooling2)
lstm1 = LSTM(128,return_sequences = True,recurrent_dropout=0.3,name='lstm1')(reshape) #output:(None,62,128)
softmax_in = Lambda(lambda x:x[:,5])(lstm1)
softmax_ = Dense(10,activation='softmax',name='softmax_')(softmax_in)
seq = Model(inputs=input_spectrogram, outputs=[softmax_])
seq.compile(loss='categorical_crossentropy', optimizer='adadelta',metrics=['accuracy'])
now it is feasible, so how to use the "real_length" from an input layer?
Related
I am trying to implement DenseNet model and I am using image dataset with 4 classes.
Code snippets:
For building model:
def denseblock(input, num_filter = 12, dropout_rate = 0.2):
global compression
temp = input
for _ in range(l):
BatchNorm = BatchNormalization()(temp)
relu = Activation('relu')(BatchNorm)
Conv2D_3_3 =Conv2D(int(num_filter*compression), (3,3), use_bias=False ,padding='same')(relu)
if dropout_rate>0:
Conv2D_3_3 = Dropout(dropout_rate)(Conv2D_3_3)
concat = Concatenate(axis=-1)([temp,Conv2D_3_3])
temp = concat
return temp
## transition Block
def transition(input, num_filter = 12, dropout_rate = 0.2):
global compression
BatchNorm = BatchNormalization()(input)
relu = Activation('relu')(BatchNorm)
Conv2D_BottleNeck = Conv2D(int(num_filter*compression), (1,1), use_bias=False ,padding='same')(relu)
if dropout_rate>0:
Conv2D_BottleNeck = Dropout(dropout_rate)(Conv2D_BottleNeck)
avg = AveragePooling2D(pool_size=(2,2))(Conv2D_BottleNeck)
return avg
#output layer
def output_layer(input):
global compression
BatchNorm = BatchNormalization()(input)
relu = Activation('relu')(BatchNorm)
AvgPooling = AveragePooling2D(pool_size=(2,2))(relu)
flat = Flatten()(AvgPooling)
output = Dense(categories, activation='softmax')(flat)
return output
creating a model with the two DenseNet blocks:
l = 7
input = Input(shape=(height, width, 3))
First_Conv2D = Conv2D(30, (3,3), use_bias=False ,padding='same')(input)
First_Block = denseblock(First_Conv2D, 30, 0.5)
First_Transition = transition(First_Block, 30, 0.5)
Last_Block = denseblock(First_Transition, 30, 0.5)
output = output_layer(Last_Block)
model = Model(inputs=[input], outputs=[output])
The error is: NameError: name 'compression' is not defined
I am guessing compression is your bottleneck width, but is does not seem defined, maybe try:
compression = 4
def denseblock(input, num_filter = 12, dropout_rate = 0.2):
temp = input
for _ in range(l):
BatchNorm = layers.BatchNormalization()(temp)
relu = layers.Activation('relu')(BatchNorm)
Conv2D_3_3 = layers.Conv2D(int(num_filter*compression), (3,3), use_bias=False ,padding='same')(relu)
if dropout_rate>0:
Conv2D_3_3 = layers.Dropout(dropout_rate)(Conv2D_3_3)
concat = layers.Concatenate(axis=-1)([temp,Conv2D_3_3])
temp = concat
return temp
## transition Blosck
def transition(input, num_filter = 12, dropout_rate = 0.2):
BatchNorm = layers.BatchNormalization()(input)
relu = layers.Activation('relu')(BatchNorm)
Conv2D_BottleNeck = layers.Conv2D(int(num_filter*compression), (1,1), use_bias=False ,padding='same')(relu)
if dropout_rate>0:
Conv2D_BottleNeck = layers.Dropout(dropout_rate)(Conv2D_BottleNeck)
avg = layers.AveragePooling2D(pool_size=(2,2))(Conv2D_BottleNeck)
return avg
#output layer
def output_layer(input):
BatchNorm = layers.BatchNormalization()(input)
relu = layers.Activation('relu')(BatchNorm)
AvgPooling = layers.AveragePooling2D(pool_size=(2,2))(relu)
flat = layers.Flatten()(AvgPooling)
output = layers.Dense(10, activation='softmax')(flat)
return output
l = 7
input = layers.Input(shape=(28, 28, 3,))
First_Conv2D = layers.Conv2D(30, (3,3), use_bias=False ,padding='same')(input)
First_Block = denseblock(First_Conv2D, 30, 0.5)
First_Transition = transition(First_Block, 30, 0.5)
Last_Block = denseblock(First_Transition, 30, 0.5)
output = output_layer(Last_Block)
model = Model(inputs=[input], outputs=[output])
I want to solve this below error.
InvalidArgumentError: Input to reshape is a tensor with 737280 values, but the requested shape requires a multiple of 184832
so I see reference.
reference
Python / Tensorflow - Input to reshape is a tensor with 92416 values, but the requested shape requires a multiple of 2304
However, looking at the answers to this problem, I do not know where to fix it.
So ,I would like to know how to check the size of the input image.
Thank you for your time.
my model:
# For multi_model
activationFunction='elu'
def build_multi2(main_input_shape, output_dim):
inputA = Input(shape=main_input_shape)
ch1_model = create_convolution_layers(inputA)
inputB = Input(shape=main_input_shape)
ch2_model = create_convolution_layers(inputB)
inputC = Input(shape=main_input_shape)
ch3_model = create_convolution_layers(inputC)
inputD = Input(shape=main_input_shape)
ch4_model = create_convolution_layers(inputD)
conv = concatenate([ch1_model, ch2_model, ch3_model, ch4_model])
conv = Flatten()(conv)
dense = Dense(512)(conv)
dense = LeakyReLU(alpha=0.1)(dense)
dense = Dropout(0.5)(dense)
output = Dense(N_class, activation='softmax')(dense)
return Model(inputs=[inputA, inputB, inputC, inputD], outputs=[output])
def create_convolution_layers(input_img):
model = Conv2D(32, (3, 3), padding='same', input_shape=main_input_shape)(input_img)
model = LeakyReLU(alpha=0.1)(model)
model = MaxPooling2D((2, 2),padding='same')(model)
model = Dropout(0.25)(model)
model = Conv2D(64, (3, 3), padding='same')(model)
model = LeakyReLU(alpha=0.1)(model)
model = MaxPooling2D(pool_size=(2, 2),padding='same')(model)
model = Dropout(0.25)(model)
model = Conv2D(128, (3, 3), padding='same')(model)
model = LeakyReLU(alpha=0.1)(model)
model = MaxPooling2D(pool_size=(2, 2),padding='same')(model)
model = Dropout(0.4)(model)
return model
my model call
# For model declaration
N_class = 20
main_input_shape = (150,150, 3)
output_dim = N_class
# opt = tf.keras.optimizers.RMSprop(lr=0.001)
opt = tf.keras.optimizers.Adam()
clf = build_multi2(main_input_shape, output_dim)
clf.compile(optimizer=opt, loss=['categorical_crossentropy'], metrics=['accuracy'])
clf.summary()
my image size: 96×96 pixel
my tensorflow. ImageDataGenerator
train_imgen = ImageDataGenerator(rescale = 1./255,
# shear_range = 0.2,
# zoom_range = 0.2,
# rotation_range=5.,
horizontal_flip = False)
'''
You have specified your input shape as (150, 150, 3) and your image shape is (96, 96, 3), these are incompatible.
You can either resize your images to (150, 150, 3) or change your input shape to be the same as your image shape.
I am getting this error when I am trying to train my model:
ValueError: Input 0 of layer dense_encoder is incompatible with the layer: expected axis -1 of input shape to have value 2048 but received input with shape [446, 98, 1024]
My model architecture is:
input1 = Input(shape=(2048), name='Image_1')
dense1 = Dense(256, kernel_initializer=tf.keras.initializers.glorot_uniform(seed = 56), name='dense_encoder')(input1)
input2 = Input(shape=(153), name='Text_Input')
emb_layer = Embedding(input_dim = vocab_size, output_dim = 300, input_length=153, mask_zero=True, trainable=False,
weights=[embedding_matrix], name="Embedding_layer")
emb = emb_layer(input2)
LSTM1 = LSTM(units=256, activation='tanh', recurrent_activation='sigmoid', use_bias=True,
kernel_initializer=tf.keras.initializers.glorot_uniform(seed=23),
recurrent_initializer=tf.keras.initializers.orthogonal(seed=7),
bias_initializer=tf.keras.initializers.zeros(), return_sequences=True, name="LSTM1")(emb)
#LSTM1_output = LSTM1(emb)
LSTM2 = LSTM(units=256, activation='tanh', recurrent_activation='sigmoid', use_bias=True,
kernel_initializer=tf.keras.initializers.glorot_uniform(seed=23),
recurrent_initializer=tf.keras.initializers.orthogonal(seed=7),
bias_initializer=tf.keras.initializers.zeros(), name="LSTM2")
LSTM2_output = LSTM2(LSTM1)
dropout1 = Dropout(0.5, name='dropout1')(LSTM2_output)
dec = tf.keras.layers.Add()([dense1, dropout1])
fc1 = Dense(256, activation='relu', kernel_initializer=tf.keras.initializers.he_normal(seed = 63), name='fc1')
fc1_output = fc1(dec)
dropout2 = Dropout(0.4, name='dropout2')(fc1_output)
output_layer = Dense(vocab_size, activation='softmax', name='Output_layer')
output = output_layer(dropout2)
encoder_decoder = Model(inputs = [input1, input2], outputs = output)
encoder_decoder.summary()
Here's my code for training the model:
for epoch in range(20):
print('EPOCH : ',epoch+1)
start = time.time()
batch_loss_tr = 0
batch_loss_vl = 0
for img, report in train_generator:
r1 = bytes_to_string(report.numpy())
img_input, rep_input, output_word = convert(img.numpy(), r1)
rep_input = pad_sequences(rep_input, maxlen=153, padding='post')
results = encoder_decoder.train_on_batch([img_input, rep_input], output_word)
batch_loss_tr += results
train_loss = batch_loss_tr/(X_train.shape[0]//14)
with train_summary_writer.as_default():
tf.summary.scalar('loss', train_loss, step = epoch)
for img, report in cv_generator:
r1 = bytes_to_string(report.numpy())
img_input, rep_input, output_word = convert(img.numpy(), r1)
rep_input = pad_sequences(rep_input, maxlen=153, padding='post')
results = encoder_decoder.test_on_batch([img_input, rep_input], output_word)
batch_loss_vl += results
The img_input shape is (417, 98, 1024) and I am getting the error for Image_1 layer.
What could be the reasons? Any help would be appreciated.
my convolutional neural network is returning only ones and zeros on softmax output (out1), anyone knows why?
def build(self):
inp = Input(self.obs_shape)
conv0 = Conv2D(32, 2, 1, padding="same", activation = "relu")(inp)
drop0 = MaxPool2D((2,2))(conv0)
conv1 = Conv2D(64, 3, 2, padding="same", activation = "relu")(drop0)
drop1 = MaxPool2D((2,2))(conv1)
flat = Flatten()(drop1)
hid0 = Dense(128, activation='relu')(flat)
hid1 = Dense(256, activation='relu')(hid0)
hid = Dense(128, activation='relu')(hid1)
out1 = Dense(self.action_count, activation='softmax')(hid)
out2 = Dense(1, activation='linear')(hid)
model = Model(inputs = [inp], outputs = [out1, out2])
model.compile(optimizer = tf.keras.optimizers.Adam(lr = self.lr),
loss = [self.actor_loss, "mse"])
return model
def actor_loss(self, y_actual, y_pred):
actions = tf.cast(y_actual[:, 0], tf.int32)
returns = y_actual[:, 1]
mask = tf.one_hot(actions, self.action_count)
logps = tf.math.log(tf.boolean_mask(y_pred, mask) + 1e-3)
entropy = -tf.math.reduce_sum(y_pred * tf.math.log(y_pred))
return -tf.math.reduce_sum(logps * returns) - 0.0001*entropy
model = Model(inputs = [inp], outputs = [out1, out2])
look at above， there only two output.
so, you function build was lock the number of output,
so only get 1 or 0 ;
in one word :you need change your models
sorry ,my english is bad .
I'm trying to build a model that looks like this:
input
/
convlayers/flatten
/ \
first_output \
\ /
second_output
but it fails at the first conv layers with the error:
ValueError: Layer conv2d_4 was called with an input that isn't a symbolic tensor.
Received type: <class 'keras.layers.convolutional.Conv2D'>.
Full input: [<keras.layers.convolutional.Conv2D object at 0x7f450d7b8630>].
All inputs to the layer should be tensors.
and the error points to the layer after the first conv with the inputshape call.
Help would be appreciated.
Here is the code:
conv1 = Conv2D(8, 4, padding = "same", strides = 2)(inputs)
conv2 = Conv2D(16 ,4, padding = "same", strides = 2)(conv1)
flat = Flatten()(conv2)
dense1 = Dense(32)(flat)
dense2 = Dense(32)(dense1)
first_output = Dense(64)(dense2)
merged = concatenate([flat,first_output])
second_output_dense1 = Dense(32)(merged)
second_output_dense2 = Dense(32)(second_output_dense1)
second_output = Dense(64)(second_output_dense2)
model = Model(inputs=conv1, outputs=[first_output,second_output])
model.compile(loss = "mse", optimizer = "adam" )
Answer:
i was under the impression that you could call the model without an input layer and just define the input in the first layer : conv1 = Conv2D(8, 4, padding = "same", strides = 2, input_shape = (6,8,8,))
but that didnt work so instead you have to delete the input shape thing and create an input layer here is the fixed code
inputs = Input(shape=(6,8,8,))
conv1 = Conv2D(8, 4, padding = "same", strides = 2, input_shape = (6,8,8,))
conv2 = Conv2D(16 ,4, padding = "same", strides = 2)(conv1)
flat = Flatten()(conv2)
dense1 = Dense(32)(flat)
dense2 = Dense(32)(dense1)
first_output = Dense(64)(dense2)
merged = concatenate([flat,first_output])
second_output_dense1 = Dense(32)(merged)
second_output_dense2 = Dense(32)(second_output_dense1)
second_output = Dense(64)(second_output_dense2)
model = Model(inputs=inputs, outputs=[first_output,second_output])
model.compile(loss = "mse", optimizer = "adam" )