Here is my model code:
encoder = Embedding(input_dim=dataset.shape[0],output_dim=300, mask_zero=True, input_length=12,embeddings_initializer='uniform')
encoder = LSTM(epochs, input_shape=(train_X.shape[1], train_X.shape[2]), return_sequences=True, unroll=True)
encoder_last = encoder[:,-1,:]
and I got the following error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-88-3967dfedaa44> in <module>
1 encoder = Embedding(input_dim=dataset.shape[0],output_dim=300, mask_zero=True, input_length=12,embeddings_initializer='uniform')
2 encoder = LSTM(epochs, input_shape=(train_X.shape[1], train_X.shape[2]), return_sequences=True, unroll=True)
----> 3 encoder_last = encoder[:,-1,:]
TypeError: 'LSTM' object is not subscriptable
How should I fix it?
I guess you want to apply the LSTM layer on the output of Embedding layer and then take the last output of LSTM. Therefore, first you need to call (i.e. apply) the layers you have defined on some tensors (i.e. output of a layer) like this:
inp = Input(shape=...)
encoder = Embedding(...)(inp) # call embedding layer on inputs
encoder = LSTM(...)(encoder) # call lstm layer on the output of embedding layer
This way the layers are connected to each other. Then you need to use a Lambda layer to slice the LSTM layer output:
encoder_last = Lambda(lambda x: x[:,-1,:])(encoder)
Related
I'm trying to test Non-Intrusive Load Monitoring algorithm which was Proposed by Kelly, 2015. The proposed architecture was,
Input (length determined by appliance duration)
1D conv (lter size=4, stride=1, number of lters=16,activation function=linear, border mode=same)
bidirectional LSTM (N=128, with peepholes)
bidirectional LSTM (N=256, with peepholes)
Fully connected (N=128, activation function=TanH)
Fully connected (N=1, activation function=linear)
Now I'm going to test this architecture in TensorFlow. I have an initial training set like fallows,
x_train_temp = (243127,)
y_train_temp = (243127,)
Then I converted these data into windows like below where each window has 250 x1 arrays.
x_train = (972, 250, 1)
y_train = (972, 250, 1)
When I'm implementing my model, It gives an error. Could you please help me to understand where the error is?
Model
input_shape=x_train.shape[1:]
model = Sequential()
model.add(tf.keras.layers.Conv1D(16,4, strides=1,activation='linear',input_shape=(x_train.shape[1:])))
model.add(tf.keras.layers.LSTM(128))
model.add(tf.keras.layers.LSTM(256))
model.add(tf.keras.layers.LSTM(128, activation='relu'))
model.add(tf.keras.layers.LSTM(128, activation='linear'))
print(model.summary())
Error
ValueError Traceback (most recent call last)
<ipython-input-47-4f5e2441909e> in <module>()
4 model.add(tf.keras.layers.Conv1D(16,4, strides=1,activation='linear',input_shape=(x_train.shape[1:])))
5 model.add(tf.keras.layers.LSTM(128))
----> 6 model.add(tf.keras.layers.LSTM(256))
7 model.add(tf.keras.layers.LSTM(128, activation='relu'))
8 model.add(tf.keras.layers.LSTM(128, activation='linear'))
5 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
178 'expected ndim=' + str(spec.ndim) + ', found ndim=' +
179 str(ndim) + '. Full shape received: ' +
--> 180 str(x.shape.as_list()))
181 if spec.max_ndim is not None:
182 ndim = x.shape.ndims
ValueError: Input 0 of layer lstm_1 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 128]
When using stacked LSTM layers use return_sequences=True as mentioned by #Ather Cheema. LSTM expects input of shape 3D tensor with shape [batch, timesteps, feature]. It is highly recommended to set return_sequences=True
return_sequences: Boolean. Whether to return the last output. in the output sequence, or the full sequence. Default: False.
Working sample code snippet
Without retuen_sequences
import tensorflow as tf
inputs = tf.random.normal([32, 10, 8])
lstm = tf.keras.layers.LSTM(4)
output = lstm(inputs)
print(output.shape)
Output
(32, 4)
#With return_sequences=True
import tensorflow as tf
inputs = tf.random.normal([32, 10, 8])
lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
print(whole_seq_output.shape)
Output
(32, 10, 4)
I am trying to create a Functional API as opposed to a Sequential API. I have built the model previously using the Sequential API, and it worked just fine. It is an LSTM, and I am having trouble with the batch_size going from the Input to the LSTM layer. The Sequential API was built as follows:
new_model = Sequential()
new_model.add(LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True))
new_model.add(Dropout(0))
new_model.add(LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True))
new_model.add(Dropout(0))
new_model.add(Dense(n_neurons1, activation='tanh'))
new_model.add(Dropout(0.1))
new_model.add(Dense(nm))
new_model.compile(loss='mse', optimizer=optimizer)
The above snippet works fine. The Functional API I am trying to get to work is as follows:
inp = Input(shape = (train_X.shape[1], train_X.shape[2]), batch_size = batch_size)
L1 = LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True)(inp)
D1 = Dropout(0)(L1)
L2 = LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True)(D1)
D2 = Dropout(0)(L2)
F1 = Dense(n_neurons1, activation='tanh')(D2)
D3 = Dropout(0.1)(F1)
out = Dense(nm)
new_model = Model(inp,out)
new_model.compile(loss='mse', optimizer=optimizer)
I get an error saying "Input() got an unexpected keyword argument 'batch_size", even though I know batch_size is an argument for the Input layer. Then, if I get rid of the argument, I get an error with the first LSTM layer saying:
"If a RNN is stateful, it needs to know its batch size. Specify the batch size of your input tensors:
If using a Sequential model, specify the batch size by passing a batch_input_shape argument to your first layer.
If using the functional API, specify the batch size by passing a batch_shape argument to your Input layer."
I have already tried updating tensorflow but that did not fix the Input() issue. Where do I go from here?
You describe passing a batch_size parameter via the functional API and getting an error suggesting "passing a batch_shape argument to your Input layer."
If you try changing batch_size = batch_size in your input layer to
batch_shape = (batch_size,train_X.shape[1], train_X.shape[2])
does that solve it?
I'm trying to use keras to make a cnn and use a SVM after the dense layer of cnn as classifier. So I'm trying to get the output after the dense layer.
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=(5), input_shape=(12,800), strides=2, padding='valid', activation='relu'))
model.add(AveragePooling1D(pool_size=2,strides=2,padding='same'))
model.add(Flatten())
model.add(Dense(7 ,activation='softmax'))
model.compile(optimizer='adam',loss='mean_squared_error',metrics=['accuracy'])
model.summary()
My code for getting the output after the dense layer
inp = model.input # input placeholder
outputs = [layer.output for layer in model.layers] # all layer outputs
functor = k.function([inp, k.learning_phase()], outputs ) # evaluation function
# Testing
test = np.random.random(input_shape)[np.newaxis,...]
layer_outs = functor([test, 1.])
But I'm getting this error
TypeError: Cannot convert 1.0 to EagerTensor of dtype int32
work with medicala images, I use VGG16 for the classification of two classes, I remove the last layer (predictions (Dense)) and I add two layers, I had an accuracy of 71% during 200 epochs, I want to use my pre-training model to locate the areas in image with Grad CAM +++, so when I call my model, I got this error !!
How can I solve this problem? Is my method correct or not? please help
vgg16_model=keras.applications.vgg16.VGG16()
vgg16_model.layers.pop()`
model=Sequential()
for layer in vgg16_model.layers:
model.add(layer)
for layer in model.layers:
layer.trainable=False
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax', name='predic'))
from keras.optimizers import SGD
#model.compile(Adam(lr=0.0001),loss='categorical_crossentropy',metrics=['accuracy'])
import time `
start = time.time()
history = model.fit_generator(generator=train_batches,
epochs=epochs,
steps_per_epoch=steps_train,
#callbacks=callbacks_list,
validation_data=valid_batches,
validation_steps=steps_valid,
shuffle=True)
end = time.time()
model = model(include_top=True, weights='imagenet',input_shape=(224,224,3))
TypeError Traceback (most recent call
last) in
----> 1 model = model(include_top=True, weights='imagenet',input_shape=(224,224,3))
TypeError: call() missing 1 required positional argument: 'inputs'
I a trying to merge 2 sequential models in keras. Here is the code:
model1 = Sequential(layers=[
# input layers and convolutional layers
Conv1D(128, kernel_size=12, strides=4, padding='valid', activation='relu', input_shape=input_shape),
MaxPooling1D(pool_size=6),
Conv1D(256, kernel_size=12, strides=4, padding='valid', activation='relu'),
MaxPooling1D(pool_size=6),
Dropout(.5),
])
model2 = Sequential(layers=[
# input layers and convolutional layers
Conv1D(128, kernel_size=20, strides=5, padding='valid', activation='relu', input_shape=input_shape),
MaxPooling1D(pool_size=5),
Conv1D(256, kernel_size=20, strides=5, padding='valid', activation='relu'),
MaxPooling1D(pool_size=5),
Dropout(.5),
])
model = merge([model1, model2], mode = 'sum')
Flatten(),
Dense(256, activation='relu'),
Dropout(.5),
Dense(128, activation='relu'),
Dropout(.35),
# output layer
Dense(5, activation='softmax')
return model
Here is the error log:
File
"/nics/d/home/dsawant/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py",
line 392, in is_keras_tensor
raise ValueError('Unexpectedly found an instance of type ' + str(type(x)) + '. ' ValueError: Unexpectedly found an instance of
type <class 'keras.models.Sequential'>. Expected a symbolic tensor
instance.
Some more log:
ValueError: Layer merge_1 was called with an input that isn't a
symbolic tensor. Received type: class 'keras.models.Sequential'.
Full input: [keras.models.Sequential object at 0x2b32d518a780,
keras.models.Sequential object at 0x2b32d521ee80]. All inputs to the
layer should be tensors.
How can I merge these 2 Sequential models that use different window sizes and apply functions like 'max', 'sum' etc to them?
Using the functional API brings you all possibilities.
When using the functional API, you need to keep track of inputs and outputs, instead of just defining layers.
You define a layer, then you call the layer with an input tensor to get the output tensor. Models and layers can be called exactly the same way.
For the merge layer, I prefer using other merge layers that are more intuitive, such as Add(), Multiply() and Concatenate() for instance.
from keras.layers import *
mergedOut = Add()([model1.output,model2.output])
#Add() -> creates a merge layer that sums the inputs
#The second parentheses "calls" the layer with the output tensors of the two models
#it will demand that both model1 and model2 have the same output shape
This same idea apply to all the following layers. We keep updating the output tensor giving it to each layer and getting a new output (if we were interested in creating branches, we would use a different var for each output of interest to keep track of them):
mergedOut = Flatten()(mergedOut)
mergedOut = Dense(256, activation='relu')(mergedOut)
mergedOut = Dropout(.5)(mergedOut)
mergedOut = Dense(128, activation='relu')(mergedOut)
mergedOut = Dropout(.35)(mergedOut)
# output layer
mergedOut = Dense(5, activation='softmax')(mergedOut)
Now that we created the "path", it's time to create the Model. Creating the model is just like telling at which input tensors it starts and where it ends:
from keras.models import Model
newModel = Model([model1.input,model2.input], mergedOut)
#use lists if you want more than one input or output
Notice that since this model has two inputs, you have to train it with two different X_training vars in a list:
newModel.fit([X_train_1, X_train_2], Y_train, ....)
Now, suppose you wanted only one input, and both model1 and model2 would take the same input.
The functional API allows that quite easily by creating an input tensor and feeding it to the models (we call the models as if they were layers):
commonInput = Input(input_shape)
out1 = model1(commonInput)
out2 = model2(commonInput)
mergedOut = Add()([out1,out2])
In this case, the Model would consider this input:
oneInputModel = Model(commonInput,mergedOut)