I am new to Keras.
I want to build a CNN network with Keras.
It is fine until I want to add a softmax dense layer.
The error is TypeError: softmax() got an unexpected keyword argument 'axis'.
I searched Keras and tried to fix it:
from keras.layers import Softmax
output_layer = Dense(units=n_classes, activation=Softmax(axis=-1), kernel_initializer='uniform')
However, it still gives error plus the warning:
UserWarning: Do not pass a layer instance (such as Softmax) as the activation argument of another layer. Instead, advanced activation layers should be used just like any other layer in a model.
What is the correct method of applying softmax?
Thank you very much.
cnn = Sequential()
kernelSize = (3, 3)
ip_activation = 'relu'
ip_conv_0 = Conv2D(filters=32, kernel_size=kernelSize, input_shape=im_shape, activation=ip_activation)
cnn.add(ip_conv_0)
ip_conv_0_1 = Conv2D(filters=64, kernel_size=kernelSize, activation=ip_activation)
cnn.add(ip_conv_0_1)
pool_0 = MaxPool2D(pool_size=(2, 2), strides=(2, 2), padding="same")
cnn.add(pool_0)
drop_layer_0 = Dropout(0.2)
cnn.add(drop_layer_0)
flat_layer_0 = Flatten()
cnn.add(Flatten())
h_dense_0 = Dense(units=128, activation=ip_activation, kernel_initializer='uniform')
cnn.add(h_dense_0)
h_dense_1 = Dense(units=64, activation=ip_activation, kernel_initializer='uniform')
cnn.add(h_dense_1)
op_activation = 'softmax'
output_layer = Dense(units=n_classes, activation=op_activation, kernel_initializer='uniform')
cnn.add(output_layer)#<---------error
TypeError: softmax() got an unexpected keyword argument 'axis'
Keras differ between the layer softmax and the activation function softmax. See here. As the error suggests you are trying to pass a whole layer where a function is needed. The most simple solution would be to just pass a string:
output_layer = Dense(units=500, activation='softmax', kernel_initializer='uniform')
Another possibility would be to pass no activation function to the layer and apply an Activation layer afterwards.
Related
guys what is the difference between activation kwarg and Activation layer in tensorflow?
here's an example :
activation kwarg :
model.add(Dense(64,activation="relu"))
Activation layer :
model.add(Dense(64))
model.add(Activation("sigmoid"))
PS: im new to tensorflow
In Dense(64,activation="relu"), the relu activation function becomes a part of Dense layer, and will be called automatically whenever this Dense layer is called.
In Activation("relu"), the relu activation function is a layer itself and is decoupled from the Dense layer. This is necessary if you want to have a reference to the tensor after Dense but before activation for, says, branching purposes.
input_tensor = Input((10,))
intermediate_tensor = Dense(64)(input_tensor)
branch_1_tensor = Activation('relu')(intermediate_tensor)
branch_2_tensor = Dense(64)(intermediate_tensor)
final_tensor = branch_1_tensor + branch_2_tensor
model = Model(inputs=input_tensor, outputs=final_tensor)
However, your model is Sequential model so your two samples are effectively equal: the relu activation function will be called automatically. To obtain a reference to the tensor before Activation in this case, you can go through model.layers and get the output of the Dense layer from within.
I am trying to produce a CNN using Keras, and wrote the following code:
batch_size = 64
epochs = 20
num_classes = 5
cnn_model = Sequential()
cnn_model.add(Conv2D(32, kernel_size=(3, 3), activation='linear',
input_shape=(380, 380, 1), padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D((2, 2), padding='same'))
cnn_model.add(Conv2D(64, (3, 3), activation='linear', padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
cnn_model.add(Conv2D(128, (3, 3), activation='linear', padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
cnn_model.add(Flatten())
cnn_model.add(Dense(128, activation='linear'))
cnn_model.add(Activation('relu'))
cnn_model.add(Dense(num_classes, activation='softmax'))
cnn_model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adam(), metrics=['accuracy'])
I want to use Keras's LeakyReLU activation layer instead of using Activation('relu'). However, I tried using LeakyReLU(alpha=0.1) in place, but this is an activation layer in Keras, and I get an error about using an activation layer and not an activation function.
How can I use LeakyReLU in this example?
All advanced activations in Keras, including LeakyReLU, are available as layers, and not as activations; therefore, you should use it as such:
from keras.layers import LeakyReLU
# instead of cnn_model.add(Activation('relu'))
# use
cnn_model.add(LeakyReLU(alpha=0.1))
Sometimes you just want a drop-in replacement for a built-in activation layer, and not having to add extra activation layers just for this purpose.
For that, you can use the fact that the activation argument can be a callable object.
lrelu = lambda x: tf.keras.activations.relu(x, alpha=0.1)
model.add(Conv2D(..., activation=lrelu, ...)
Since a Layer is also a callable object, you could also simply use
model.add(Conv2D(..., activation=tf.keras.layers.LeakyReLU(alpha=0.1), ...)
which now works in TF2. This is a better solution as this avoids the need to use a custom_object during loading as #ChristophorusReyhan mentionned.
you can import the function to make the code cleaner and then use it like any other activation.
if you choose not to define alpha, don't forget to add brackets "LeakyReLU()"
from tensorflow.keras.layers import LeakyReLU
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(512, activation=LeakyReLU()))
model.add(tf.keras.layers.Dense(512, activation=LeakyReLU(alpha=0.1)))
I am trying to create a Functional API as opposed to a Sequential API. I have built the model previously using the Sequential API, and it worked just fine. It is an LSTM, and I am having trouble with the batch_size going from the Input to the LSTM layer. The Sequential API was built as follows:
new_model = Sequential()
new_model.add(LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True))
new_model.add(Dropout(0))
new_model.add(LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True))
new_model.add(Dropout(0))
new_model.add(Dense(n_neurons1, activation='tanh'))
new_model.add(Dropout(0.1))
new_model.add(Dense(nm))
new_model.compile(loss='mse', optimizer=optimizer)
The above snippet works fine. The Functional API I am trying to get to work is as follows:
inp = Input(shape = (train_X.shape[1], train_X.shape[2]), batch_size = batch_size)
L1 = LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True)(inp)
D1 = Dropout(0)(L1)
L2 = LSTM(n_neurons, batch_input_shape=(batch_size,train_X.shape[1], train_X.shape[2]), activation='tanh', stateful=True, return_sequences=True)(D1)
D2 = Dropout(0)(L2)
F1 = Dense(n_neurons1, activation='tanh')(D2)
D3 = Dropout(0.1)(F1)
out = Dense(nm)
new_model = Model(inp,out)
new_model.compile(loss='mse', optimizer=optimizer)
I get an error saying "Input() got an unexpected keyword argument 'batch_size", even though I know batch_size is an argument for the Input layer. Then, if I get rid of the argument, I get an error with the first LSTM layer saying:
"If a RNN is stateful, it needs to know its batch size. Specify the batch size of your input tensors:
If using a Sequential model, specify the batch size by passing a batch_input_shape argument to your first layer.
If using the functional API, specify the batch size by passing a batch_shape argument to your Input layer."
I have already tried updating tensorflow but that did not fix the Input() issue. Where do I go from here?
You describe passing a batch_size parameter via the functional API and getting an error suggesting "passing a batch_shape argument to your Input layer."
If you try changing batch_size = batch_size in your input layer to
batch_shape = (batch_size,train_X.shape[1], train_X.shape[2])
does that solve it?
I am building a cnn_rnn network for image classification. I am getting an error while running the following python code in my jupyter notebook.
# model
model1 = Sequential()
# first convolutional layer
model1.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape(160, 120, 3)))
# second convolutional layer
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
#Adding a pooling Layer
model1.add(MaxPooling2D(pool_size=(3, 3)))
#Adding dropouts
model1.add(Dropout(0.25))
# flatten and put a fully connected layer
model1.add(Flatten())
model1.add(Dense(32, activation='relu')) # fully connected
#Adding RNN N/W
model1.add(LSTM(32, return_sequences=True))
model1.add(TimeDistributed(Dense(5, activation='softmax')))
I also tried adding input_shape=(160, 120, 3) as a parameter to the LSTM function but to no avail. Please Help!
P.S: I also tried using GRU instead of LSTM but got the same error.
Update: Please note the model.summary() results
enter image description here
Your error is due to your use of Flatten and Dense BEFORE the LSTM layer.
LSTM layers require the input to be in the shape of (Batchsize x Length x Feature depth0) (or some variant), whereas your flatten changes the Conv2D output from (B x H x W x F) to (B x W * F * H) if that makes sense. If you want to use this architecture, I'd recommend using a Reshape layer to flatten the dimension you want, and using a Conv1D of kernel size 1 (same as a fully-connected layer) before the LSTM layer.
Or, if you want to use this exact code, add this before your LSTM layer and it should work:
model1.add(Reshape(target_shape=(1,54*40,32))
It's 54 and 40 due a pool_size of (3,3).
I a trying to merge 2 sequential models in keras. Here is the code:
model1 = Sequential(layers=[
# input layers and convolutional layers
Conv1D(128, kernel_size=12, strides=4, padding='valid', activation='relu', input_shape=input_shape),
MaxPooling1D(pool_size=6),
Conv1D(256, kernel_size=12, strides=4, padding='valid', activation='relu'),
MaxPooling1D(pool_size=6),
Dropout(.5),
])
model2 = Sequential(layers=[
# input layers and convolutional layers
Conv1D(128, kernel_size=20, strides=5, padding='valid', activation='relu', input_shape=input_shape),
MaxPooling1D(pool_size=5),
Conv1D(256, kernel_size=20, strides=5, padding='valid', activation='relu'),
MaxPooling1D(pool_size=5),
Dropout(.5),
])
model = merge([model1, model2], mode = 'sum')
Flatten(),
Dense(256, activation='relu'),
Dropout(.5),
Dense(128, activation='relu'),
Dropout(.35),
# output layer
Dense(5, activation='softmax')
return model
Here is the error log:
File
"/nics/d/home/dsawant/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py",
line 392, in is_keras_tensor
raise ValueError('Unexpectedly found an instance of type ' + str(type(x)) + '. ' ValueError: Unexpectedly found an instance of
type <class 'keras.models.Sequential'>. Expected a symbolic tensor
instance.
Some more log:
ValueError: Layer merge_1 was called with an input that isn't a
symbolic tensor. Received type: class 'keras.models.Sequential'.
Full input: [keras.models.Sequential object at 0x2b32d518a780,
keras.models.Sequential object at 0x2b32d521ee80]. All inputs to the
layer should be tensors.
How can I merge these 2 Sequential models that use different window sizes and apply functions like 'max', 'sum' etc to them?
Using the functional API brings you all possibilities.
When using the functional API, you need to keep track of inputs and outputs, instead of just defining layers.
You define a layer, then you call the layer with an input tensor to get the output tensor. Models and layers can be called exactly the same way.
For the merge layer, I prefer using other merge layers that are more intuitive, such as Add(), Multiply() and Concatenate() for instance.
from keras.layers import *
mergedOut = Add()([model1.output,model2.output])
#Add() -> creates a merge layer that sums the inputs
#The second parentheses "calls" the layer with the output tensors of the two models
#it will demand that both model1 and model2 have the same output shape
This same idea apply to all the following layers. We keep updating the output tensor giving it to each layer and getting a new output (if we were interested in creating branches, we would use a different var for each output of interest to keep track of them):
mergedOut = Flatten()(mergedOut)
mergedOut = Dense(256, activation='relu')(mergedOut)
mergedOut = Dropout(.5)(mergedOut)
mergedOut = Dense(128, activation='relu')(mergedOut)
mergedOut = Dropout(.35)(mergedOut)
# output layer
mergedOut = Dense(5, activation='softmax')(mergedOut)
Now that we created the "path", it's time to create the Model. Creating the model is just like telling at which input tensors it starts and where it ends:
from keras.models import Model
newModel = Model([model1.input,model2.input], mergedOut)
#use lists if you want more than one input or output
Notice that since this model has two inputs, you have to train it with two different X_training vars in a list:
newModel.fit([X_train_1, X_train_2], Y_train, ....)
Now, suppose you wanted only one input, and both model1 and model2 would take the same input.
The functional API allows that quite easily by creating an input tensor and feeding it to the models (we call the models as if they were layers):
commonInput = Input(input_shape)
out1 = model1(commonInput)
out2 = model2(commonInput)
mergedOut = Add()([out1,out2])
In this case, the Model would consider this input:
oneInputModel = Model(commonInput,mergedOut)