I have seen some examples of transfer learning where one can use pre-trained models from keras.application (Xception, VGG16, VGG19, ResNet50 e.t.c) but what I want is to transfer the learning from the model I saved using model.save('model.h5')
This is my current model:
model = Sequential()
model.add(Embedding(max_words, embedding_dim, input_length=maxlen))
model.add(LSTM(32))
model.add(Dropout(0.6))
model.add(Dense(2, activation='sigmoid'))
model.compile(optimizer='rmsprop', loss='binary_crossentropy',metrics=['acc'])
model.fit(sequences, labels, epochs=10, batch_size=32, validation_split=0.2)
Now, Instead of saying
model_base = keras.applications.vgg16.VGG16(include_top=False, weights='imagenet')
I want to load the saved model probably with load_model('model.h5') and add it as a layer to my current model.
try this
model = Sequential()
model.add(Embedding(max_words, embedding_dim, input_length=maxlen))
model.add(LSTM(32))
model.add(Dropout(0.6))
model.add(Dense(2, activation='sigmoid'))
model.layers[0].set_weights([embedding_matrix])
model.layers[0].trainable = False
model.load_weights('model.h5')
model.compile(optimizer='rmsprop', loss='binary_crossentropy',metrics=['acc'])
model_base = model
Dont forget to remove your classifier
Related
I want to use second order optimizer instead of using SGD, Adam, Adagrad, AdaDelta etc in neural network model. So I found AdaHessian as an optimizer. I have the following code:
# define model
model = Sequential()
model.add(Dense(132, input_dim=66, activation='linear'))
model.add(Dropout(0.5))
model.add(Dense(88, activation='linear'))
model.add(Dropout(0.5))
model.add(Dense(44, activation='linear'))
model.add(Dropout(0.5))
model.add(Dense(22, activation='linear'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='relu'))
model.compile(loss='mse', optimizer=optimizer = Adahessian(params=model), metrics=['accuracy'])
# fit model
model.fit(trainX, trainy, epochs=50, validation_split=0.2,verbose=0)
# evaluate the model
loss, test_acc = model.evaluate(testX, testy, verbose=0)
return model, loss, test_acc
Can anyone help me what params= means in AdaHessian() and what should I do to write i.e., model.parameters() or others? But I got error message:
AttributeError: 'Sequential' object has no attribute 'parameters'
Please show instructions so I can follow to implement second order optimizers in compilation of model.
I build a model in Keras for CNN. I want to save this model and reuse this trained model for transfer learning as a pre-trained model. I do not understand what is the proper way to save the model for reusing it for transfer learning. Again how to load my pre-trained model in Keras so that I can add some layers after load the previous model.
This is my model that I build
model_cnn = Sequential() # initilaizing the Sequential nature for CNN model
# Adding the embedding layer which will take in maximum of 450 words as input and provide a 32 dimensional output of those words which belong in the top_words dictionary
model_cnn.add(Embedding(vocab_size, embedding_size, input_length=max_len))
model_cnn.add(Conv1D(32, 3, padding='same', activation='relu'))
model_cnn.add(Conv1D(64, 3, padding='same', activation='relu'))
model_cnn.add(MaxPooling1D())
model_cnn.add(Flatten())
model_cnn.add(Dense(250, activation='relu'))
model_cnn.add(Dense(2, activation='softmax'))
# optimizer = keras.optimizers.Adam(lr=0.001)
model_cnn.compile(
loss='categorical_crossentropy',
optimizer=sgd,
metrics=['acc',Precision(),Recall(),]
)
history_cnn = model_cnn.fit(
X_train, y_train,
validation_data=(X_val, y_val),
batch_size = 64,
epochs=epochs,
verbose=1
)
The recommended way to save model, is saving with SavedModel format:
dir = "target_directory"
model_cnn.save(dir) # it will save a .pb file with assets and variables folders
Then you can load it:
model_cnn = tf.keras.models.load_model(dir)
Now, you can add some layers and make another model. For example:
input = tf.keras.Input(shape=(128,128,3))
x = model_cnn(input)
x = tf.keras.layers.Dense(1, activation='sigmoid')(x)
new_model = tf.keras.models.Model(inputs=input, outputs=x)
I am not sure where i am wrong in this code. My goal is to train my dataset for binary classification using LSTM and GRU.
[the output comes with module wrapper and GRU not executing please check the image][1]
#BUILD THE MODEL
top_words = 10000
embedding_vecor_length = 32
model = Sequential()
model.add(Embedding(top_words, embedding_vecor_length, input_length=X.shape[1]))
#model.add(Dropout(0.2))
model.add(GRU(100,dropout=0.2, recurrent_dropout=0.2, return_sequences=True))
model.add(LSTM(100,dropout=0.2, recurrent_dropout=0.2))
#model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='Adam',
metrics=['accuracy'])
print(model.summary())
model.summary()
```
[1]: https://i.stack.imgur.com/14pyl.jpg
I trained and saved a Bidirectional LSTM model in Keras successfully with:
model = Sequential()
model.add(Bidirectional(LSTM(N_HIDDEN_NEURONS,
return_sequences=True,
activation="tanh",
input_shape=(SEGMENT_TIME_SIZE, N_FEATURES))))
model.add(Bidirectional(LSTM(N_HIDDEN_NEURONS)))
model.add(Dropout(0.5))
model.add(Dense(N_CLASSES, activation='sigmoid'))
model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train,
batch_size=BATCH_SIZE,
epochs=N_EPOCHS,
validation_data=[X_test, y_test])
model.save('model_keras/model.h5')
However, when I want to load it with:
model = load_model('model_keras/model.h5')
I get an error:
ValueError: You are trying to load a weight file containing 3 layers
into a model with 0 layers.
I also tried different methods like saving and loading model architecture and weights separately but none of them worked for me. Also, previously, when I was using normal (unidirectional) LSTMs, loading the model worked fine.
As mentioned by #mpariente and #today, the input_shape is an argument of Bidirectional, not LSTM, see Keras documentation. My solution:
# Model
model = Sequential()
model.add(Bidirectional(LSTM(N_HIDDEN_NEURONS,
return_sequences=True,
activation="tanh"),
input_shape=(SEGMENT_TIME_SIZE, N_FEATURES)))
model.add(Bidirectional(LSTM(N_HIDDEN_NEURONS)))
model.add(Dropout(0.5))
model.add(Dense(N_CLASSES, activation='sigmoid'))
model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train,
batch_size=BATCH_SIZE,
epochs=N_EPOCHS,
validation_data=[X_test, y_test])
model.save('model_keras/model.h5')
and then, to load, simply do:
model = load_model('model_keras/model.h5')
Assuming I fit the following neural network for a binary classification problem:
model = Sequential()
model.add(Dense(21, input_dim=19, init='uniform', activation='relu'))
model.add(Dense(80, init='uniform', activation='relu'))
model.add(Dense(80, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(x2, training_target, nb_epoch=10, batch_size=32, verbose=0,validation_split=0.1, shuffle=True,callbacks=[hist])
How would I boost the neural network using AdaBoost? Does keras have any commands for this?
This can be done as follows:
First create a model (for reproducibility make it as a function):
def simple_model():
# create model
model = Sequential()
model.add(Dense(25, input_dim=x_train.shape[1], kernel_initializer='normal', activation='relu'))
model.add(Dropout(0.2, input_shape=(x_train.shape[1],)))
model.add(Dense(10, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
Then put it inside the sklearn wrapper:
ann_estimator = KerasRegressor(build_fn= simple_model, epochs=100, batch_size=10, verbose=0)
Then and finally boost it:
boosted_ann = AdaBoostRegressor(base_estimator= ann_estimator)
boosted_ann.fit(rescaledX, y_train.values.ravel())# scale your training data
boosted_ann.predict(rescaledX_Test)
Keras itself does not implement adaboost. However, Keras models are compatible with scikit-learn, so you probably can use AdaBoostClassifier from there: link. Use your model as the base_estimator after you compile it, and fit the AdaBoostClassifier instance instead of model.
This way, however, you will not be able to use the arguments you pass to fit, such as number of epochs or batch_size, so the defaults will be used. If the defaults are not good enough, you might need to build your own class that implements the scikit-learn interface on top of your model and passes proper arguments to fit.
Apparently, neural networks are not compatible with the sklearn Adaboost, see https://github.com/scikit-learn/scikit-learn/issues/1752