I just started learning NN recently on python with keras, I've a pretty obvious question that nobody seems to ever mention its answer.
the question is very simple.
what happen after you get the data, build the model and train your network ?
every tutorial go through this thoroughly, but never mention how to use your trained model or store it after that.
so for example I written this simple code with keras to train a network on MNIST :
model = Sequential()
model.add(Convolution2D(32, kernel_size=3,data_format="channels_first",
activation='relu', input_shape=(1,28,28)))
model.add(Convolution2D(32, (3 ,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
#compiling
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
#fitting and training
model.fit(X_train, Y_train,batch_size=32, epochs=1, verbose=1)
now how do i store the final network and reuse it again after i close the the the editor ?
for example if i wanna build a simple web interface to upload a MNIST pic and run it through the pre-trained model and detect the answer.
How can i store the trained Model with Python , access it with JS or php, run the uploaded picture through it , and return the output back to the user.
thanks, and sorry if my question seems stupid or obvious.
This is an example how you can save your neural network in keras in json and h5:
# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
and this is how you can load it again:
# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")
print("Loaded model from disk")
Finally you can evaluate the loaded model on new test data:
# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" % (loaded_model.metrics_names[1], score[1]*100))
Related
I have a multiclass(108 classes) classification model which I want to apply transfer learning to the classification layer. I want to deploy this model in a low computing resource device (Raspberry Pi) and I thought to implement the classification layer in pure numpy instead of using Keras or TF.
Below is my original model.
from tensorflow.keras.models import Sequential, Model, LSTM, Embedding
model = Sequential()
model.add(Embedding(108, 50, input_length=10))
model.add((LSTM(32, return_sequences=False)))
model.add(Dense(108, activation="softmax"))
model.compile(loss="categorical_crossentropy", optimizer=Adam(lr=0.001), metrics=['accuracy'])
model.summary()
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)
history = model.fit(X_train, y_train, epochs=10, batch_size=32, validation_split=0.5, callbacks=[es]).history
I split this model into two parts, encoder and decoder as follows. decoder is the classification layer which I want to convert into NumPy model and then do the on-device transfer learning later.
encoder = Sequential([
Embedding(108, 50, input_length=10),
GRU(32, return_sequences=False)
])
decoder = Sequential([
Dense(108, activation="softmax")
])
model = Model(inputs=encoder.input, outputs=decoder(encoder.output))
model.compile(loss="categorical_crossentropy", optimizer=Adam(lr=0.001), metrics=['accuracy'])
model.summary()
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.5, callbacks=[es]).history
I have a few questions related to this approach.
Only way i know to train this model is, first to train the encoder and decoder. then
train NumPy classification layer using trained encoder outputs.
Is there any way i can train the NumPy model at the same time when i train the encoder (without using the above Keras decoder part and Model)? I can't use Model as I can't use Keras or TF in raspberry Pi during the transfer learning.
If there is no any way to train encoder and Numpy model at the same time,
How to use learned decoder weights as the starting weights of the Numpy Model instead of starting from random weights?
What is the most efficient code (or way) to implement the Numpy classification layer (decoder)? It requires a highly efficient model as i do the transfer learning on Raspberry Pi for incoming streaming data.
Once i trained the model for reasonable data, i plan to convert the encoder into TFLite and do the inference
Highly appreciate any help or guidance to achieve this as I'm new to NumPy-based NN implementations.
Thanks in advance
I have been trying to save the weights of my neural network model so that I could use a few of its layers for another neural network model to be trained on another dataset.
pre-trained model:
model = Sequential()
model.add(tf.keras.layers.Dense(100, input_shape=(X_train_orig_sm.shape)))
model.add(tf.keras.layers.Activation('relu'))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Dense(10))
model.add(tf.keras.layers.Activation('relu'))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Dense(1))
model.add(tf.keras.layers.Activation('sigmoid'))
model.summary()
# need sparse otherwise shape is wrong. check why
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print('Fitting the data to the model')
batch_size = 20
epochs = 10
history = model.fit(X_train_orig_sm, Y_train_orig_sm, batch_size=batch_size, epochs=epochs, verbose=1, validation_split=0.2)
print('Evaluating the test data on the model')
How I saved the weights of neural network:
model.save_weights("dnn_model.h5")
How I try to load the weights of neural network:
dnn_model=model.load_weights("dnn_model.h5")
dnn_model.layers[5]
While trying to load the model, I get the following error:
AttributeError: 'NoneType' object has no attribute 'layers'
I dont seem to understand why the layers of the neural network are not recognised even though the pre-trained neural network is trained before the model was saved. Any advice, solution or direction will be highly appreciated. Thank you.
When you call model.save_weights("dnn_model.h5"), you only save the "weights" of the model. You do not save the actual structure of the model. That's why you cannot access the layers etc.
To save the actual model, you can call the below.
# save
model.save('dnn_model') # save as pb
model.save('dnn_model.h5') # save as HDF5
# load
dnn_model = tf.keras.models.load_model('dnn_model') # load as pb
dnn_model = tf.keras.models.load_model('dnn_model.h5') # load as HDF5
Note: You do not need to add an extension to the name to save as pb.
Source: https://www.tensorflow.org/tutorials/keras/save_and_load
I built a model for text summarization, I created a small document (text file) with its summary, then I trained the model on that, I created again the same type of document for test, the training and test documents are pretty similar but with different data.
For example, the training document contains:
name : train
family name : train
The test document:
name : test
family name : test
I was hoping that after training the model, it will remember the structure of the important sentences, after testing I've got an accuracy of 100%.
The problem is when I train the model on another document, the previous test gives lower accuracy, it's like it forgets the previous training.
here is my model:
model = Sequential()
model.add(Embedding(200,64, input_length=max_sent_length))
model.add(Conv1D(filters=64, kernel_size=3, padding='same', activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
for i in range(0,len(xtrains)):
model.fit(xtrains[i],ytrains[i], epochs=200, batch_size=64, shuffle=False)
I've searched about that and the answers I got is that refitting the model doesn't reset weights, so I was wondering why whenever I train the model on new documents I get lower accuracy for the previous tests, whereas at the beginning I received an accuracy of 100%.
How can I solve this problem?
I've created keras model to recognize human activity, based on data from mobile accelerometer:
model = Sequential()
model.add(Reshape((const.PERIOD, const.N_FEATURES), input_shape=(240,)))
model.add(Conv1D(100, 10, activation='relu', input_shape=(const.PERIOD, const.N_FEATURES)))
model.add(Conv1D(100, 10, activation='relu'))
model.add(MaxPooling1D(const.N_FEATURES))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(7, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
I've tested model, and the accuracy after ten epochs is like 85-90%. I don't know, but when I converse my model to TF Lite and I run interpreter in my android app, there's horrible predictions. What can be reason of that bad results? No compatibility on keras -> tensorflow -> tensorflow lite line? Should I run it with another way, using something like servlet + keras model?
A few suggestions:
Try to visualize your tflite graph with
https://lutzroeder.github.io/netron/. See if there's anything
unexpected.
Try to debug with tensorflow lite's python API first. Feed the same
input to the keras model and tflite model and compare the output
tensor.
The question:
With a keras model (partly) specified such as this:
# create model
model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=10)
Is it in any way possible to save all details in the model for later use?
The details:
I've been following an example from machinelearningmastery.com and trying to modify and add traits / arguments of the model such as
activation='relu'
activation='sigmoid'
metrics=['accuracy']
And as the question suggests, I'd like to store model settings for later use.
I understand that these arguments are parts of different functions, but shouldn't it be possible all the same?
What I've tried:
1. model.save() and model.load()
Only returns
AttributeError: 'Sequential' object has no attribute 'load'
2. model.get_config()
Here I've been able to find some of the settings such as:
[{'class_name': 'Dense', 'config': {'activation': 'relu',
But I haven't found a way to load that config as a standalone model, and more often than not, I can't seem to find all settings.
3. I've also checked other posts such as Keras - Reuse weights from a previous layer - converting to keras tensor, but all aspects of the model don't seem to be covered.
Any suggestions?
Instead of trying model.load() try using load_model() provided by keras to load the model you saved using model.save()
from keras.models import load_model
load_model(filepath)
Also you can save the model as json using model.to_json() and load from json using model_from_json()
You can see more ways to save and load a model in Keras Documentation
here
model.save() will do the trick to save the model, to load it use from keras.models import load_model and use model=load_model(model_name) to load the model