I want to train several times a file ConvNet.py in order to produce some statistics about its training like precision, confussion matrices, etc. So, I tried (in google colab) to do something like
for k in range(10:
%run ConvNet.py
The first training goes well, but when in begin the second, arise a problem. It says that "weights variable already defined, disallowed" (weights is the fist variable that I define in ConvNet.py) and the script stops.
I tried clearing variables with os kill, but there is still problems. How can I fix this?
It's probably better if you do the iterated training in Python directly.
You haven't shared much about your setup, but you can do something like:
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
def build_model():
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=(28,28,)))
model.add(tf.keras.layers.Dense(32, activation="relu"))
model.add(tf.keras.layers.Dense(10, activation="sigmoid"))
model.compile(optimizer=tf.train.AdamOptimizer(),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
return model
for i in range(10):
model = build_model()
model.fit(x_train, y_train)
model.save_weights(f"./weights-{i}.hdf5")
And when you want to do the analysis:
for i in range(10):
model = build_model()
model.load_weights(f"./weights-{i}.hdf5")
do_analysis(model)
Related
I want to save a trained Keras model, so that it can be used in the Django REST backend of an application. I did a lot of research, but it seems there isn't any way to use these models without TensorFlow installed.
So, what is the use of this storage? I don't want to install a heavy library like TensorFlow on the server. I tested saving with pickle and joblib, as well as Keras' own model.save().
Is there a way to load this model without installing TensorFlow and only with Keras itself?
This is a part of my code,
from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout
xtrain, ytrain = np.array(xtrain), np.array(ytrain)
ytrain = np.reshape(ytrain, (ytrain.shape[0], 1, 1))
model = Sequential()
model.add(LSTM(150, return_sequences=True, input_shape=(xtrain.shape[1], 1)))
model.add(LSTM(150, return_sequences=False))
model.add(Dense(25))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(xtrain, ytrain, batch_size=1, epochs=7)
model.save('model.h5')
which normally works perfectly, but if I use the model elsewhere, I get this error:
ModuleNotFoundError: No module named 'tensorflow'
You do not need to use TensorFlow in production. You can use coefficient by replacing what random functions in your programming language.
Sample: Input array, time coefficients matrixes, and unboxed system inputs to output with feedback system in the box containers.
temp = tf.random.normal([10], 1, 0.2, tf.float32)
temp = np.asarray(temp) * np.asarray([ coefficient_0, coefficient_1, coefficient_2, coefficient_3, coefficient_4, coefficient_5, coefficient_6, coefficient_7, coefficient_8, coefficient_9 ]) #action = actions['up']
temp = tf.nn.softmax(temp)
action = int(np.argmin(temp))
I am currently attempting to used keras tuner to create a model for my CNN, though I am having some issues with saving my model for future use.
As I am used to it, I could regularly just save my model with model.save(filename) to receive a .model file; however, when attempting this with code such as:
tuner = RandomSearch(
build_model,
objective = "val_accuracy",
max_trials = 5,
executions_per_trial = 1,
directory = LOG_DIR
)
tuner.search(x= x_train, y= y_train, epochs= 1, batch_size=64, validation_data= (x_test, y_test))
bestModels = tuner.get_best_models(num_models=1)
highestScoreModel= models[0]
highestScoreModel.fit(x=x_train, y=y_train, batch_size=64, epochs=5, verbose=1, validation_split=0.2)
highestScoreModel.save("Trained_Model")
I received a Trained_Model folder with no model within it, only the parameters. If anyone could assist me with saving the actual trained model I'd be very thankful.
================= Edit / Update ================
I have now found a way to obtain the trial_id by sifting the generated trial files. Though, when I run:
trial_id = getID()
tuner.save_model(trial_id=trial_id, model=highestScoreModel, step=0)
Nothing seems to happen, no save file appears. Again, I would be grateful for any help on this matter.
Try this
Tuner.save_model(trial_id, model, step=0)
where
trial_id is your model trial number
model is your trained model
and step is epochs numbers
I am using CUDA/CUDNN to train multiple tensorflow keras models on my GPU (for an evolutionary algorithm attempting to optimize hyperparameters). Initially, the program would crash with an Out of Memory error after a couple generations. Eventually, I found that using a new sub-process for every model would clear the GPU memory automatically.
However, each process seems to reinitialize CUDA (loading dynamic libraries from the .dll files), which is incredibly time-consuming. Is there any method to avoid this?
Code is pasted below. The function "fitness_wrapper" is called for each individual.
def fitness_wrapper(indiv):
fit = multi.processing.Value('d', 0.0)
if __name__ == '__main__':
process = multiprocessing.Process(target=fitness, args=(indiv, fit))
process.start()
process.join()
return (fit.value,)
def fitness(indiv, fit):
model = tf.keras.Sequential.from_config(indiv['architecture'])
optimizer_dict = indiv['optimizer']
opt = tf.keras.optimizers.Adam(learning_rate=optimizer_dict['lr'], beta_1=optimizer_dict['b1'],
beta_2=optimizer_dict['b2'],
epsilon=optimizer_dict['epsilon'])
model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])
model.fit(data_split[0], data_split[2], batch_size=32, epochs=5)
fit = model.evaluate(data_split[1], data_split[3])[1]
Turns out the solution is to just use tensorflow.backend.clear_session() after each model (rather than the subprocesses). I had tried this before and it didn't work, but for some reason this time it fixed everything.
You should also delete the model and call reset_default_graph(), apparently.
def fitness(indiv, fit):
model = tf.keras.Sequential.from_config(indiv['architecture'])
optimizer_dict = indiv['optimizer']
opt = tf.keras.optimizers.Adam(learning_rate=optimizer_dict['lr'], beta_1=optimizer_dict['b1'],
beta_2=optimizer_dict['b2'],
epsilon=optimizer_dict['epsilon'])
model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])
model.fit(data_split[0], data_split[2], batch_size=32, epochs=5)
fit = model.evaluate(data_split[1], data_split[3])[1]
del model
tf.keras.backend.clear_session()
tf.compat.v1.reset_default_graph()
return fit
I am making the binary sound classification model by Keras on Python3.7. I have been make the sound classification model on MATLAB however some specifically layer is not installed on MATLAB (ex. GRU). So I try to convert to Keras deep learning model from MATLAB deep learning model.
The original MATLAB code is shown bellow:
inputsize=[31,69]
layers = [ ...
sequenceInputLayer(inputsize(1))
bilstmLayer(200,'OutputMode','last')
fullyConnectedLayer(2)
softmaxLayer
classificationLayer
]
options = trainingOptions('adam', ...
'MaxEpochs',30, ...
'MiniBatchSize', 200, ...
'InitialLearnRate', 0.01, ...
'GradientThreshold', 1, ...
'ExecutionEnvironment',"auto",...
'plots','training-progress', ...
'Verbose',false);
This model get to the accuracy is 0.955.
The Keras code based on MATLAB code is shown below:
# traindatasize=(86400,31,69)
inputsize=(31,69)
batchsize=200
epochs=30
model = Sequential()
model.add(Bidirectional(LSTM(200, input_shape=inputsize)))
model.add(Dense(2, activation='softmax'))
model.compile(optimizer=RMSprop(), loss='binary_crossentropy', metrics=['accuracy'])
model.fit(traindata, trainlabel, batch_size=batchsize, epochs=epochs, verbose=1)
This model get to the accuracy is 0.444
I don't understand what is the effect.
The traindata used same data from STFT and normalize before train those model using standard deviation and mean average.
Please some comments.
Python 3.7 on Anaconda
Keras 2.2.4
I think that's because the MATLAB code uses the Adam optimizer for training, and you defined RMSprop instead in:
model.compile(optimizer=RMSprop(),loss='binary_crossentropy',metrics=['accuracy'])
instead, use:
from keras import optimizers
adam = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, amsgrad=False)
...
model.compile(optimizer=adam,loss='binary_crossentropy',metrics=['accuracy'])
check if this improves the answer.
I'm working with the code from the Tensorflow beginner mnist tutorial (code below).
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)
Since the tutorial stops after evaluating the model, I have added a few lines to make actual predictions.
predictions = model.predict([x_test])
print(np.argmax(predictions[0]))
So far it all works. However I would like to see the image the model tries to predict to know what's going on. I therefore tried the following:
import matplotlib.pyplot as plt
plt.imshow(x_test[0])
plt.show()
This does not work. I don't get an error message, it just won't show the picture. Furthermore if I try to add a simple print-statement after the matplotlib code that is ignored too and won't be printed. Any idea what's going on?
You report that .show() does not work properly with your current config. Rather than call that you may prefer this:
plt.savefig('plot.png')
You can use that with a headless driver if you want:
import matplotlib
matplotlib.use('Agg')