Using Keras, I am trying to loop a training session 10 times with different splits of data. Though, after every loop my accuracy increases a lot probably because it doesn't reset and sees new data in new groups (data trained on might appear in test next loop)
I expected model.fit to reset it over, as pr. an answer here saying it does so but I can't get it to. I then tried K.clear_session() in the start of the loop, as pr. example 1 here, but it does nothing. I can save an untrained model the first time and reload it at start of loop, but this seems like a bad method/bad practice. What can I do instead/am I doing wrong?
from tensorflow.keras import backend as K
for i in range(0, 10):
print("Starting loop " + str(i))
K.clear_session()
model = keras.Model(inputs=inputs, outputs=outputs, name="SchoolProject")
model.compile(loss=tensorflow.keras.losses.binary_crossentropy, optimizer=tensorflow.keras.optimizers.Adam(lr=hpInitialLearningRate), metrics=['accuracy'])
trainData, valData, testData, trainTruth, valTruth, testTruth = getTrainValAndTestSet()
model.fit(trainData, trainTruth, epochs=hpEpochs, verbose=1, callbacks=callbacks_list, validation_data=(valData, valTruth))
score = model.evaluate(testData, testTruth, verbose=1)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
testAccList.append(score[1])
print("Ending loop " + str(i))
The easiest way would be to define your model inside the loop. Here's an example. You'll see that every iteration, the accuracy starts at random before improving.
import tensorflow as tf
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255
for i in range(5):
model = tf.keras.Sequential([
tf.keras.Input(shape=(28, 28)),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(32, activation='relu'),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dense(10, activation="softmax"),
])
model.compile(loss="sparse_categorical_crossentropy", optimizer="adam",
metrics=["accuracy"])
model.fit(x_train, y_train, batch_size=16, epochs=1, validation_split=0.1)
Resetting the weights manually is a little more complicated.
Related
I tried with this one but i think its not really good working so any idea ?
the code:
import tensorflow as tf
import numpy as np
(x_train, y_train), (x_test, y_test) = load_data_MEB()
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=10, validation_data=(x_test, y_test))
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', test_acc)
The project is the recognition of nanocrystals (present in cement) on SEM.
We have a database of 1000 SEM pictures where we can see the crystals, we need a program in python or any other programming language (CNN but ideally FFNN) to recognize these crystals and eventually their shape (square and triangular).
There is a lot already out there about saving models, but I'm struggling to work out how I can save my model only when it improves upon val_accuracy. My model looks like this:
model = keras.Sequential([
keras.layers.Embedding(numberOfWords,
embedding_vector_length, input_length=1000),
keras.layers.LSTM(128),
keras.layers.Dropout(0.3),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dropout(0.3),
keras.layers.Dense(64, activation='relu'),
keras.layers.Dropout(0.3),
keras.layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5), loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=200, batch_size=32,
validation_data=(x_test, y_test))
During training, I want to save the model after the first epoch. Then, after every epoch, if val_accuracy has been improved upon, I want to overwrite the old model with the new one.
How do I do this?
you just have to define a Callback-List and enter it into the model.fit declaration: Keras_fit In this example its just safing the best weigths, so its actually overwriting the old ones, and safes it into an hdf5 format. Hope that solved your problem :)
from keras.callbacks import ModelCheckpoint
filepath="weights.best.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1,save_best_only=True, mode='max')
callbacks_list = [checkpoint]
model.fit(x_train, y_train, epochs=200,,callbacks=callbacks_list, batch_size=32,
validation_data=(x_test, y_test))
I hope you are having a great day!
I recently tried to train a regression model by using TensorFlow and I completed my code by following the instruction in here.
data = pd.read_csv('regret.csv')
max_regret = data['regret'].max()
data['regret'] = data['regret'] / max_regret # Normalize Regrets
regret_labels = data.pop('regret')
def build_model():
model = keras.Sequential([
layers.Dense(64, activation='relu', input_shape=[len(data.keys())]),
layers.Dense(64, activation='relu'),
layers.Dense(1)
])
optimizer = tf.keras.optimizers.RMSprop(0.001)
model.compile(loss='mse',
optimizer=optimizer,
metrics=['mae', 'mse'])
return model
model = build_model()
class PrintDot(keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs):
if epoch % 100 == 0: print('')
print('.', end='')
EPOCHS = 1000
# early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=100)
history = model.fit(
data, regret_labels,
epochs=EPOCHS, validation_split=0.2, verbose=0,
callbacks=[PrintDot()])
test = model.predict(data)
loss, mae, mse = model.evaluate(data, regret_labels, verbose=2)
However, I encountered a problem that all the predictions were the same, even though the model.evaluate() gave me different statistics by trials.
I also attached the file via this link.
Would you take a look at it and give me some ideas to solve it? Thanks in advance!
You can try this approach as below which split your data set into training and test set before fitting into the model.
You can try this approach as below which split your data set into training and test set before fitting into the model.
import pandas as pd
import keras
from keras.models import Sequential
from keras.layers.core import Dense
from keras.optimizers import RMSprop
data = pd.read_csv('regret.csv')
max_regret = data['regret'].max()
data['regret'] = data['regret'] / max_regret
len(data.keys())
data
train_dataset = data.sample(frac=0.8,random_state=0)
test_dataset = data.drop(train_dataset.index)
train_labels = train_dataset.pop('regret')
test_labels = test_dataset.pop('regret')
def build_model():
model = Sequential()
model.add(Dense(64, activation='relu', input_shape=(27,)))
model.add(Dense(64, activation='relu'))
model.add(Dense(1))
optimizer = RMSprop(0.001)
model.compile(loss='mse',
optimizer=optimizer,
metrics=['mae', 'mse'])
return model
model = build_model()
class PrintDot(keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs):
if epoch % 100 == 0: print('')
print('.', end='')
EPOCHS = 1000
early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=100)
history = model.fit(
train_dataset, train_labels,
epochs=EPOCHS, validation_split=0.2, verbose=0,
callbacks=[PrintDot()])
test = model.predict(test_dataset)
The result have slightly changes as below:
enter image description here
What you could do better is min max scaling all the attributes.
Hope it can help you.
I'm trying to find a way to visualize which numbers in the mnist dataset a model was able to correctly identify and which ones it wasn't.
What I can't seem to find is if such a visualization is possible in tensorboard or if I would need to use/create something else to achieve it.
I'm currently working from the basic tutorial provided for tensorflow 2.0 with tensorboard added.
import datetime
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit(x_train,
y_train,
epochs=5,
validation_data=(x_test, y_test),
callbacks=[tensorboard_callback])
model.evaluate(x_test, y_test)
It appears the what-if tool is what I was looking for, it allows you to visually sort testing data depending on whether it was correctly or incorrectly identified by the model.
If you want to test it out here is their demo that I used to get the above image and they have multiple other demos on the tools site.
I have a simple loop which in every iteration creates an LSTM (with the same parameters) and fit it to the same data.
The problem is that it takes more and more time during itterations.
batch_size = 10
optimizer = optimizers.adam(lr=0.001)
number_unites = 20
for counter_temp in range(1,100):
t0 = time.time()
model = None
model = Sequential()
model.add(LSTM(units=number_unites, batch_input_shape=(None, batch_size, 1), return_sequences=False))
model.add(Dense(1))
model.compile(loss='mean_absolute_error', optimizer=optimizer, metrics=['accuracy'])
model.fit(x_train, y_train, epochs=50, validation_data=(x_train_val, y_train_val), verbose=0)
print(time.time()-t0)
Thanks to DanielMöller, it is solved.
I used keras.backend.clear_session() to remove "Keras's graph".
from keras.backend import clear_session
clear_session()