My code is
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 5)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(2)])
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(X_train, train_labels, epochs=10)
And my output is
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3920) 0
dense (Dense) (None, 128) 501888
dense_1 (Dense) (None, 2) 258
=================================================================
Total params: 502,146
Trainable params: 502,146
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
219/219 [==============================] - 2s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 2/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 3/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 4/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 5/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 6/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 7/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 8/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 9/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
Epoch 10/10
219/219 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0000e+00
<keras.callbacks.History at 0x7f8750280790>
Why do all training accuracy converge to 0? My dataset is
print(X_train.shape)
print(X_test.shape)
(7000, 28, 28, 5)
(3000, 28, 28, 5)
print(train_labels.shape)
(7000, 1)
And I tried other models, including con2D model or logistic regression model, but accuracy is always 0. That's really weird. Does the issue come from my dataset? My train_labels only contains 1s and (-1)s.
Try adjusting the learning rates or the label is not suitable, because the loss function value return is NaN.
First you need to consider the label as an int or float format.
Look into the sample dataset label distributions and vary, have 1 or 2 (1, 2) as your networks required.
If from_logits is enabled, you need to compare the output of the networks with the label and logits shape return. Example (1, 2) with 2 number of the output layer.
The Flatten layer is working in old versions. You should use the Input layer that is suitable with a dataset or you rename it as 'flatten_input' as your input layer name.
The rest is about data and networks suitable tasks, contrast input, and target. Try to add more layers or image alignment to create a contrast of data, but if the data is not an image, but a screen or shared information in resizes scales, networks should match to data.
Sample: Working with the Flatten layer, you need to map of the input name to it.
dataset = {
"flatten_input" :[],
"label" : []
}
dataset["flatten_input"].append(tf.constant(image, shape=(1, 28, 28, 1)))
dataset["label"].append(tf.constant(label, shape=(1, 1, 1, 64)))
Sample: Simply working on the MNIST dataset
import tensorflow as tf
import tensorflow_datasets as tfds
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: DataSets
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
ds = tfds.load('mnist', split='train', shuffle_files=True)
ds = ds.shuffle(1024).batch(64).prefetch(tf.data.experimental.AUTOTUNE)
assert isinstance(ds, tf.data.Dataset)
for example in ds.take(1):
image, label = example["image"], example["label"]
ls_image = []
ls_label = []
for i in range(label.shape[0]):
ls_image.append(tf.constant(image[i], shape=(1, 28, 28, 1)).numpy())
### should reflects the label in number format ###
ls_label.append(tf.constant(0, shape=(1, 1, 1, 1)).numpy())
image = tf.constant( ls_image, shape=(64, 1, 784, 1) )
label = tf.constant( ls_label, shape=(64, 1, 1, 1) )
dataset = tf.data.Dataset.from_tensor_slices(( image, label ))
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Model Initialize
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
model = tf.keras.Sequential([
tf.keras.layers.InputLayer(input_shape=(784, 1)),
tf.keras.layers.Dense(256),
tf.keras.layers.Dense(256),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(2)
])
model.summary()
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Optimizer
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
optimizer = tf.keras.optimizers.Nadam(
learning_rate=0.01, beta_1=0.9, beta_2=0.999, epsilon=1e-07,
name='Nadam'
)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Loss Fn
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
lossfn = tf.keras.losses.SparseCategoricalCrossentropy(
from_logits=True,
reduction=tf.keras.losses.Reduction.AUTO,
name='sparse_categorical_crossentropy'
)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Model Summary
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
model.compile(optimizer=optimizer, loss=lossfn, metrics=['accuracy'])
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Training
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
history = model.fit( dataset, epochs=50 )
Output: Numbers happen
Epoch 1/50
51/64 [======================>.......] - ETA: 0s - loss: 7.0123e-09 - accuracy: 1.0000
Related
I am building a custom Keras layer that it is essentially the softmax function with a base parameter which is trainable. While the layer works on its own, when placed inside a sequential model, model.summary() determines its output shape as None and model.fit() raises a presumably linked exception:
ValueError: as_list() is not defined on an unknown TensorShape.
In other custom layers (including obviously the Linear example from keras) the output shape can be determined after .build() is called. By looking at model.summary()'s source code, as well as keras.layers.Layer, there is this #property Layer.output_shape that fails to automatically determine the output shape.
Then I tried overwriting the property and manually returning the input_shape argument passed to my layer's .build() method after saving it (softmax does not change the shape of the input), but this didn't work either: If i make a call to super().output_shape before returning my value, model.summary() determines the shape as ?, while if I don't, the value may be shown seemingly correct, but in both cases, I get the exact same error during .fit().
Is there something special about the code iside call() that prevents keras from understanding the shape of the output?
Alteratively, is there a piece of documentation I have missed?
My layer:
class B_Softmax(keras.layers.Layer):
def __init__(self, b_init_mean=10, b_init_var=0.001):
super(B_Softmax, self).__init__()
self.b_init = tf.random_normal_initializer(b_init_mean, b_init_var)
self._out_shape = None
def build(self, input_shape):
self.b = tf.Variable(
initial_value = self.b_init(shape=(1,), dtype='float32'),
trainable=True
)
self._out_shape = input_shape
def call(self, inputs):
# This is an implementation of Softmax for batched inputs
# where the factor b is added to the exponents
nominators = tf.math.exp(self.b * inputs)
denominator = tf.reduce_sum(nominators, axis=1)
denominator = tf.squeeze(denominator)
denominator = tf.expand_dims(denominator, -1)
s = tf.divide(nominators, denominator)
return s
#property
def output_shape(self): # If I comment out this function, summary prints 'None'
self.output_shape # If I leave this line, summary prints '?'
return self._out_shape # If the above line is commented out, summary prints '10' (correctly)
# but the same error is triggered in all three cases
The layer works on its own:
>>> A = tf.constant([[1,2,3], [7,5,6]], dtype="float32")
>>> layer = B_Softmax(1.0)
>>> layer(A)
<tf.Tensor: shape=(2, 3), dtype=float32, numpy=
array([[0.08991686, 0.24461554, 0.6654676 ],
[0.6654677 , 0.08991687, 0.24461551]], dtype=float32)>
But when I try to include it inside a model, the summary doesn't look right:
input_dim = 5
model = keras.Sequential([
Dense(32, activation='relu', input_shape=(input_dim,)),
Dense(num_classes, activation="softmax"),
B_Softmax(1.0)
])
model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_10 (Dense) (None, 32) 192
dense_11 (Dense) (None, 10) 330
b__softmax_18 (B_Softmax) None <-------------------1-------- "None", "?", or "10" (in a hacky way) may be printted
=================================================================
Total params: 523
Trainable params: 523
Non-trainable params: 0
And training fails:
batch_size = 128
epochs = 15
model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1040, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1030, in run_step **
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 894, in train_step
return self.compute_metrics(x, y, y_pred, sample_weight)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 987, in compute_metrics
self.compiled_metrics.update_state(y, y_pred, sample_weight)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 480, in update_state
self.build(y_pred, y_true)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 398, in build
y_pred)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 526, in _get_metric_objects
return [self._get_metric_object(m, y_t, y_p) for m in metrics]
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 526, in <listcomp>
return [self._get_metric_object(m, y_t, y_p) for m in metrics]
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 548, in _get_metric_object
y_p_rank = len(y_p.shape.as_list())
ValueError: as_list() is not defined on an unknown TensorShape.
This doesn't directly solves the issue, rather side-stepping it: Instead of using squeeze and expand_dims, the former of which seems to be problematic for Tensorflow to keep track of, we use keepdims=True in the summation to keep the axes aligned correctly for the softmax denominator.
def call(self, inputs):
# This is an implementation of Softmax for batched inputs
# where the factor b is added to the exponents
nominators = tf.math.exp(self.b * inputs)
denominator = tf.reduce_sum(nominators, axis=1, keepdims=True)
s = tf.divide(nominators, denominator)
return s
Arguably, it would be much preferable to make use of the built-in softmax:
def call(self, inputs):
return tf.nn.softmax(self.b * inputs)
You could implement the compute_output_shape method in your Layer subclass:
def compute_output_shape(self, input_shape):
return [(None, out_shape)]
Where out_shape contains the dimensionality of the output, or you can replace the whole tuple to have any output shape you want.
I found no problem with the codes, I think input parameters are important there are some remarks done this way:
I use the dataset and the model.fit() I also use it without splits that is because I create a sample with one record.
Model input, I modified from Input( 5, ) to Input ( 1, 5 ) that matches the dataset creates a shape ( that is why I added choice number 1 into the summary )
Categorize, BatchSize, Number of class, LossFN, and Optimizers are adjusted by the output dimensions of the networks by number_classes parameters.
Sample: << Loss FN does not decisions the model but Input and Output does, not necessary tell how it created removed what it does not use or comment it >>
import tensorflow as tf
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Class / Definition
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
class B_Softmax(tf.keras.layers.Layer):
def __init__(self, b_init_mean=10, b_init_var=0.001):
super(B_Softmax, self).__init__()
self.b_init = tf.random_normal_initializer(b_init_mean, b_init_var)
self._out_shape = None
def build(self, input_shape):
self.b = tf.Variable(
initial_value = self.b_init(shape=(1,), dtype='float32'),
trainable=True
)
self._out_shape = input_shape
def call(self, inputs):
# This is an implementation of Softmax for batched inputs
# where the factor b is added to the exponents
nominators = tf.math.exp(self.b * inputs)
denominator = tf.reduce_sum(nominators, axis=1)
denominator = tf.squeeze(denominator)
denominator = tf.expand_dims(denominator, -1)
s = tf.divide(nominators, denominator)
return s
# #property
# def output_shape(self): # If I comment out this function, summary prints 'None'
# self.output_shape # If I leave this line, summary prints '?'
# return self._out_shape # If the above line is commented out, summary prints '10' (correctly)
# but the same error is triggered in all three cases
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Variables
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
A = tf.constant([[1,2,3], [7,5,6]], dtype="float32")
batch_size = 128
epochs = 15
input_dim = 5
num_classes = 1
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Dataset
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
start = 3
limit = 16
delta = 3
sample = tf.range( start, limit, delta )
sample = tf.cast( sample, dtype=tf.float32 )
sample = tf.constant( sample, shape=( 1, 1, 1, 5 ) )
dataset = tf.data.Dataset.from_tensor_slices(( sample, tf.constant( [0], shape=( 1, 1, 1, 1 ), dtype=tf.int64)))
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Model Initialize
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
layer = B_Softmax(1.0)
print( layer(A) )
model = tf.keras.Sequential([
tf.keras.layers.Dense(32, activation='relu', input_shape=(1, input_dim)),
tf.keras.layers.Dense(num_classes, activation="softmax"),
B_Softmax(1.0)
])
model.summary()
model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Working
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
model.fit(dataset, batch_size=batch_size, epochs=epochs, validation_data=dataset)
Output:
tf.Tensor(
[[0.09007736 0.24477491 0.6651477 ]
[0.66514784 0.09007736 0.24477486]], shape=(2, 3), dtype=float32)
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 1, 32) 192
dense_1 (Dense) (None, 1, 1) 33
b__softmax_1 (B_Softmax) None 1
=================================================================
Total params: 226
Trainable params: 226
Non-trainable params: 0
_________________________________________________________________
Epoch 1/15
1/1 [==============================] - 5s 5s/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 2/15
1/1 [==============================] - 0s 14ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 3/15
1/1 [==============================] - 0s 15ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 4/15
1/1 [==============================] - 0s 13ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 5/15
1/1 [==============================] - 0s 14ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 6/15
1/1 [==============================] - 0s 12ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 7/15
1/1 [==============================] - 0s 13ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 8/15
1/1 [==============================] - 0s 12ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 9/15
1/1 [==============================] - 0s 12ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 10/15
1/1 [==============================] - 0s 12ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 11/15
1/1 [==============================] - 0s 12ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 12/15
1/1 [==============================] - 0s 15ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 13/15
1/1 [==============================] - 0s 14ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 14/15
1/1 [==============================] - 0s 15ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
Epoch 15/15
1/1 [==============================] - 0s 14ms/step - loss: 0.0000e+00 - accuracy: 0.0000e+00 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00
C:\Python310>
I'm trying to achieve binary classification using MobileNetV2 in TensorFlow. I have two folders A and B and I'm using image_dataset_from_directory function to make them into two classes for training.
BATCH_SIZE = 32
IMG_SIZE = (224, 224)
train_directory = "Train_set/"
test_directory = "Test_set/"
train_dataset = image_dataset_from_directory(train_directory, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)
validation_dataset = image_dataset_from_directory(test_directory, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)
I'm preprocessing the input before passing it to the net.
preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input```
Then I'm creating the model using the code:
def alpaca_model(image_shape=IMG_SIZE):
input_shape = image_shape + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=input_shape,
include_top=False, # <== Important!!!!
weights='imagenet') # From imageNet
# Freeze the base model by making it non trainable
base_model.trainable = False
# create the input layer (Same as the imageNetv2 input size)
inputs = tf.keras.Input(shape=input_shape)
# data preprocessing using the same weights the model was trained on
x = preprocess_input(inputs)
# set training to False to avoid keeping track of statistics in the batch norm layer
x = base_model(x, training=False)
# Add the new Binary classification layers
# use global avg pooling to summarize the info in each channel
x = tf.keras.layers.GlobalAveragePooling2D()(x)
#include dropout with probability of 0.2 to avoid overfitting
x = tf.keras.layers.Dropout(0.2)(x)
# create a prediction layer with one neuron (as a classifier only needs one)
prediction_layer = tf.keras.layers.Dense(1, activation="sigmoid")
outputs = prediction_layer(x)
model = tf.keras.Model(inputs, outputs)
return model
The model summary looks something like this
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_4 (InputLayer) [(None, 224, 224, 3)] 0
tf.math.truediv_1 (TFOpLamb (None, 224, 224, 3) 0
da)
tf.math.subtract_1 (TFOpLam (None, 224, 224, 3) 0
bda)
mobilenetv2_1.00_224 (Funct (None, 7, 7, 1280) 2257984
ional)
global_average_pooling2d_1 (None, 1280) 0
(GlobalAveragePooling2D)
dropout_1 (Dropout) (None, 1280) 0
dense_1 (Dense) (None, 1) 1281
=================================================================
Total params: 2,259,265
Trainable params: 1,281
Non-trainable params: 2,257,984
_________________________________________________________________
Then the model is compiled using the following:
loss_function=tf.keras.losses.BinaryCrossentropy()
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
metrics=['accuracy', tf.metrics.Recall(), tf.metrics.Precision()]
These are the stats of model.fit and model.evaluate
total_epochs = 5
history_fine = model2.fit(train_dataset, epochs=total_epochs, validation_data=validation_dataset)
Epoch 1/5
54/54 [==============================] - 213s 3s/step - loss: 0.2236 - accuracy: 0.9013 - recall: 0.9149 - precision: 0.8852 - val_loss: 0.0856 - val_accuracy: 0.9887 - val_recall: 0.9950 - val_precision: 0.9803
Epoch 2/5
54/54 [==============================] - 217s 4s/step - loss: 0.0614 - accuracy: 0.9855 - recall: 0.9928 - precision: 0.9776 - val_loss: 0.0439 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950
Epoch 3/5
54/54 [==============================] - 216s 4s/step - loss: 0.0316 - accuracy: 0.9948 - recall: 0.9988 - precision: 0.9905 - val_loss: 0.0297 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950
Epoch 4/5
54/54 [==============================] - 217s 4s/step - loss: 0.0258 - accuracy: 0.9954 - recall: 1.0000 - precision: 0.9905 - val_loss: 0.0373 - val_accuracy: 0.9910 - val_recall: 0.9850 - val_precision: 0.9949
Epoch 5/5
54/54 [==============================] - 220s 4s/step - loss: 0.0242 - accuracy: 0.9942 - recall: 0.9988 - precision: 0.9893 - val_loss: 0.0225 - val_accuracy: 0.9977 - val_recall: 1.0000 - val_precision: 0.9950
model2.evaluate(validation_dataset)
14/14 [==============================] - 15s 354ms/step - loss: 0.0225 - accuracy: 0.9977 - recall: 1.0000 - precision: 0.9950
The stats are really good. But when I use the same validation set and check the prediction for individual pics from both folders A and B and plot the predictions, the points don't seem to be linearly seperable.
A = []
for i in os.listdir("Test_set\A"):
location = f"Test_set\A\{i}"
my_image = tf.keras.preprocessing.image.load_img(location, target_size=(224, 224))
preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input
#preprocess the image
my_image = tf.keras.preprocessing.image.img_to_array(my_image)
my_image = my_image.reshape((1, my_image.shape[0], my_image.shape[1],
my_image.shape[2]))
my_image = preprocess_input(my_image)
#make the prediction
prediction = model2.predict(my_image)
# print(prediction)
A.append(float(prediction))
B = []
for i in os.listdir("Test_set\B"):
location = f"Test_set\B\{i}"
my_image = tf.keras.preprocessing.image.load_img(location, target_size=(224, 224))
preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input
#preprocess the image
my_image = tf.keras.preprocessing.image.img_to_array(my_image)
my_image = my_image.reshape((1, my_image.shape[0], my_image.shape[1],
my_image.shape[2]))
my_image = preprocess_input(my_image)
#make the prediction
prediction = model2.predict(my_image)
# print(prediction)
B.append(float(prediction))
Since you have two classes you should replace
prediction_layer = tf.keras.layers.Dense(1, activation="sigmoid")
with
prediction_layer = tf.keras.layers.Dense(2, activation="softmax")
The number of units in the classifier's final layer is equal to the number of classes.
After this, you must re-train the model.
import numpy as np
import pandas as pd
from numpy.random import seed
import tensorflow as tf
from tensorflow import keras
from keras import Sequential
from keras.layers import Dense, Conv1D, MaxPooling2D, Activation
from sklearn.model_selection import train_test_split
seed(1)
tf.random.set_seed(2)
droprate = 0.5
dataset = pd.read_csv('filecounts.csv')
data = np.array(pd.get_dummies(dataset['counts']))
model = Sequential()
model.add(Conv1D(8, kernel_size=3, padding="same", activation="relu",input_shape=(12, 12, 10)))
model.add(MaxPooling2D(pool_size=2))
...
model.add(Conv1D(4, kernel_size=3, padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=2))
...
model.add(Conv1D(1, kernel_size=3, padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=2))
...
model.add(Activation("softmax"))
sgd = keras.optimizers.SGD(learning_rate=1)
train, test = train_test_split(data, test_size=0.5)
model.compile(optimizer=sgd, loss='binary_crossentropy', metrics=['accuracy'])
model.fit(train, epochs=100, batch_size=10)
_, accuracy = model.evaluate(test, verbose=0, steps=1)
print('Accuracy: %.2f' % (accuracy*100))
Conv1D expects input shape 3+D tensor with shape: batch_shape + (steps, input_dim)and output 3+D tensor with shape: batch_shape + (new_steps, filters) with or without padding='same'.
Error is due to Maxpooling2D expects 4D tensor with shape (batch_size, rows, cols, channels).
Working sample code
import tensorflow as tf
import numpy as np
import tensorflow.keras as keras
X_train = np.random.random((12,12,10))
y_train = np.random.random((12, 1))
model = tf.keras.Sequential()
model.add(keras.layers.Conv1D(8, kernel_size=3, padding="same", activation="relu",input_shape=(12, 10)))
model.add(keras.layers.MaxPool1D(pool_size=2))
model.add(keras.layers.Conv1D(4, kernel_size=3, padding="same", activation="relu"))
model.add(keras.layers.MaxPool1D(pool_size=2))
model.add(keras.layers.Conv1D(1, kernel_size=3, padding="same", activation="relu"))
model.add(keras.layers.MaxPool1D(pool_size=2))
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(units = 128, activation = 'relu'))
model.add(keras.layers.Dense(units = 1, activation = 'softmax'))
model.compile(optimizer = 'adam',loss = 'binary_crossentropy',metrics = ['accuracy'])
model.fit(X_train, y_train, epochs = 15)
Output
Epoch 1/10
1/1 [==============================] - 1s 944ms/step - loss: 0.6914 - accuracy: 0.0000e+00
Epoch 2/10
1/1 [==============================] - 0s 9ms/step - loss: 0.6900 - accuracy: 0.0000e+00
Epoch 3/10
1/1 [==============================] - 0s 9ms/step - loss: 0.6885 - accuracy: 0.0000e+00
Epoch 4/10
1/1 [==============================] - 0s 7ms/step - loss: 0.6870 - accuracy: 0.0000e+00
Epoch 5/10
1/1 [==============================] - 0s 8ms/step - loss: 0.6856 - accuracy: 0.0000e+00
Epoch 6/10
1/1 [==============================] - 0s 9ms/step - loss: 0.6841 - accuracy: 0.0000e+00
Epoch 7/10
1/1 [==============================] - 0s 8ms/step - loss: 0.6828 - accuracy: 0.0000e+00
Epoch 8/10
1/1 [==============================] - 0s 14ms/step - loss: 0.6814 - accuracy: 0.0000e+00
Epoch 9/10
1/1 [==============================] - 0s 7ms/step - loss: 0.6801 - accuracy: 0.0000e+00
Epoch 10/10
1/1 [==============================] - 0s 11ms/step - loss: 0.6789 - accuracy: 0.0000e+00
<keras.callbacks.History at 0x7eff56169810>
I'm pretty new to machine learning, and this is my first project using tensorflow and keras. I'm trying to predict a numerical value given a data set but my model is working.
x_train, x_test, y_train, y_test = train_test_split(dates, prices, test_size=0.33)
x_train = np.reshape(x_train,(x_train.shape[0], 1, x_train.shape[1]))
model = Sequential()
model.add(LSTM(30, return_sequences=True, input_shape= (x_train.shape[0], 1)))
# model.add(Flatten())
model.add(Dropout(0.25))
model.add(Dense(100,activation = 'relu'))
model.add(Dropout(0.25))
model.add(Dense(1,activation='softmax'))
model.compile(optimizer='adam',loss='mean_squared_error', metrics=['accuracy'])
model.fit(x_train, y_train, batch_size=32, epochs=5)
This is my model above, but whenever it runs it shows this below:
Epoch 1/5
WARNING:tensorflow:Model was constructed with shape (None, 1686, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 1686, 1), dtype=tf.float32, name='lstm_input'), name='lstm_input', description="created by layer 'lstm_input'"), but it was called on an input with incompatible shape (None, 1, 1).
WARNING:tensorflow:Model was constructed with shape (None, 1686, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 1686, 1), dtype=tf.float32, name='lstm_input'), name='lstm_input', description="created by layer 'lstm_input'"), but it was called on an input with incompatible shape (None, 1, 1).
53/53 [==============================] - 2s 2ms/step - loss: 1314.1159 - accuracy: 0.0000e+00
Epoch 2/5
53/53 [==============================] - 0s 2ms/step - loss: 1332.1348 - accuracy: 0.0000e+00
Epoch 3/5
53/53 [==============================] - 0s 2ms/step - loss: 1307.5851 - accuracy: 0.0000e+00
Epoch 4/5
53/53 [==============================] - 0s 2ms/step - loss: 1327.0625 - accuracy: 0.0000e+00
Epoch 5/5
53/53 [==============================] - 0s 2ms/step - loss: 1314.4220 - accuracy: 0.0000e+00
<tensorflow.python.keras.callbacks.History at 0x7f1cdfc56668>
Does anyone know how to fix this and improve accuracy and loss?
Help is greatly appreciated.
If you are trying to predict a numerical value then you are NOT doing a classification problem but rather a regression problem. Therefore your dense layer with 1 neuron should have no activation function.
Replace your last layer of Sequential model with this:
model.add(Dense(1, kernel_initializer='normal'))
Regression task dont need softmax as activation function because softmax is mostly used in multi class classification
I am training a simple machine learning model that takes a 1D description of a physical system (502 elements) and predicts the total energy (1 element). As I am new to TensorFlow I have used a simple dense neural network with two hidden layers of 64 neurons each:
Model: "total_energy"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
charge_density_x_max (InputL [(None, 502)] 0
_________________________________________________________________
hidden_1 (Dense) (None, 64) 32192
_________________________________________________________________
hidden_2 (Dense) (None, 64) 4160
_________________________________________________________________
dense (Dense) (None, 1) 65
=================================================================
Total params: 36,417
Trainable params: 36,417
Non-trainable params: 0
_________________________________________________________________
This is my source code for the training, evaluation and prediction:
# imports
import os
import ast
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
# load the dataset from the csv file
data = pd.read_csv('1e_data.csv')
# load in the data
x_train = np.zeros(shape=(600, 502))
x_test = np.zeros(shape=(400, 502))
y_train = np.zeros(shape=(600))
y_test = np.zeros(shape=(400))
for i in range(0, 1000):
if i < 600:
x_train[i,:] = np.append(np.array(ast.literal_eval(data.loc[i,'n'])), float(data.loc[i,'xmax']))
y_train[i] = float(data.loc[i,'E'])
else:
x_test[i-600,:] = np.append(np.array(ast.literal_eval(data.loc[i,'n'])), float(data.loc[i,'xmax']))
y_test[i-600] = float(data.loc[i,'E'])
# build the neural network model
inputs = tf.keras.Input(shape=(502,), name='charge_density_x_max')
hidden1 = tf.keras.layers.Dense(64, activation='sigmoid', name='hidden_1')(inputs)
hidden2 = tf.keras.layers.Dense(64, activation='sigmoid', name='hidden_2')(hidden1)
outputs = tf.keras.layers.Dense(1)(hidden2)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='total_energy')
# save the info of the model
with open('model_info.dat','w') as fh:
model.summary(print_fn=lambda x: fh.write(x + '\n'))
# compile the model
model.compile(optimizer='adam', loss='mean_absolute_percentage_error', metrics=['accuracy'])
# perform the training
model.fit(x_train, y_train, epochs=10)
# evaluate the model for accuracy
model.evaluate(x_test, y_test, verbose=2)
Yet when I run this it seems to do no training at all, giving an accuracy of 0.0000e+00:
Epoch 1/10
600/600 [==============================] - 0s 196us/sample - loss: 289.0616 - acc: 0.0000e+00
Epoch 2/10
600/600 [==============================] - 0s 37us/sample - loss: 144.5967 - acc: 0.0000e+00
Epoch 3/10
600/600 [==============================] - 0s 46us/sample - loss: 97.2109 - acc: 0.0000e+00
Epoch 4/10
600/600 [==============================] - 0s 46us/sample - loss: 108.0698 - acc: 0.0000e+00
Epoch 5/10
600/600 [==============================] - 0s 47us/sample - loss: 84.5921 - acc: 0.0000e+00
Epoch 6/10
600/600 [==============================] - 0s 38us/sample - loss: 79.9309 - acc: 0.0000e+00
Epoch 7/10
600/600 [==============================] - 0s 38us/sample - loss: 80.6755 - acc: 0.0000e+00
Epoch 8/10
600/600 [==============================] - 0s 47us/sample - loss: 87.5954 - acc: 0.0000e+00
Epoch 9/10
600/600 [==============================] - 0s 46us/sample - loss: 73.6634 - acc: 0.0000e+00
Epoch 10/10
600/600 [==============================] - 0s 38us/sample - loss: 78.0825 - acc: 0.0000e+00
400/400 - 0s - loss: 70.3813 - acc: 0.0000e+00
I have probably made a simple mistake here, but I do not know how to begin debugging. This should perform at least some training, but at the moment it seems to just skip the training and give an accuracy of 0.
You are in a regression setting, where accuracy is meaningless (it is meaningful only for classification problems); see What function defines accuracy in Keras when the loss is mean squared error (MSE)? for more details (it is applicable in your case, too, despite the use of a different loss).
The fact that your network does indeed learn is apparent from the reduction in your loss, which is the actual quantity of interest in regression problems (you simply don't need any metrics here).
Independently of the above, you should probably change the sigmoid activations to relu (we normally do not use sigmoid nowadays for intermediate layers).