We Keep Coding

Custom estimator can't be deepcopied by cross_val_score

I have a custom estimator that i implemented myself and i am not able to use cross_val_score(), which i believe it has something to do with my predict() method. Here is the full error trace:
Traceback (most recent call last):
File "/Users/joann/Desktop/Implementações ML/Adaboost Classifier/test.py", line 30, in <module>
ada2_score = cross_val_score(ada_2, X, y, cv=5)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 390, in cross_val_score
error_score=error_score)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 236, in cross_validate
for train, test in cv.split(X, y, groups))
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 1004, in __call__
if self.dispatch_one_batch(iterator):
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 835, in dispatch_one_batch
self._dispatch(tasks)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 754, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 209, in apply_async
result = ImmediateResult(func)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 590, in __init__
self.results = batch()
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 256, in __call__
for func, args, kwargs in self.items]
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 256, in <listcomp>
for func, args, kwargs in self.items]
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 544, in _fit_and_score
test_scores = _score(estimator, X_test, y_test, scorer)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 591, in _score
scores = scorer(estimator, X_test, y_test)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/metrics/_scorer.py", line 89, in __call__
score = scorer(estimator, *args, **kwargs)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/metrics/_scorer.py", line 371, in _passthrough_scorer
return estimator.score(*args, **kwargs)
File "/Users/joann/Desktop/Implementações ML/Adaboost Classifier/Adaboost.py", line 92, in score
scr_pred = self.predict(X)
File "/Users/joann/Desktop/Implementações ML/Adaboost Classifier/Adaboost.py", line 73, in predict
clf_pred = clf.predict(X)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn_extensions/extreme_learning_machines/elm.py", line 614, in predict
class_predictions = self.binarizer.inverse_transform(raw_predictions)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/preprocessing/_label.py", line 528, in inverse_transform
self.classes_, threshold)
File "/Users/joann/opt/anaconda3/lib/python3.7/site-packages/sklearn/preprocessing/_label.py", line 750, in _inverse_binarize_thresholding
format(y.shape))
ValueError: output_type='binary', but y.shape = (30, 3)
My predict(self, X) method returns a vector of size n_samples with the predictions for the X parameter. I also made a score() function as follows:
def score(self, X, y):
scr_pred = self.predict(X)
return sum(scr_pred == y) / X.shape[0]
This method simply computes the accuracy of the model given the samples. Either if i use this score() method or set a cross_val_score(... , scoring="accuracy") it is not working.
Note: i am aware of this question/answer but this doesn't apply to my case because i can confirm the consistence of my constructor:
def __init__(self, estimators=["MLP"], n_rounds=5, random_state=10):
self.estimators = estimators
self.n_rounds = n_rounds
self.random_state = random_state
UPDATE:
Further research led me to this topic, where it is explained that sklearn can't deepcopy Estimators with transformers. However, it is mandatory for my estimator to run LabelBinarizer to transform data to get the predictions. So i update the question title to the proper issue.`

However the problem statement of yours is not clear here but however looking at the error it seems you are trying a multiclass classification.
The problem here is that you might have in your code at some point have not done the preprocessing correctly as the error is logged from inverse_binarize_thresholding which is raised due to below functionality of sklearn pre-prosessing:
def _inverse_binarize_thresholding(y, output_type, classes, threshold):
if output_type == "binary" and y.ndim == 2 and y.shape[1] > 2:
raise ValueError("output_type='binary', but y.shape = {0}".
format(y.shape))
There must be some missing transformation or pre-prosessing in your code and you have to use LabelBinarizer correctly
Go through the below documentation and backtrack the error to fix your code
documentation

Tensorflow Image Generator passing Tensor with dtype=string instead of Tensor with dtype=float32 to loss function

I am following the YOLO v1 paper to create an object detector from scratch with Tensorflow and python. My dataset is a set of images and a 7x7x12 Tensor that represents the label for the image. I import the image names and labels (as a string) into a dataframe from a CSV using pandas, and then do some operations on the labels to turn them into Tensors. I then create a generator using ImageGenerator.flow_from_dataframe(), and the feed that generator as the input for my model. I end up getting the following error when the model tries to call the custom loss function that I created:
File "/home/michael/Desktop/YOLO_Detector/src/main.py", line 61, in <module>
epochs=10)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 342, in fit
total_epochs=epochs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 128, in run_one_epoch
batch_outs = execution_function(iterator)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 98, in execution_function
distributed_function(input_fn))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in __call__
result = self._call(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
*args, **kwds))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 85, in distributed_function
per_replica_function, args=args)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 763, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1819, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 2164, in _call_for_each_replica
return fn(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py", line 292, in wrapper
return func(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 433, in train_on_batch
output_loss_metrics=model._output_loss_metrics)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 312, in train_on_batch
output_loss_metrics=output_loss_metrics))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 253, in _process_single_batch
training=training))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 167, in _model_loss
per_sample_losses = loss_fn.call(targets[i], outs[i])
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/losses.py", line 221, in call
return self.fn(y_true, y_pred, **self._fn_kwargs)
File "/home/michael/Desktop/YOLO_Detector/src/utils.py", line 25, in yolo_loss_function
pos = kb.mean(y_true-y_pred)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/math_ops.py", line 902, in binary_op_wrapper
return func(x, y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 10104, in sub
"Sub", x=x, y=y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 576, in _apply_op_helper
param_name=input_name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 61, in _SatisfiesTypeConstraint
", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'x' has DataType string not in list of allowed values: bfloat16, float16, float32, float64, uint8, int8, uint16, int16, int32, int64, complex64, complex128
When I use the python debugger to check the y_true that is being passed to the loss function, I see the following tensor:
Tensor("IteratorGetNext:1", shape=(None, 1), dtype=string)
however when I manually check the label of the image by calling the following python code, I get a tensor with the correct shape and values:
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
-------------Output--------------
<class 'tensorflow.python.framework.ops.EagerTensor'>
(7, 7, 12)
<dtype: 'float32'>
Below is the code that I am using to create the dataset and train the model:
import tensorflow as tf
from tensorflow import keras
import pandas as pd
import matplotlib.pyplot as plt
import cv2
import numpy as np
import os
from src import utils, model
path = "/home/michael/Desktop/YOLO_Detector/dataset/labels.csv"
train_df = pd.read_csv(path, delim_whitespace=True, header=None)
train_df.columns = ['filename', 'output_tensor']
train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: utils.string_to_tensor(x))
# train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: tf.expand_dims(x, 3))
image_generator = keras.preprocessing.image.ImageDataGenerator(rescale=1. / 255.,
validation_split=0.2)
train_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode='raw',
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="training"
)
validation_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode="raw",
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="validation"
)
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
model = model.create_model()
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate=1e-2,
decay_rate=0.0005,
decay_steps=100000
)
sgd = keras.optimizers.SGD(learning_rate=lr_schedule, momentum=0.9)
model.compile(optimizer=sgd, loss=utils.yolo_loss_function, metrics=['accuracy'])
model.fit(x=train_gen,
epochs=10)
I am using Tensorflow 2 with ROCm, and Eager Execution is on. How do I get the y_true tensor to be the correct output label (7x7x12 Tensor with dtype=float32) instead of it being a string?

I figured out the issue. The issue is that you cannot correctly store a Tensor or a Numpy array in a Pandas Dataframe. I ended up having to manually create the image/tensor pairs by doing the following:
img_list = []
labels_list = []
for i in range(64):
labels_list.append(utils.string_to_numpy(train_df["output_tensor"][i]))
image = tf.keras.preprocessing.image.load_img(f"/home/michael/Desktop/YOLO_Detector/dataset/{train_df['filename'][i]}", target_size=(448, 448))
image_arr = keras.preprocessing.image.img_to_array(image) / 255.0
img_list.append(image_arr)
img = np.asarray(img_list)
label = np.asarray(labels_list)
and then calling the img as x and label as y in model.fit()

Error while feeding tf.Dataset to fit(): KeyError: 'embedding_input'

I'm using TensorFlow 2.0 Datasets to feed my model's fit function. Here is the code:
def build_model(self):
self.g_Model = Sequential()
self.g_Model.add(Embedding(self.g_Max_features, output_dim=256))
self.g_Model.add(LSTM(128))
self.g_Model.add(Dropout(0.5))
self.g_Model.add(Dense(1, activation='sigmoid'))
self.g_Model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
def train_model(self, filenames):
lstm_feature_description = {
'X': tf.io.FixedLenFeature(CONFIG.g_keras_lstm_max_document_length, tf.float32),
'y': tf.io.FixedLenFeature((), tf.int64),
}
def _parse_lstm_function(example_proto):
return tf.io.parse_single_example(serialized=example_proto, features=lstm_feature_description)
self.build_model()
# Start Preparing The Data
raw_lstm_dataset = tf.data.TFRecordDataset(CONFIG.g_record_file_lstm)
parsed_lstm_dataset = raw_lstm_dataset.map(_parse_lstm_function)
parsed_lstm_dataset = parsed_lstm_dataset.shuffle(CONFIG.g_shuffle_s).batch(CONFIG.g_Batch_size)
self.g_Model.fit(parsed_lstm_dataset, epochs=2)
But I receive the following error:
Traceback (most recent call last):
File "keras_lstm_v2.py", line 79, in train_model
1/Unknown - 0s 0s/step self.g_Model.fit(parsed_lstm_dataset, epochs=2)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 324, in fit
total_epochs=epochs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 503, in _call
self._initialize(args, kwds, add_initializers_to=initializer_map)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 408, in _initialize
*args, **kwds))
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 1848, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "venv_tf_new\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 66, in distributed_function
model, input_iterator, mode)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 118, in _prepare_feed_values
inputs = [inputs[key] for key in model._feed_input_names]
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 118, in <listcomp>
inputs = [inputs[key] for key in model._feed_input_names]
KeyError: 'embedding_input'
I've seen this thread, however it doesn't clarify things up for me. As far as I understood there is a problem with the loaded data, but according to documentation for Datasets it should work out of the box, so I couldn't figure out how to fix it.
Any help is appreciated. Thanks!

You need to declare your model's inputs somewhere, typically something like
model = tf.keras.Model(inputs=inputs, outputs=outputs)
Try taking out the last line in your model building function
self.g_Model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
and move it to your train function, once you have declared your model you can use the output of the model as your output layer and declare your input layer with something like
input_size = CONFIG.g_keras_lstm_max_document_length
input_layer = tf.keras.layers.Input(input_size)
output_layer = self.build_model()
model = tf.keras.Model(inputs=input_layer, outputs=output_layer )
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit( .... )

I found the solution myself today. Actually, it was two errors in my code:
The .fit() function could not determine where inputs and labels are as noted by #NiallJG above. However, the solution provided didn't solve the problem, so I fixed it up in the following way:
1.1 In my build_model function I've added "name" to Embedding layer:
self.g_Model.add(Embedding(input_dim=self.g_Max_features, output_dim=256, name='X'))
1.2 To match this name I've actually needed to change my lstm_feature_description, so it contains "_input" postfix:
def train_model(self, filenames):
lstm_feature_description = {
'X_input': tf.io.FixedLenFeature(CONFIG.g_keras_lstm_max_document_length, tf.float32),
'y': tf.io.FixedLenFeature((), tf.int64),
}
My _parse_lstm_function was returning data to the Dataset in the
wrong manner, causing "IndexError: list index out of range" error. Here is how the modified function should look like:
def _parse_lstm_function(example_proto):
# Parse the input tf.Example proto using the dictionary above.
parsed = tf.io.parse_single_example(serialized=example_proto, features=lstm_feature_description)
return parsed["X_input"], parsed["y"]
This allowed the model to .fit() correctly, except now I've left with OOM error, but this will be covered in another question.

Could not find matching function to call loaded from the SavedModel

I wrote below code and it suppose to load a model followed by a predictive run of an element from MNIST dataset. At the beginning of the execution the code works fine and I get my desired prediction, but then suddenly I did get the below error following error and I'm not sure if this could be related to .predict arguments.
My code:
# importing libraries
import tensorflow as tf # deep learning library. Tensors are just multi-dimensional arrays
import gzip,sys,pickle # dataset manipulation library
# importing MNIST dataset
f = gzip.open('mnist.pkl.gz', 'rb')
if sys.version_info < (3,):
data = pickle.load(f)
else:
data = pickle.load(f, encoding='bytes')
f.close()
(x_train, _), (x_test, _) = data
print("-----------------------dataset ready-----------------------")
# using an expample from x_test / to remove later
# preprocessing
x_test = tf.keras.utils.normalize(x_test, axis=1) # scales data between 0 and 1
# importing model
new_model = tf.keras.models.load_model('epic_num_reader.model')
print("-----------------------model ready-----------------------")
# getting prediction
predictions = new_model.predict(x_test[0])
import numpy as np
print("-----------------------predection ready-----------------------")
print(np.argmax(predictions))
The error message:
-----------------------dataset ready-----------------------
2019-10-27 00:36:58.767359: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
-----------------------model ready-----------------------
Traceback (most recent call last):
File "c:\Users\lotfi\Desktop\DigitsDetector\main1.py", line 24, in <module>
predictions = new_model.predict(x_test[0])
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 909, in predict
use_multiprocessing=use_multiprocessing)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 462, in predict
steps=steps, callbacks=callbacks, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 444, in _model_iteration
total_epochs=1)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 503, in _call
self._initialize(args, kwds, add_initializers_to=initializer_map)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 408, in _initialize
*args, **kwds))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 1848, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 73, in distributed_function
per_replica_function, args=(model, x, y, sample_weights))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 760, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 1787, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 2132, in _call_for_each_replica
return fn(*args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\autograph\impl\api.py", line 292, in wrapper
return func(*args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 162, in _predict_on_batch
return predict_on_batch(model, x)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 370, in predict_on_batch
return model(inputs) # pylint: disable=not-callable
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 847, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\sequential.py", line 270, in call
outputs = layer(inputs, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 847, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\utils.py", line 57, in return_outputs_and_add_losses
outputs, losses = fn(inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 494, in _call
results = self._stateful_fn(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 1822, in __call__
graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\saved_model\function_deserialization.py", line 262, in restored_function_body
"\n\n".join(signature_descriptions)))
Error message continued:
ValueError: Could not find matching function to call loaded from the SavedModel. Got:
Positional arguments (1 total):
* Tensor("inputs:0", shape=(None, 28), dtype=float32)
Keyword arguments: {}
Expected these arguments to match one of the following 1 option(s):
Option 1:
Positional arguments (1 total):
* TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name='inputs')
Keyword arguments: {}

Note: I think your problem is on Predict Model part. In that part you have used x_test[0] which is not matching with the pre-trained model array dimension. You have to use x_test instead of x_test[0]. enter image description here
#Use This Code TO Solve Your Problem
import tensorflow as tf # deep learning library. Tensors are just multi-dimensional arrays
mnist = tf.keras.datasets.mnist # mnist is a dataset of 28x28 images of handwritten digits and their labels
(x_train, y_train),(x_test, y_test) = mnist.load_data() # unpacks images to x_train/x_test and labels to y_train/y_test
x_train = tf.keras.utils.normalize(x_train, axis=1) # scales data between 0 and 1
x_test = tf.keras.utils.normalize(x_test, axis=1) # scales data between 0 and 1
model = tf.keras.models.Sequential() # a basic feed-forward model
model.add(tf.keras.layers.Flatten()) # takes our 28x28 and makes it 1x784
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # a simple fully-connected layer, 128 units, relu activation
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # a simple fully-connected layer, 128 units, relu activation
model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax)) # our output layer. 10 units for 10 classes. Softmax for probability distribution
model.compile(optimizer='adam', # Good default optimizer to start with
loss='sparse_categorical_crossentropy', # how will we calculate our "error." Neural network aims to minimize loss.
metrics=['accuracy']) # what to track
model.fit(x_train, y_train, epochs=3) # train the model
val_loss, val_acc = model.evaluate(x_test, y_test) # evaluate the out of sample data with model
print(val_loss) # model's loss (error)
print(val_acc) # model's accuracy
--------------------------Save Model----------------------------------------
model.save('epic_num_reader.model') # save the model
--------------------------Load Model----------------------------------------
new_model = tf.keras.models.load_model('epic_num_reader.model') # Load the model
--------------------------Predict Model-------------------------------------
predictions = new_model.predict(x_test)
print(predictions)
--------------------------visualize Prediction------------------------------
plt.imshow(x_test[0],cmap=plt.cm.binary)
plt.show()
-------------------------- Validated Prediction-----------------------------
import numpy as np
print(np.argmax(predictions[0]))

Had the same issue.
Try:
pip install tf-nightly
Solution is from here:
https://github.com/tensorflow/tensorflow/issues/35446 - comment from oanush.
But this may break calling of tensorboards if you have one.
Step by step solution for this is below:
pip uninstall tensorflow
pip uninstall tensorboard
pip install -q tf-nightly
pip install --ignore-installed tf-nightly
Got if from here: https://github.com/tensorflow/tensorboard/issues/2226 - comment from mmehedin, 29 Jun 2019.

Tensorflow FailedPreconditionError: Attempting to use uninitialized value beta1_power

I am trying to set up a simple convolutional network in Tensorflow. I'll try to keep the amount of code at a minimum level. Here's the class:
class ConvNet(object):
def __init__(self, input, labels, dataset):
self.input = input
self.true_labels = labels
#'dataset' is an instance of a class that
#I am using to read the training images
self.data = dataset
self.build()
self._optimize = None
self.sess = tf.Session()
self.sess.run(tf.global_variables_initializer())
def build(self):
self.conv_layers = []
# create 3 conv layers using tf.nn.conv2d
# and append them to self.conv_layers
# create flattening layer using tf.nn.reshape
self.fc_layers = []
# create 2 fully connected layers using tf.matmul
# and append them to self.fc_layers
#property
def optimize(self):
"""Return the optimize operation."""
if not self._optimize:
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits = self.fc_layers[-1],
labels = self.true_labels)
cost = tf.reduce_mean(cross_entropy)
self._optimize = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(cost)
return self._optimize
def train(self, num_epochs=1, batch_size=16):
epochs_done = 0
while not epochs_done == num_epochs:
# get next batch from training data
x_batch, y_true_batch, _, _ = self.data.train.next_batch(batch_size)
feed_dict_train = {self.input: x_batch, self.true_labels: y_true_batch}
#even if I initialize variables here, same error occurs
#self.sess.run(tf.global_variables_initializer())
self.sess.run(self.optimize, feed_dict=feed_dict_train)
if self.data.train.epochs_done > epochs_done:
#print stuff...
epochs_done += 1
self.sess.close()
I was trying to run the optimize operation to train the network on a simple dataset (e.g. images of cats vs dogs). Here is the main:
if __name__ == "__main__":
classes = ['dogs', 'cats']
num_classes = len(classes)
batch_size = 16
img_size = 128
num_channels = 3
#read training data and resize images to 128 x 128 pixels
data = dataset.read_train_sets(img_size, classes)
x = tf.placeholder(tf.float32, shape = [None, img_size, img_size, num_channels], name='x')
y_true = tf.placeholder(tf.float32, shape = [None, num_classes], name = 'y_true')
net = ConvNet(x, y_true, data)
net.train()
However I keep getting the following error (it's actually much longer than this, I can post it if necessary):
Caused by op 'beta1_power/read', defined at:
File "<string>", line 1, in <module>
File "/usr/lib/python3.5/idlelib/run.py", line 124, in main
ret = method(*args, **kwargs)
File "/usr/lib/python3.5/idlelib/run.py", line 351, in runcode
exec(code, self.locals)
File "/home/gian/face_recognition/model.py", line 366, in <module>
net.train()
File "/home/gian/face_recognition/model.py", line 309, in train
self.sess.run(self.optimize, feed_dict=feed_dict_train)
File "/home/gian/face_recognition/model.py", line 269, in optimize
self._optimize = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(cost)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 409, in minimize
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 552, in apply_gradients
self._create_slots([_get_variable_for(v) for v in var_list])
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/adam.py", line 124, in _create_slots
colocate_with=first_var)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/training/optimizer.py", line 665, in _create_non_slot_variable
v = variable_scope.variable(initial_value, name=name, trainable=False)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 2157, in variable
use_resource=use_resource)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 2147, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py", line 2130, in default_variable_creator
constraint=constraint)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 235, in __init__
constraint=constraint)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 391, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 142, in identity
return gen_array_ops.identity(input, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 3053, in identity
"Identity", input=input, name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3290, in create_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1654, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value beta1_power
[[Node: beta1_power/read = Identity[T=DT_FLOAT, _class=["loc:#Variable"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](beta1_power)]]
Can somebody help me figure out where the problem is? Thanks a lot

This error generally occurs when you haven't initialized the optimizer.
So just add self.optimize before you initialize all the global variables.
Your code should look like this.
def __init__(self, input, labels, dataset):
self.input = input
self.true_labels = labels
#'dataset' is an instance of a class that
#I am using to read the training images
self.data = dataset
self.build()
self._optimize = None
self.sess = tf.Session()
self.optimize()
self.sess.run(tf.global_variables_initializer())