I wrote below code and it suppose to load a model followed by a predictive run of an element from MNIST dataset. At the beginning of the execution the code works fine and I get my desired prediction, but then suddenly I did get the below error following error and I'm not sure if this could be related to .predict arguments.
My code:
# importing libraries
import tensorflow as tf # deep learning library. Tensors are just multi-dimensional arrays
import gzip,sys,pickle # dataset manipulation library
# importing MNIST dataset
f = gzip.open('mnist.pkl.gz', 'rb')
if sys.version_info < (3,):
data = pickle.load(f)
else:
data = pickle.load(f, encoding='bytes')
f.close()
(x_train, _), (x_test, _) = data
print("-----------------------dataset ready-----------------------")
# using an expample from x_test / to remove later
# preprocessing
x_test = tf.keras.utils.normalize(x_test, axis=1) # scales data between 0 and 1
# importing model
new_model = tf.keras.models.load_model('epic_num_reader.model')
print("-----------------------model ready-----------------------")
# getting prediction
predictions = new_model.predict(x_test[0])
import numpy as np
print("-----------------------predection ready-----------------------")
print(np.argmax(predictions))
The error message:
-----------------------dataset ready-----------------------
2019-10-27 00:36:58.767359: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
-----------------------model ready-----------------------
Traceback (most recent call last):
File "c:\Users\lotfi\Desktop\DigitsDetector\main1.py", line 24, in <module>
predictions = new_model.predict(x_test[0])
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 909, in predict
use_multiprocessing=use_multiprocessing)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 462, in predict
steps=steps, callbacks=callbacks, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 444, in _model_iteration
total_epochs=1)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 503, in _call
self._initialize(args, kwds, add_initializers_to=initializer_map)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 408, in _initialize
*args, **kwds))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 1848, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 73, in distributed_function
per_replica_function, args=(model, x, y, sample_weights))
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 760, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 1787, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\distribute\distribute_lib.py", line 2132, in _call_for_each_replica
return fn(*args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\autograph\impl\api.py", line 292, in wrapper
return func(*args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 162, in _predict_on_batch
return predict_on_batch(model, x)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 370, in predict_on_batch
return model(inputs) # pylint: disable=not-callable
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 847, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\sequential.py", line 270, in call
outputs = layer(inputs, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 847, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\keras\saving\saved_model\utils.py", line 57, in return_outputs_and_add_losses
outputs, losses = fn(inputs, *args, **kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 494, in _call
results = self._stateful_fn(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 1822, in __call__
graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "C:\Users\lotfi\Anaconda3\envs\tf\lib\site-packages\tensorflow_core\python\saved_model\function_deserialization.py", line 262, in restored_function_body
"\n\n".join(signature_descriptions)))
Error message continued:
ValueError: Could not find matching function to call loaded from the SavedModel. Got:
Positional arguments (1 total):
* Tensor("inputs:0", shape=(None, 28), dtype=float32)
Keyword arguments: {}
Expected these arguments to match one of the following 1 option(s):
Option 1:
Positional arguments (1 total):
* TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name='inputs')
Keyword arguments: {}
Note: I think your problem is on Predict Model part. In that part you have used x_test[0] which is not matching with the pre-trained model array dimension. You have to use x_test instead of x_test[0]. enter image description here
#Use This Code TO Solve Your Problem
import tensorflow as tf # deep learning library. Tensors are just multi-dimensional arrays
mnist = tf.keras.datasets.mnist # mnist is a dataset of 28x28 images of handwritten digits and their labels
(x_train, y_train),(x_test, y_test) = mnist.load_data() # unpacks images to x_train/x_test and labels to y_train/y_test
x_train = tf.keras.utils.normalize(x_train, axis=1) # scales data between 0 and 1
x_test = tf.keras.utils.normalize(x_test, axis=1) # scales data between 0 and 1
model = tf.keras.models.Sequential() # a basic feed-forward model
model.add(tf.keras.layers.Flatten()) # takes our 28x28 and makes it 1x784
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # a simple fully-connected layer, 128 units, relu activation
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # a simple fully-connected layer, 128 units, relu activation
model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax)) # our output layer. 10 units for 10 classes. Softmax for probability distribution
model.compile(optimizer='adam', # Good default optimizer to start with
loss='sparse_categorical_crossentropy', # how will we calculate our "error." Neural network aims to minimize loss.
metrics=['accuracy']) # what to track
model.fit(x_train, y_train, epochs=3) # train the model
val_loss, val_acc = model.evaluate(x_test, y_test) # evaluate the out of sample data with model
print(val_loss) # model's loss (error)
print(val_acc) # model's accuracy
--------------------------Save Model----------------------------------------
model.save('epic_num_reader.model') # save the model
--------------------------Load Model----------------------------------------
new_model = tf.keras.models.load_model('epic_num_reader.model') # Load the model
--------------------------Predict Model-------------------------------------
predictions = new_model.predict(x_test)
print(predictions)
--------------------------visualize Prediction------------------------------
plt.imshow(x_test[0],cmap=plt.cm.binary)
plt.show()
-------------------------- Validated Prediction-----------------------------
import numpy as np
print(np.argmax(predictions[0]))
Had the same issue.
Try:
pip install tf-nightly
Solution is from here:
https://github.com/tensorflow/tensorflow/issues/35446 - comment from oanush.
But this may break calling of tensorboards if you have one.
Step by step solution for this is below:
pip uninstall tensorflow
pip uninstall tensorboard
pip install -q tf-nightly
pip install --ignore-installed tf-nightly
Got if from here: https://github.com/tensorflow/tensorboard/issues/2226 - comment from mmehedin, 29 Jun 2019.
Related
So, I was doing my code like the one below. I want to Fit the ANN into the Training set, but the error occurred like this. I was confused about how to solve it, I already try several suggestions on Google, but it still came out an error. I also tried several codes to display the result, but most of the errors occurred is because of the fitting like this one. So, I was thinking that the main problem my code can't run is because of the fitting model.
#Importing necessary Libraries
import numpy as np
import pandas as pd
import tensorflow as tf
import keras
#dataset
data = pd.read_excel("E:\\MATKUL LUV\\THESIS\\DATASETS\\DPM1 1052.xlsx")
#print (data.head)
# Separate Target Variable and Predictor Variables
TargetVariable=['DPM1Fault']
Predictors=['DPM1Cx', 'DPM1Cy']
X=data[Predictors].values
y=data[TargetVariable].values
# Split the data into training and testing set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# Quick sanity check with the shapes of Training and testing datasets
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
# importing the libraries
from keras.models import Sequential
from keras.layers import Dense
# create ANN model
model = Sequential()
# Defining the Input layer and FIRST hidden layer, both are same!
model.add(Dense(units=5, input_dim=2, kernel_initializer='normal', activation='relu'))
# Defining the Second layer of the model
# after the first layer we don't have to specify input_dim as keras configure it automatically
model.add(Dense(units=4, kernel_initializer='normal', activation='relu'))
# The output neuron is a single fully connected node
# Since we will be predicting a single number
model.add(Dense(1, kernel_initializer='normal'))
# Compiling the model
model.compile(loss='mean_squared_error', optimizer='adam')
# Fitting the ANN to the Training set
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
and the result like this
Epoch 1/50
---------------------------------------------------------------------------
UnimplementedError Traceback (most recent call last)
Input In [52], in <cell line: 23>()
20 model.compile(loss='mean_squared_error', optimizer='adam')
22 # Fitting the ANN to the Training set
---> 23 model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
File ~\anaconda3\lib\site-packages\keras\utils\traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
File ~\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py:52, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
50 try:
51 ctx.ensure_initialized()
---> 52 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
53 inputs, attrs, num_outputs)
54 except core._NotOkStatusException as e:
55 if name is not None:
UnimplementedError: Graph execution error:
Detected at node 'mean_squared_error/Cast' defined at (most recent call last):
File "C:\Users\16agn\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\16agn\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "C:\Users\16agn\anaconda3\lib\site-packages\traitlets\config\application.py", line 846, in launch_instance
app.start()
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 677, in start
self.io_loop.start()
File "C:\Users\16agn\anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "C:\Users\16agn\anaconda3\lib\asyncio\base_events.py", line 601, in run_forever
self._run_once()
File "C:\Users\16agn\anaconda3\lib\asyncio\base_events.py", line 1905, in _run_once
handle._run()
File "C:\Users\16agn\anaconda3\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
await self.process_one()
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
await dispatch(*args)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
await result
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 662, in execute_request
reply_content = await reply_content
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 360, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 532, in run_cell
return super().run_cell(*args, **kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2863, in run_cell
result = self._run_cell(
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2909, in _run_cell
return runner(coro)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3106, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3309, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3369, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\16agn\AppData\Local\Temp\ipykernel_33292\314088425.py", line 23, in <cell line: 23>
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1650, in fit
tmp_logs = self.train_function(iterator)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1249, in train_function
return step_function(self, iterator)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1233, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1222, in run_step
outputs = model.train_step(data)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1024, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1082, in compute_loss
return self.compiled_loss(
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\compile_utils.py", line 265, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 152, in __call__
losses = call_fn(y_true, y_pred)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 284, in call
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 1499, in mean_squared_error
y_true = tf.cast(y_true, y_pred.dtype)
Node: 'mean_squared_error/Cast'
Cast string to float is not supported
[[{{node mean_squared_error/Cast}}]] [Op:__inference_train_function_11279]
Please help me to solve my problem, I don't know how to make the program works
The error is due to a mismatch between the target data and the output layer in the model. The target data is of shape (N, 1), but the output layer of the model has shape (N,), which means the model is expecting a 1D array instead of a 2D array. You can resolve the issue by reshaping the target data to a 1D array using the reshape method from numpy:
y_train = y_train.reshape(-1,)
y_test = y_test.reshape(-1,)
Before fitting the model:
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
Due to the size of my data, I have my training, validation and test data fed via a generator as below:
def sequence_generator(data_type):
data_type = data_type.decode('ascii')
sequence_folder = f"{model_attributes['sequences_path']}/{data_type}"
for numpy_file in random.sample(os.listdir(sequence_folder),len(os.listdir(sequence_folder))):
if numpy_file.endswith(".npz"):
with np.load(f"{sequence_folder}/{numpy_file}") as numpy_data:
sequences = numpy_data['x']
labels = numpy_data['y']
labels = tf.one_hot(labels, 3)
for X,y in random.sample(list(zip(sequences,labels)),len(sequences)):
yield X,y
train_dataset = tf.data.Dataset.from_generator(generator=sequence_generator,args= ['train'], output_types = (tf.float32, tf.float32))# output_shapes = ((train_shape), (3,)))
train_dataset = train_dataset.cache().batch(BATCH).prefetch(tf.data.AUTOTUNE)
history = model.fit(train_dataset, epochs=EPOCHS, validation_data=validation_dataset, callbacks=[tensorboard,checkpoint,earlystop,lr_callback])
From my previous questions accepted answer, I want to be able to apply weighting to 2 of the 3 classes. The accepted answers code is:
sample_weight = np.ones(shape=(len(train_y),))
sample_weight[(train_y[:, 1] == 1) | (train_y[:, 2] == 1)] = 1.5
model = get_model()
model.fit(
train_x,
train_y,
sample_weight=sample_weight,
....,
...,
..,
)
But given that my training data is behind the generator code I am using, I am scratching my head on how best to apply the weighting and with minimal overhead
EDIT
I added the following code as recommended
class_weight = {0: 1.,
1: 2.,
2: 2.}
history = model.fit(train_dataset, epochs=EPOCHS, validation_data=validation_dataset,class_weight=class_weight, callbacks=[tensorboard,checkpoint,earlystop,lr_callback])
but I get this error
File "Train.py", line 1993, in build_and_train_model
history = model.fit(train_dataset, epochs=EPOCHS, validation_data=validation_dataset,class_weight=class_weight, callbacks=[tensorboard,checkpoint,earlystop,lr_callback])
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1147, in fit
steps_per_execution=self._steps_per_execution)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1364, in get_data_handler
return DataHandler(*args, **kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1175, in __init__
class_weight, distribute)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1186, in _configure_dataset_and_inferred_steps
dataset = dataset.map(_make_class_weight_map_fn(class_weight))
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 1925, in map
return MapDataset(self, map_func, preserve_cardinality=True)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 4487, in __init__
use_legacy_function=use_legacy_function)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 3712, in __init__
self._function = fn_factory()
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\eager\function.py", line 3135, in get_concrete_function
*args, **kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\eager\function.py", line 3100, in _get_concrete_function_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\eager\function.py", line 3444, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\eager\function.py", line 3289, in _create_graph_function
capture_by_value=self._capture_by_value),
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\func_graph.py", line 999, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 3687, in wrapped_fn
ret = wrapper_helper(*args)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 3617, in wrapper_helper
ret = autograph.tf_convert(self._func, ag_ctx)(*nested_args)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 692, in wrapper
return converted_call(f, args, kwargs, options=options)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 382, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 463, in _call_unconverted
return f(*args, **kwargs)
File "Location\\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py", line 1400, in _class_weights_map_fn
if y.shape.rank > 2:
TypeError: '>' not supported between instances of 'NoneType' and 'int
Not sure what is being referred to as NoneType, from the research I read I get the inclination that it has something to do with one_hotting maybe?
This looks a great moment to use class_weight instead of sample_weight:
Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class.
I ended up doing this by modifying the generator:
train_dataset = tf.data.Dataset.from_generator(generator=sequence_generator,args= ['train'], output_types = (tf.float32, tf.float32, tf.float32))# output_shapes = ((train_shape), (3,)))
def sequence_generator(data_type):
data_type = data_type.decode('ascii')
sequence_folder = f"{model_attributes['sequences_path']}/{data_type}"
for numpy_file in random.sample(os.listdir(sequence_folder),len(os.listdir(sequence_folder))):
if numpy_file.endswith(".npz"):
with np.load(f"{sequence_folder}/{numpy_file}") as numpy_data:
sequences = numpy_data['x']
labels = numpy_data['y']
class_weight = {0: 1, 1: 2 ,2: 2}
labels = tf.one_hot(labels, 3)
sample_weights = np.ones(shape=(len(labels),))
sample_weights[(labels[:, 1] == 1) | (labels[:, 2] == 1) ] = 2
for X,y,sample_weight in random.sample(list(zip(sequences,labels,sample_weights)),len(sequences)):
yield X,y,sample_weight
The class_weights method did not seem to work. This article seemed to shed some light on it
I have created a transfer learning model using Resnet50. I want to perform K-fold cross-validation on my model after which I want to find the average AUC value and standard deviation. However, I am getting an error message while performing the task. I have created a separate Files.csv file which contains the image names and their corresponding labels. I am not sure if this is the correct method or not. Please let me know if there is any other process. Please find my code below:
from tensorflow.keras.applications import ResNet50
from tensorflow.keras.applications.resnet50 import preprocess_input
from tensorflow.keras import Model, layers
from tensorflow.keras.models import load_model, model_from_json
from tensorflow.keras.layers import GlobalAveragePooling2D, Dropout, Dense, Input
import numpy as np
import pandas as pd
import os
from sklearn.model_selection import KFold, StratifiedKFold
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
train_data = pd.read_csv('Files.csv')
Y = train_data[['label']]
kf = KFold(n_splits = 5)
from tensorflow.keras.preprocessing.image import ImageDataGenerator
idg = ImageDataGenerator(rescale = 1./255,
horizontal_flip=True,
rotation_range=40,
zoom_range= 0.2,
shear_range=0.2,
width_shift_range=0.2,
height_shift_range=0.2,)
validation_datagen = ImageDataGenerator(rescale = 1./255)
def get_model_name(k):
return 'model_'+str(k)+'.h5'
from keras import models
from keras.layers import Dense, Flatten
from tensorflow.keras import optimizers
from keras.applications.vgg16 import VGG16
from tensorflow.keras.applications import ResNet50
image_dir=r'D:/regionGrowing_MLT/NewSavedRGBImages/Training'
VALIDATION_ACCURACY = []
VALIDAITON_LOSS = []
save_dir = 'C:/Users/warid'
fold_var = 1
for train_index, val_index in kf.split(np.zeros(n),Y):
training_data = train_data.iloc[train_index]
validation_data = train_data.iloc[val_index]
train_data_generator = idg.flow_from_dataframe(training_data, directory = image_dir,
x_col = "filename", y_col = "label",
class_mode = "categorical", shuffle = True)
valid_data_generator = idg.flow_from_dataframe(validation_data, directory = image_dir,
x_col = "filename", y_col = "label",
class_mode = "categorical", shuffle = True)
# CREATE NEW MODEL
model = models.Sequential()
model.add(ResNet50(weights='imagenet', include_top=False, input_shape=(224,224,3)))
model.add(Flatten())
model.add(ChannelAttention(32, 8))
model.add(SpatialAttention(7))
model.add(Dense(256, activation='relu', name='fc1'))
model.add(Dense(128, activation='relu', name='fc2'))
model.add(layers.Dropout(0.5)) #### used for regularization (to aviod overfitting)
model.add(Dense(2, activation='sigmoid'))
# model.summary()
model.compile(optimizer=optimizers.Adam(learning_rate=2e-5),
loss='binary_crossentropy',
metrics=['accuracy'])
# COMPILE NEW MODEL
# CREATE CALLBACKS
checkpoint = tf.keras.callbacks.ModelCheckpoint(save_dir+get_model_name(fold_var),
monitor='val_accuracy', verbose=1,
save_best_only=True, mode='max')
callbacks_list = [checkpoint]
# There can be other callbacks, but just showing one because it involves the model name
# This saves the best model
# FIT THE MODEL
history = model.fit(train_data_generator,
epochs=num_epochs,
callbacks=callbacks_list,
validation_data=valid_data_generator)
#PLOT HISTORY
# :
# :
# LOAD BEST MODEL to evaluate the performance of the model
model.load_weights("/saved_models/model_"+str(fold_var)+".h5")
results = model.evaluate(valid_data_generator)
results = dict(zip(model.metrics_names,results))
VALIDATION_ACCURACY.append(results['accuracy'])
VALIDATION_LOSS.append(results['loss'])
tf.keras.backend.clear_session()
fold_var += 1
After running this code, I am getting the following error message:
Found 3076 validated image filenames belonging to 2 classes.
Found 769 validated image filenames belonging to 1 classes.
Epoch 1/5
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
Input In [75], in <cell line: 7>()
39 callbacks_list = [checkpoint]
40 # There can be other callbacks, but just showing one because it involves the model name
41 # This saves the best model
42 # FIT THE MODEL
---> 43 history = model.fit(train_data_generator,
44 epochs=num_epochs,
45 callbacks=callbacks_list,
46 validation_data=valid_data_generator)
47 #PLOT HISTORY
48 # :
49 # :
50
51 # LOAD BEST MODEL to evaluate the performance of the model
52 model.load_weights("/saved_models/model_"+str(fold_var)+".h5")
File ~\anaconda3\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'sequential_2/flatten_2/Reshape' defined at (most recent call last):
File "C:\Users\warid\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\warid\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "C:\Users\warid\anaconda3\lib\site-packages\traitlets\config\application.py", line 846, in launch_instance
app.start()
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 677, in start
self.io_loop.start()
File "C:\Users\warid\anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "C:\Users\warid\anaconda3\lib\asyncio\base_events.py", line 601, in run_forever
self._run_once()
File "C:\Users\warid\anaconda3\lib\asyncio\base_events.py", line 1905, in _run_once
handle._run()
File "C:\Users\warid\anaconda3\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
await self.process_one()
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
await dispatch(*args)
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
await result
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 662, in execute_request
reply_content = await reply_content
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 360, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "C:\Users\warid\anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 532, in run_cell
return super().run_cell(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2863, in run_cell
result = self._run_cell(
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2909, in _run_cell
return runner(coro)
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3106, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3309, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "C:\Users\warid\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3369, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\warid\AppData\Local\Temp\ipykernel_37076\2928028949.py", line 43, in <cell line: 7>
history = model.fit(train_data_generator,
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 1409, in fit
tmp_logs = self.train_function(iterator)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 1051, in train_function
return step_function(self, iterator)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 1040, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 1030, in run_step
outputs = model.train_step(data)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 889, in train_step
y_pred = self(x, training=True)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\training.py", line 490, in __call__
return super().__call__(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\sequential.py", line 374, in call
return super(Sequential, self).call(inputs, training=training, mask=mask)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\functional.py", line 458, in call
return self._run_internal_graph(
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\functional.py", line 596, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\engine\base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "C:\Users\warid\anaconda3\lib\site-packages\keras\layers\reshaping\flatten.py", line 98, in call
return tf.reshape(inputs, flattened_shape)
Node: 'sequential_2/flatten_2/Reshape'
Input to reshape is a tensor with 4194304 values, but the requested shape requires a multiple of 100352
[[{{node sequential_2/flatten_2/Reshape}}]] [Op:__inference_train_function_37893]
I am following the YOLO v1 paper to create an object detector from scratch with Tensorflow and python. My dataset is a set of images and a 7x7x12 Tensor that represents the label for the image. I import the image names and labels (as a string) into a dataframe from a CSV using pandas, and then do some operations on the labels to turn them into Tensors. I then create a generator using ImageGenerator.flow_from_dataframe(), and the feed that generator as the input for my model. I end up getting the following error when the model tries to call the custom loss function that I created:
File "/home/michael/Desktop/YOLO_Detector/src/main.py", line 61, in <module>
epochs=10)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 342, in fit
total_epochs=epochs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 128, in run_one_epoch
batch_outs = execution_function(iterator)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 98, in execution_function
distributed_function(input_fn))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in __call__
result = self._call(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
*args, **kwds))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 85, in distributed_function
per_replica_function, args=args)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 763, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1819, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 2164, in _call_for_each_replica
return fn(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py", line 292, in wrapper
return func(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 433, in train_on_batch
output_loss_metrics=model._output_loss_metrics)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 312, in train_on_batch
output_loss_metrics=output_loss_metrics))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 253, in _process_single_batch
training=training))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 167, in _model_loss
per_sample_losses = loss_fn.call(targets[i], outs[i])
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/losses.py", line 221, in call
return self.fn(y_true, y_pred, **self._fn_kwargs)
File "/home/michael/Desktop/YOLO_Detector/src/utils.py", line 25, in yolo_loss_function
pos = kb.mean(y_true-y_pred)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/math_ops.py", line 902, in binary_op_wrapper
return func(x, y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 10104, in sub
"Sub", x=x, y=y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 576, in _apply_op_helper
param_name=input_name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 61, in _SatisfiesTypeConstraint
", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'x' has DataType string not in list of allowed values: bfloat16, float16, float32, float64, uint8, int8, uint16, int16, int32, int64, complex64, complex128
When I use the python debugger to check the y_true that is being passed to the loss function, I see the following tensor:
Tensor("IteratorGetNext:1", shape=(None, 1), dtype=string)
however when I manually check the label of the image by calling the following python code, I get a tensor with the correct shape and values:
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
-------------Output--------------
<class 'tensorflow.python.framework.ops.EagerTensor'>
(7, 7, 12)
<dtype: 'float32'>
Below is the code that I am using to create the dataset and train the model:
import tensorflow as tf
from tensorflow import keras
import pandas as pd
import matplotlib.pyplot as plt
import cv2
import numpy as np
import os
from src import utils, model
path = "/home/michael/Desktop/YOLO_Detector/dataset/labels.csv"
train_df = pd.read_csv(path, delim_whitespace=True, header=None)
train_df.columns = ['filename', 'output_tensor']
train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: utils.string_to_tensor(x))
# train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: tf.expand_dims(x, 3))
image_generator = keras.preprocessing.image.ImageDataGenerator(rescale=1. / 255.,
validation_split=0.2)
train_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode='raw',
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="training"
)
validation_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode="raw",
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="validation"
)
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
model = model.create_model()
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate=1e-2,
decay_rate=0.0005,
decay_steps=100000
)
sgd = keras.optimizers.SGD(learning_rate=lr_schedule, momentum=0.9)
model.compile(optimizer=sgd, loss=utils.yolo_loss_function, metrics=['accuracy'])
model.fit(x=train_gen,
epochs=10)
I am using Tensorflow 2 with ROCm, and Eager Execution is on. How do I get the y_true tensor to be the correct output label (7x7x12 Tensor with dtype=float32) instead of it being a string?
I figured out the issue. The issue is that you cannot correctly store a Tensor or a Numpy array in a Pandas Dataframe. I ended up having to manually create the image/tensor pairs by doing the following:
img_list = []
labels_list = []
for i in range(64):
labels_list.append(utils.string_to_numpy(train_df["output_tensor"][i]))
image = tf.keras.preprocessing.image.load_img(f"/home/michael/Desktop/YOLO_Detector/dataset/{train_df['filename'][i]}", target_size=(448, 448))
image_arr = keras.preprocessing.image.img_to_array(image) / 255.0
img_list.append(image_arr)
img = np.asarray(img_list)
label = np.asarray(labels_list)
and then calling the img as x and label as y in model.fit()
I'm using TensorFlow 2.0 Datasets to feed my model's fit function. Here is the code:
def build_model(self):
self.g_Model = Sequential()
self.g_Model.add(Embedding(self.g_Max_features, output_dim=256))
self.g_Model.add(LSTM(128))
self.g_Model.add(Dropout(0.5))
self.g_Model.add(Dense(1, activation='sigmoid'))
self.g_Model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
def train_model(self, filenames):
lstm_feature_description = {
'X': tf.io.FixedLenFeature(CONFIG.g_keras_lstm_max_document_length, tf.float32),
'y': tf.io.FixedLenFeature((), tf.int64),
}
def _parse_lstm_function(example_proto):
return tf.io.parse_single_example(serialized=example_proto, features=lstm_feature_description)
self.build_model()
# Start Preparing The Data
raw_lstm_dataset = tf.data.TFRecordDataset(CONFIG.g_record_file_lstm)
parsed_lstm_dataset = raw_lstm_dataset.map(_parse_lstm_function)
parsed_lstm_dataset = parsed_lstm_dataset.shuffle(CONFIG.g_shuffle_s).batch(CONFIG.g_Batch_size)
self.g_Model.fit(parsed_lstm_dataset, epochs=2)
But I receive the following error:
Traceback (most recent call last):
File "keras_lstm_v2.py", line 79, in train_model
1/Unknown - 0s 0s/step self.g_Model.fit(parsed_lstm_dataset, epochs=2)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 324, in fit
total_epochs=epochs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 503, in _call
self._initialize(args, kwds, add_initializers_to=initializer_map)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 408, in _initialize
*args, **kwds))
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 1848, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 2150, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\function.py", line 2041, in _create_graph_function
capture_by_value=self._capture_by_value),
File "venv_tf_new\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 358, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 66, in distributed_function
model, input_iterator, mode)
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 118, in _prepare_feed_values
inputs = [inputs[key] for key in model._feed_input_names]
File "venv_tf_new\lib\site-packages\tensorflow_core\python\keras\engine\training_v2_utils.py", line 118, in <listcomp>
inputs = [inputs[key] for key in model._feed_input_names]
KeyError: 'embedding_input'
I've seen this thread, however it doesn't clarify things up for me. As far as I understood there is a problem with the loaded data, but according to documentation for Datasets it should work out of the box, so I couldn't figure out how to fix it.
Any help is appreciated. Thanks!
You need to declare your model's inputs somewhere, typically something like
model = tf.keras.Model(inputs=inputs, outputs=outputs)
Try taking out the last line in your model building function
self.g_Model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
and move it to your train function, once you have declared your model you can use the output of the model as your output layer and declare your input layer with something like
input_size = CONFIG.g_keras_lstm_max_document_length
input_layer = tf.keras.layers.Input(input_size)
output_layer = self.build_model()
model = tf.keras.Model(inputs=input_layer, outputs=output_layer )
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit( .... )
I found the solution myself today. Actually, it was two errors in my code:
The .fit() function could not determine where inputs and labels are as noted by #NiallJG above. However, the solution provided didn't solve the problem, so I fixed it up in the following way:
1.1 In my build_model function I've added "name" to Embedding layer:
self.g_Model.add(Embedding(input_dim=self.g_Max_features, output_dim=256, name='X'))
1.2 To match this name I've actually needed to change my lstm_feature_description, so it contains "_input" postfix:
def train_model(self, filenames):
lstm_feature_description = {
'X_input': tf.io.FixedLenFeature(CONFIG.g_keras_lstm_max_document_length, tf.float32),
'y': tf.io.FixedLenFeature((), tf.int64),
}
My _parse_lstm_function was returning data to the Dataset in the
wrong manner, causing "IndexError: list index out of range" error. Here is how the modified function should look like:
def _parse_lstm_function(example_proto):
# Parse the input tf.Example proto using the dictionary above.
parsed = tf.io.parse_single_example(serialized=example_proto, features=lstm_feature_description)
return parsed["X_input"], parsed["y"]
This allowed the model to .fit() correctly, except now I've left with OOM error, but this will be covered in another question.