I'm trying to get an mnist cnn working so as to do predictions on one image at a time. I have taken the tensorflow tutorial code and am trying to use estimator.predict with the model but am currently getting the error:
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 1 values, but the requested shape requires a multiple of 784
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](fifo_queue_DequeueUpTo/_53, Reshape/shape)]]
If I print the predict_data list given to the predict input function it contains 784 elements.
The model trained OK and evaluated OK.
The model is already trained so I have skipped the training code here but this is what I have:
def main(unused_argv):
# Load training and eval data
mnist = tf.contrib.learn.datasets.load_dataset("mnist")
train_data = mnist.train.images # Returns np.array
train_labels = np.asarray(mnist.train.labels, dtype=np.int32)
eval_data = mnist.test.images # Returns np.array
eval_labels = np.asarray(mnist.test.labels, dtype=np.int32)
# Create the Estimator
mnist_classifier = tf.estimator.Estimator(
model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model")
# Set up logging for predictions
# Log the values in the "Softmax" tensor with label "probabilities"
tensors_to_log = {"probabilities": "softmax_tensor"}
logging_hook = tf.train.LoggingTensorHook(
tensors=tensors_to_log, every_n_iter=50)
# # Train the model
# train_input_fn = tf.estimator.inputs.numpy_input_fn(
# x={"x": train_data},
# y=train_labels,
# batch_size=100,
# num_epochs=None,
# shuffle=True)
# mnist_classifier.train(
# input_fn=train_input_fn,
# steps=20000,
# hooks=[logging_hook])
# Evaluate the model and print results
# eval_input_fn = tf.estimator.inputs.numpy_input_fn(
# x={"x": eval_data},
# y=eval_labels,
# num_epochs=1,
# shuffle=False)
# eval_results = mnist_classifier.evaluate(input_fn=eval_input_fn)
# print(eval_results)
predict_data = eval_data[1]
predict_input_fn = tf.estimator.inputs.numpy_input_fn(
x={"x": predict_data},
y=None,
batch_size=1,
num_epochs=1,
shuffle=False,
num_threads=1)
predict_results = mnist_classifier.predict(predict_input_fn)
print(predict_data)
for idx, prediction in enumerate(predict_results):
print(idx)
# print(prediction)
Any help getting this working would be greatly appreciated.
Update: I tried reshape as suggested below, but get the same error. The full trace is:
Traceback (most recent call last):
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1323, in _do_call
return fn(*args)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1302, in _run_fn
status, run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 1 values, but the requested shape requires a multiple of 784
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](fifo_queue_DequeueUpTo/_53, Reshape/shape)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Workspace\eclipse\mnist_cnn\cnn_mnist.py", line 180, in <module>
tf.app.run()
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "D:\Workspace\eclipse\mnist_cnn\cnn_mnist.py", line 170, in main
for idx, prediction in enumerate(predict_results):
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\estimator\estimator.py", line 420, in predict
preds_evaluated = mon_sess.run(predictions)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 521, in run
run_metadata=run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 892, in run
run_metadata=run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 967, in run
raise six.reraise(*original_exc_info)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\six.py", line 693, in reraise
raise value
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 952, in run
return self._sess.run(*args, **kwargs)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1024, in run
run_metadata=run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\training\monitored_session.py", line 827, in run
return self._sess.run(*args, **kwargs)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 889, in run
run_metadata_ptr)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1317, in _do_run
options, run_metadata)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\client\session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 1 values, but the requested shape requires a multiple of 784
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](fifo_queue_DequeueUpTo/_53, Reshape/shape)]]
Caused by op 'Reshape', defined at:
File "D:\Workspace\eclipse\mnist_cnn\cnn_mnist.py", line 180, in <module>
tf.app.run()
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "D:\Workspace\eclipse\mnist_cnn\cnn_mnist.py", line 170, in main
for idx, prediction in enumerate(predict_results):
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\estimator\estimator.py", line 411, in predict
features, None, model_fn_lib.ModeKeys.PREDICT, self.config)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\estimator\estimator.py", line 694, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "D:\Workspace\eclipse\mnist_cnn\cnn_mnist.py", line 31, in cnn_model_fn
input_layer = tf.reshape(features["x"], [-1, 28, 28, 1])
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 3937, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\ops.py", line 2956, in create_op
op_def=op_def)
File "C:\Users\artma\Miniconda3\envs\vpilot\lib\site-packages\tensorflow\python\framework\ops.py", line 1470, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 1 values, but the requested shape requires a multiple of 784
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](fifo_queue_DequeueUpTo/_53, Reshape/shape)]]
Update: Seems to have cracked it. Thanks to xdurch0 for putting me on the right track.
predict_data is just a 784-element vector. This will be treated as a dataset containing 784 elements (i.e. distinct inputs), each of which is a scalar. You need to reshape your predict_data to (1, 784) to let TF know this is a dataset with one element, which is a 784-element vector. E.g. predict_data[np.newaxis, :] or predict_data.reshape((1, 784)).
It seems the line:
predict_data = eval_data[1]
Was generating an np array of (784,), which even predict_data.reshape((1,784)) would not fix.
Whereas:
predict_data = eval_data[1:2]
Produces an np array of (1,784) which estimator.predict is now happy with.
Related
So, I was doing my code like the one below. I want to Fit the ANN into the Training set, but the error occurred like this. I was confused about how to solve it, I already try several suggestions on Google, but it still came out an error. I also tried several codes to display the result, but most of the errors occurred is because of the fitting like this one. So, I was thinking that the main problem my code can't run is because of the fitting model.
#Importing necessary Libraries
import numpy as np
import pandas as pd
import tensorflow as tf
import keras
#dataset
data = pd.read_excel("E:\\MATKUL LUV\\THESIS\\DATASETS\\DPM1 1052.xlsx")
#print (data.head)
# Separate Target Variable and Predictor Variables
TargetVariable=['DPM1Fault']
Predictors=['DPM1Cx', 'DPM1Cy']
X=data[Predictors].values
y=data[TargetVariable].values
# Split the data into training and testing set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# Quick sanity check with the shapes of Training and testing datasets
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
# importing the libraries
from keras.models import Sequential
from keras.layers import Dense
# create ANN model
model = Sequential()
# Defining the Input layer and FIRST hidden layer, both are same!
model.add(Dense(units=5, input_dim=2, kernel_initializer='normal', activation='relu'))
# Defining the Second layer of the model
# after the first layer we don't have to specify input_dim as keras configure it automatically
model.add(Dense(units=4, kernel_initializer='normal', activation='relu'))
# The output neuron is a single fully connected node
# Since we will be predicting a single number
model.add(Dense(1, kernel_initializer='normal'))
# Compiling the model
model.compile(loss='mean_squared_error', optimizer='adam')
# Fitting the ANN to the Training set
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
and the result like this
Epoch 1/50
---------------------------------------------------------------------------
UnimplementedError Traceback (most recent call last)
Input In [52], in <cell line: 23>()
20 model.compile(loss='mean_squared_error', optimizer='adam')
22 # Fitting the ANN to the Training set
---> 23 model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
File ~\anaconda3\lib\site-packages\keras\utils\traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
File ~\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py:52, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
50 try:
51 ctx.ensure_initialized()
---> 52 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
53 inputs, attrs, num_outputs)
54 except core._NotOkStatusException as e:
55 if name is not None:
UnimplementedError: Graph execution error:
Detected at node 'mean_squared_error/Cast' defined at (most recent call last):
File "C:\Users\16agn\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\16agn\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "C:\Users\16agn\anaconda3\lib\site-packages\traitlets\config\application.py", line 846, in launch_instance
app.start()
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 677, in start
self.io_loop.start()
File "C:\Users\16agn\anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "C:\Users\16agn\anaconda3\lib\asyncio\base_events.py", line 601, in run_forever
self._run_once()
File "C:\Users\16agn\anaconda3\lib\asyncio\base_events.py", line 1905, in _run_once
handle._run()
File "C:\Users\16agn\anaconda3\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
await self.process_one()
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
await dispatch(*args)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
await result
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 662, in execute_request
reply_content = await reply_content
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 360, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "C:\Users\16agn\anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 532, in run_cell
return super().run_cell(*args, **kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2863, in run_cell
result = self._run_cell(
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2909, in _run_cell
return runner(coro)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3106, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3309, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "C:\Users\16agn\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3369, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\16agn\AppData\Local\Temp\ipykernel_33292\314088425.py", line 23, in <cell line: 23>
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler
return fn(*args, **kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1650, in fit
tmp_logs = self.train_function(iterator)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1249, in train_function
return step_function(self, iterator)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1233, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1222, in run_step
outputs = model.train_step(data)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1024, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\training.py", line 1082, in compute_loss
return self.compiled_loss(
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\engine\compile_utils.py", line 265, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 152, in __call__
losses = call_fn(y_true, y_pred)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 284, in call
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "C:\Users\16agn\anaconda3\lib\site-packages\keras\losses.py", line 1499, in mean_squared_error
y_true = tf.cast(y_true, y_pred.dtype)
Node: 'mean_squared_error/Cast'
Cast string to float is not supported
[[{{node mean_squared_error/Cast}}]] [Op:__inference_train_function_11279]
Please help me to solve my problem, I don't know how to make the program works
The error is due to a mismatch between the target data and the output layer in the model. The target data is of shape (N, 1), but the output layer of the model has shape (N,), which means the model is expecting a 1D array instead of a 2D array. You can resolve the issue by reshaping the target data to a 1D array using the reshape method from numpy:
y_train = y_train.reshape(-1,)
y_test = y_test.reshape(-1,)
Before fitting the model:
model.fit(X_train, y_train ,batch_size = 20, epochs = 50, verbose=1)
I am trying to train a model using the TPU Estimator API on Cloud TPU. The error logs, and the code for reading my input data are attached below. I tried using the python debugger to determine where the bug is encountered. The control doesn't go out of the traing_input_fn function before the error is encountered. So, I believe my data pipeline is the source of the problem. Can someone please help me out with this problem? I will be happy to provide any more information, if necessary. Thanks
INFO:tensorflow:Error recorded from training_loop: The features to the model returned by input_fn must have static shape. Tensor: Tensor("Inf[25/1805]
dequeue:0", shape=(16, ?, 50, 1024), dtype=float32, device=/device:TPU_REPLICATED_CORE:0)
INFO:tensorflow:training_loop marked as finished
WARNING:tensorflow:Reraising captured error
Traceback (most recent call last):
File "estimator_task.py", line 303, in <module>
main(**arguments)
File "estimator_task.py", line 261, in main
estimator.train(input_fn=train_input_fn, max_steps=train_steps, hooks=hooks)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2457, in train
rendezvous.raise_errors()
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/error_handling.py", line 128, in raise_error$
six.reraise(typ, value, traceback)
File "/home/abi/.local/lib/python3.5/site-packages/six.py", line 693, in reraise
raise value
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2452, in train
saving_listeners=saving_listeners)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 358, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1124, in _train_mode$
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1154, in _train_mode$
_default
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2251, in _call_model$
fn
config)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1112, in _call_model$
fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2558, in _model_fn
_train_on_tpu_system(ctx, model_fn_wrapper, dequeue_fn))
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2893, in _train_on_t$
u_system
device_assignment=ctx.device_assignment)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu.py", line 890, in split_compile_and_shar$
name=name)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu.py", line 689, in split_compile_and_repl$
cate
outputs = computation(*computation_inputs)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2886, in multi_tpu_t$
ain_steps_on_single_shard
[_INITIAL_LOSS])
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/training_loop.py", line 208, in repeat
cond, body_wrapper, inputs=inputs, infeed_queue=infeed_queue, name=name)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/training_loop.py", line 170, in while_loop
condition_wrapper, body_wrapper, inputs, name="", parallel_iterations=1)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3556, in while_loop
return_same_structure)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3087, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3022, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/training_loop.py", line 121, in body_wrapper
outputs = body(*(inputs + dequeue_ops))
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/training_loop.py", line 204, in body_wrapper
return [i + 1] + _convert_to_list(body(*args))
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1359, in train_step
self._call_model_fn(features, labels))
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1552, in _call_model_
fn
self._validate_model_features_and_labels(features, labels, is_export_mode)
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1546, in _validate_mo
del_features_and_labels
validate(features, 'features')
File "/home/abi/anaconda3/envs/myenv_3_5/lib/python3.5/site-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1538, in validate
' Tensor: {}'.format(obj_name, obj))
ValueError: The features to the model returned by input_fn must have static shape. Tensor: Tensor("InfeedQueue/dequeue:0", shape=(16, ?, 50, 1024), dt
ype=float32, device=/device:TPU_REPLICATED_CORE:0)
This is my training data pipeline
def train_input_fn(params):
def decode_example(example_proto, t=50, dim=1024):
features = tf.parse_single_example(
example_proto,
features = {
'X': tf.FixedLenSequenceFeature([], tf.float32, allow_missing=True),
'Y': tf.FixedLenSequenceFeature([], tf.int64, allow_missing=True),
}
)
feat = features['X']
feat = tf.squeeze(feat)
feat.set_shape([t, dim])
labels = features['Y']
labels = tf.cast(labels, dtype=tf.int32)
return feat, labels
train_files = params["train_filenames"]
batch_size = params['batch_size']
dataset = tf.data.TFRecordDataset(train_files, num_parallel_reads=8)
dataset = dataset.apply(
tf.contrib.data.shuffle_and_repeat(buffer_size=100))
dataset = dataset.apply(
tf.contrib.data.map_and_batch(decode_example, batch_size, drop_remainder=False))
dataset = dataset.prefetch(1)
dataset = dataset.apply(tf.contrib.data.batch_and_drop_remainder(batch_size))
return dataset
I had a very similar problem and I solved it by setting the shape dimensions with an explicit value, in your case:
feat.set_shape([50, 1024])
Not very convenient, but it worked for me.
I just run a simple code and want to get accuracy after training. I load the model that I saved, but when I want to get accuracy, I get something wrong. Why?
# coding=utf-8
from color_1 import read_and_decode, get_batch, get_test_batch
import AlexNet
import cv2
import os
import time
import numpy as np
import tensorflow as tf
import AlexNet_train
import math
batch_size=128
num_examples = 1000
crop_size=56
def evaluate(test_x, test_y):
image_holder = tf.placeholder(tf.float32, [batch_size, 56, 56, 3], name='x-input')
label_holder = tf.placeholder(tf.int32, [batch_size], name='y-input')
y = AlexNet.inference(image_holder,evaluate,None)
correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(label_holder,1))
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
saver = tf.train.Saver()
with tf.Session() as sess:
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
coord = tf.train.Coordinator()
sess.run(init_op)
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
ckpt=tf.train.get_checkpoint_state(AlexNet_train.MODEL_SAVE_PATH)
if ckpt and ckpt.model_checkpoint_path:
ckpt_name = os.path.basename(ckpt.model_checkpoint_path)
global_step = ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1]
saver.restore(sess, os.path.join(AlexNet_train.MODEL_SAVE_PATH, ckpt_name))
print('Loading success, global_step is %s' % global_step)
step=0
image_batch, label_batch = sess.run([test_x, test_y])
accuracy_score=sess.run(accuracy,feed_dict={image_holder: image_batch,
label_holder: label_batch})
print("After %s training step(s),validation "
"precision=%g" % (global_step, accuracy_score))
coord.request_stop()
coord.join(threads)
def main(argv=None):
test_image, test_label = read_and_decode('val.tfrecords')
test_images, test_labels = get_test_batch(test_image, test_label, batch_size, crop_size)
evaluate(test_images, test_labels)
if __name__=='__main__':
tf.app.run()
And here is error,it said that this line in my code is wrong:" correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(label_holder,1))"
Traceback (most recent call last):
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 80, in <module>
tf.app.run()
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 76, in main
evaluate(test_images, test_labels)
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 45, in evaluate
label_holder: label_batch})
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected dimension in the range [-1, 1), but got 1
[[Node: ArgMax_1 = ArgMax[T=DT_INT32, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_y-input_0, ArgMax_1/dimension)]]
Caused by op u'ArgMax_1', defined at:
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 80, in <module>
tf.app.run()
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 76, in main
evaluate(test_images, test_labels)
File "/home/vrview/tensorflow/example/char/tfrecords/AlexNet/Alex_save/AlexNet_test.py", line 22, in evaluate
correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(label_holder,1))
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 263, in argmax
return gen_math_ops.arg_max(input, axis, name)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 168, in arg_max
name=name)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/vrview/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): Expected dimension in the range [-1, 1), but got 1
[[Node: ArgMax_1 = ArgMax[T=DT_INT32, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_y-input_0, ArgMax_1/dimension)]]
How to solve it?
Taking the part of this answer related to the problem here:
tf.argmax's definition states:
axis: A Tensor. Must be one of the following types: int32, int64.
int32, 0 <= axis < rank(input). Describes which axis of the input
Tensor to reduce across.
It seems, then, that the only way to run argmax on the last axis of the tensor is by giving it axis=-1, because of the "strictly less than" sign in the definition of the function.
I am implementing a convolutional autoencoder that has to reduce the dimension of camera images per image at real time. Therefore my batch size is one. Y_pred is the output of the network. I want to evaluate this so I can see what my network is outputting.
This is the code I use to import my data and feed the placeholders:
def inputs(image_file_path):
filenames = [image_file_path]
filename_queue = tf.train.string_input_producer(filenames)
read_input = read_image(filename_queue)
return read_input
with tf.Session() as sess:
image = inputs(file_path)
coord = tf.train.Coordinator()
init = tf.global_variables_initializer()
sess.run(init)
threads = tf.train.start_queue_runners(sess=sess, coord = coord)
image = tf.cast(image/255, tf.float32)
###data is just one image, so I set the dimension [batch, height, width, channels] to [1,120,160,3]
image = tf.reshape(image, [1, 120,160,3])
X_data = image.eval()
sess.run(train_step,{X_input:X_data, Y_true: X_data})
output = Y_pred
###Everything works fine up until this point.
print(Y_pred.eval())
coord.request_stop()
coord.join(threads)
Then the placeholders are here:
X_input = tf.placeholder(tf.float32, [None,120,160,3])
Y_true = tf.placeholder(tf.float32, [None,120,160,3])
The error I'm getting:
Traceback (most recent call last):
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1039, in _do_call
return fn(*args)
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1021, in _run_fn
status, run_metadata)
File "C:\Python35\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Python35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float
[[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Julian\workspaceNeon\Scriptie\Autoencoder\__init__.py", line 164, in <module>
print(output.eval())
File "C:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 569, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "C:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 3741, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 778, in run
run_metadata_ptr)
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "C:\Python35\lib\site-packages\tensorflow\python\client\session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float
[[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op 'Placeholder', defined at:
File "C:\Users\Julian\workspaceNeon\Scriptie\Autoencoder\__init__.py", line 50, in <module>
X_input = tf.placeholder(tf.float32, [None,120,160,3])
File "C:\Python35\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1507, in placeholder
name=name)
File "C:\Python35\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 1997, in _placeholder
name=name)
File "C:\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 768, in apply_op
op_def=op_def)
File "C:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1228, in __init__
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype float
[[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
tf.contrib.crf doesn't seem to support sequences of length 1.
For example, if I run the example on https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/crf (mirror) and replace num_words = 20 by num_words = 1:
import numpy as np
import tensorflow as tf
# Data settings.
num_examples = 10
num_words = 1
num_features = 100
num_tags = 5
# Random features.
x = np.random.rand(num_examples, num_words, num_features).astype(np.float32)
# Random tag indices representing the gold sequence.
y = np.random.randint(num_tags, size=[num_examples, num_words]).astype(np.int32)
# All sequences in this example have the same length, but they can be variable in a real model.
sequence_lengths = np.full(num_examples, num_words - 1, dtype=np.int32)
# Train and evaluate the model.
with tf.Graph().as_default():
with tf.Session() as session:
# Add the data to the TensorFlow graph.
x_t = tf.constant(x)
y_t = tf.constant(y)
sequence_lengths_t = tf.constant(sequence_lengths)
# Compute unary scores from a linear layer.
weights = tf.get_variable("weights", [num_features, num_tags])
matricized_x_t = tf.reshape(x_t, [-1, num_features])
matricized_unary_scores = tf.matmul(matricized_x_t, weights)
unary_scores = tf.reshape(matricized_unary_scores,
[num_examples, num_words, num_tags])
# Compute the log-likelihood of the gold sequences and keep the transition
# params for inference at test time.
log_likelihood, transition_params = tf.contrib.crf.crf_log_likelihood(
unary_scores, y_t, sequence_lengths_t)
# Add a training op to tune the parameters.
loss = tf.reduce_mean(-log_likelihood)
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
# Train for a fixed number of iterations.
session.run(tf.global_variables_initializer())
for i in range(1000):
tf_unary_scores, tf_transition_params, _ = session.run(
[unary_scores, transition_params, train_op])
if i % 100 == 0:
correct_labels = 0
total_labels = 0
for tf_unary_scores_, y_, sequence_length_ in zip(tf_unary_scores, y,
sequence_lengths):
# Remove padding from the scores and tag sequence.
tf_unary_scores_ = tf_unary_scores_[:sequence_length_]
y_ = y_[:sequence_length_]
# Compute the highest scoring sequence.
viterbi_sequence, _ = tf.contrib.crf.viterbi_decode(
tf_unary_scores_, tf_transition_params)
# Evaluate word-level accuracy.
correct_labels += np.sum(np.equal(viterbi_sequence, y_))
total_labels += sequence_length_
accuracy = 100.0 * correct_labels / float(total_labels)
print("Accuracy: %.2f%%" % accuracy)
I get the error message:
Traceback (most recent call last):
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1022, in _do_call
return fn(*args)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1004, in _run_fn
status, run_metadata)
File "C:\Anaconda\envs\py35\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:#rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 47, in <module>
[unary_scores, transition_params, train_op])
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
run_metadata_ptr)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\client\session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:#rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
Caused by op 'gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3', defined at:
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 41, in <module>
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\training\optimizer.py", line 288, in minimize
grad_loss=grad_loss)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\training\optimizer.py", line 354, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gradients_impl.py", line 482, in gradients
in_grads = grad_fn(op, *out_grads)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_grad.py", line 186, in _TensorArrayScatterGrad
grad = g.gather(indices)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 348, in gather
element_shape=element_shape)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2226, in _tensor_array_gather_v3
element_shape=element_shape, name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
op_def=op_def)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 2395, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\ops.py", line 1264, in __init__
self._traceback = _extract_stack()
...which was originally created as op 'rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3', defined at:
File "C:\Users\Francky\Documents\GitHub\nlp\neurodeid\test\CRF_v2.py", line 37, in <module>
unary_scores, y_t, sequence_lengths_t)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\contrib\crf\python\ops\crf.py", line 156, in crf_log_likelihood
log_norm = crf_log_norm(inputs, sequence_lengths, transition_params)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\contrib\crf\python\ops\crf.py", line 123, in crf_log_norm
dtype=dtypes.float32)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 545, in dynamic_rnn
dtype=dtype)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 663, in _dynamic_rnn_loop
for ta, input_ in zip(input_ta, flat_input))
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py", line 663, in <genexpr>
for ta, input_ in zip(input_ta, flat_input))
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 400, in unstack
indices=math_ops.range(0, num_elements), value=value, name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\tensor_array_ops.py", line 428, in scatter
name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 2492, in _tensor_array_scatter_v3
name=name)
File "C:\Anaconda\envs\py35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
op_def=op_def)
UnimplementedError (see above for traceback): TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:#rnn/TensorArray_1"], dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, rnn/TensorArrayUnstack/range, gradients/rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]
num_words = 2 or num_words = 5 work. I see on
https://github.com/tensorflow/tensorflow/blob/e121667dc609de978a223c56ee906368d2c4ceef/tensorflow/contrib/crf/python/ops/crf.py#L121 (mirror) is already decrementing the sequence_length by 1:
# Compute the alpha values in the forward algorithm in order to get the
# partition function.
forward_cell = CrfForwardRnnCell(transition_params)
_, alphas = rnn.dynamic_rnn(
cell=forward_cell,
inputs=rest_of_input,
sequence_length=sequence_lengths - 1,
initial_state=first_input,
dtype=dtypes.float32)
log_norm = math_ops.reduce_logsumexp(alphas, [1])
return log_norm
However, changing sequence_lengths = np.full(num_examples, num_words - 1, dtype=np.int32) to sequence_lengths = np.full(num_examples, num_words, dtype=np.int32) does not solve the issue when num_words = 1 .
How to fix this issue so that the CRF layer supports sequences of length 1?
Tested with TensorFlow 1.0.0 on Windows 7 SP1 x64 Ultimate and TensorFlow-GPU 1.0.0 on Ubuntu 14.04.4 LTS x64. I created an issue in the TensorFlow repository but it got closed due to inactivity: https://github.com/tensorflow/tensorflow/issues/7751 (mirror)