I am attempting to create a deep CNN that can classify each individual pixel in an image. I am replicating architecture from the image below taken from this paper. In the paper it is mentioned that deconvolutions are used so that any size of input is possible. This can be seen in the image below.
Github Repository
Currently, I have hard coded my model to accept images of size 32x32x7, but I would like to accept any size of input. What changes would I need to make to my code to accept variable sized input?
x = tf.placeholder(tf.float32, shape=[None, 32*32*7])
y_ = tf.placeholder(tf.float32, shape=[None, 32*32*7, 3])
...
DeConnv1 = tf.nn.conv3d_transpose(layer1, filter = w, output_shape = [1,32,32,7,1], strides = [1,2,2,2,1], padding = 'SAME')
...
final = tf.reshape(final, [1, 32*32*7])
W_final = weight_variable([32*32*7,32*32*7,3])
b_final = bias_variable([32*32*7,3])
final_conv = tf.tensordot(final, W_final, axes=[[1], [1]]) + b_final
Dynamic placeholders
Tensorflow allows to have multiple dynamic (a.k.a. None) dimensions in placeholders. The engine won't be able to ensure correctness while the graph is built, hence the client is responsible for feeding the correct input, but it provides a lot of flexibility.
So I'm going from...
x = tf.placeholder(tf.float32, shape=[None, N*M*P])
y_ = tf.placeholder(tf.float32, shape=[None, N*M*P, 3])
...
x_image = tf.reshape(x, [-1, N, M, P, 1])
to...
# Nearly all dimensions are dynamic
x_image = tf.placeholder(tf.float32, shape=[None, None, None, None, 1])
label = tf.placeholder(tf.float32, shape=[None, None, 3])
Since you intend to reshape the input to 5D anyway, so why don't use 5D in x_image right from the start. At this point, the second dimension of label is arbitrary, but we promise tensorflow that it will match with x_image.
Dynamic shapes in deconvolution
Next, the nice thing about tf.nn.conv3d_transpose is that its output shape can be dynamic. So instead of this:
# Hard-coded output shape
DeConnv1 = tf.nn.conv3d_transpose(layer1, w, output_shape=[1,32,32,7,1], ...)
... you can do this:
# Dynamic output shape
DeConnv1 = tf.nn.conv3d_transpose(layer1, w, output_shape=tf.shape(x_image), ...)
This way the transpose convolution can be applied to any image and the result will take the shape of x_image that was actually passed in at runtime.
Note that static shape of x_image is (?, ?, ?, ?, 1).
All-Convolutional network
Final and most important piece of the puzzle is to make the whole network convolutional, and that includes your final dense layer too. Dense layer must define its dimensions statically, which forces the whole neural network fix input image dimensions.
Luckily for us, Springenberg at al describe a way to replace an FC layer with a CONV layer in "Striving for Simplicity: The All Convolutional Net" paper. I'm going to use a convolution with 3 1x1x1 filters (see also this question):
final_conv = conv3d_s1(final, weight_variable([1, 1, 1, 1, 3]))
y = tf.reshape(final_conv, [-1, 3])
If we ensure that final has the same dimensions as DeConnv1 (and others), it'll make y right the shape we want: [-1, N * M * P, 3].
Combining it all together
Your network is pretty large, but all deconvolutions basically follow the same pattern, so I've simplified my proof-of-concept code to just one deconvolution. The goal is just to show what kind of network is able to handle images of arbitrary size. Final remark: image dimensions can vary between batches, but within one batch they have to be the same.
The full code:
sess = tf.InteractiveSession()
def conv3d_dilation(tempX, tempFilter):
return tf.layers.conv3d(tempX, filters=tempFilter, kernel_size=[3, 3, 1], strides=1, padding='SAME', dilation_rate=2)
def conv3d(tempX, tempW):
return tf.nn.conv3d(tempX, tempW, strides=[1, 2, 2, 2, 1], padding='SAME')
def conv3d_s1(tempX, tempW):
return tf.nn.conv3d(tempX, tempW, strides=[1, 1, 1, 1, 1], padding='SAME')
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
def max_pool_3x3(x):
return tf.nn.max_pool3d(x, ksize=[1, 3, 3, 3, 1], strides=[1, 2, 2, 2, 1], padding='SAME')
x_image = tf.placeholder(tf.float32, shape=[None, None, None, None, 1])
label = tf.placeholder(tf.float32, shape=[None, None, 3])
W_conv1 = weight_variable([3, 3, 1, 1, 32])
h_conv1 = conv3d(x_image, W_conv1)
# second convolution
W_conv2 = weight_variable([3, 3, 4, 32, 64])
h_conv2 = conv3d_s1(h_conv1, W_conv2)
# third convolution path 1
W_conv3_A = weight_variable([1, 1, 1, 64, 64])
h_conv3_A = conv3d_s1(h_conv2, W_conv3_A)
# third convolution path 2
W_conv3_B = weight_variable([1, 1, 1, 64, 64])
h_conv3_B = conv3d_s1(h_conv2, W_conv3_B)
# fourth convolution path 1
W_conv4_A = weight_variable([3, 3, 1, 64, 96])
h_conv4_A = conv3d_s1(h_conv3_A, W_conv4_A)
# fourth convolution path 2
W_conv4_B = weight_variable([1, 7, 1, 64, 64])
h_conv4_B = conv3d_s1(h_conv3_B, W_conv4_B)
# fifth convolution path 2
W_conv5_B = weight_variable([1, 7, 1, 64, 64])
h_conv5_B = conv3d_s1(h_conv4_B, W_conv5_B)
# sixth convolution path 2
W_conv6_B = weight_variable([3, 3, 1, 64, 96])
h_conv6_B = conv3d_s1(h_conv5_B, W_conv6_B)
# concatenation
layer1 = tf.concat([h_conv4_A, h_conv6_B], 4)
w = tf.Variable(tf.constant(1., shape=[2, 2, 4, 1, 192]))
DeConnv1 = tf.nn.conv3d_transpose(layer1, filter=w, output_shape=tf.shape(x_image), strides=[1, 2, 2, 2, 1], padding='SAME')
final = DeConnv1
final_conv = conv3d_s1(final, weight_variable([1, 1, 1, 1, 3]))
y = tf.reshape(final_conv, [-1, 3])
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=label, logits=y))
print('x_image:', x_image)
print('DeConnv1:', DeConnv1)
print('final_conv:', final_conv)
def try_image(N, M, P, B=1):
batch_x = np.random.normal(size=[B, N, M, P, 1])
batch_y = np.ones([B, N * M * P, 3]) / 3.0
deconv_val, final_conv_val, loss = sess.run([DeConnv1, final_conv, cross_entropy],
feed_dict={x_image: batch_x, label: batch_y})
print(deconv_val.shape)
print(final_conv.shape)
print(loss)
print()
tf.global_variables_initializer().run()
try_image(32, 32, 7)
try_image(16, 16, 3)
try_image(16, 16, 3, 2)
Theoretically, it's possible. you need to set the image size of the input and label image place holder to none, and let the graph dynamically infer the image size from input data.
However, have to be careful when you define the graph. Need to use tf.shape instead of tf.get_shape(). the former dynamically infer the shape only when you session.run, the latter can get the shape when you define the graph. But when input size is set to none, the latter does not get true reshape (maybe just return None).
And to make things complicated, if you use tf.layers.conv2d or upconv2d, sometimes these high level functions do not like tf.shape, because it seems they assume the shape information are available during graph construction.
I hope I have better working example to show the points above. I'll put this answer as a placeholder and will come back and add more stuff if I get a chance.
Related
I want to apply "tf.nn.max_pool()" on a single image but I get a result with dimension that is totally different than the input:
import tensorflow as tf
import numpy as np
ifmaps_1 = tf.Variable(tf.random_uniform( shape=[ 7, 7, 3], minval=0, maxval=3, dtype=tf.int32))
ifmaps=tf.dtypes.cast(ifmaps_1, dtype=tf.float64)
ofmaps_tf = tf.nn.max_pool([ifmaps], ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1], padding="SAME")[0] # no padding
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
print("ifmaps_tf = ")
print(ifmaps.eval())
print("ofmaps_tf = ")
result = sess.run(ofmaps_tf)
print(result)
I think this is related to trying to apply pooling to single example not on a batch. I need to do the pooling on a single example.
Any help is appreciated.
Your input is (7,7,3), kernel size is (3,3) and stride is (2,2). So if you do not want any paddings, (state in your comment), you should use padding="VALID", that will return a (3,3) tensor as output. If you use padding="SAME", it will return (4,4) tensor.
Usually, the formula of calculating output size for SAME pad is:
out_size = ceil(in_sizei/stride)
For VALID pad is:
out_size = ceil(in_size-filter_size+1/stride)
When I build the FCN for segmentation, I want the images to keep the original size of input data, so I use the fully convolution layers. When I choose the fixed input size, such as (224, 224), the transpose conv works fine. However, when I changed the code of using (224, 224) to (h, w), I meet the following error. I googled before, but I didn't figure it out. Can anyone help me? Thanks.
Error information:
InvalidArgumentError (see above for traceback): Conv2DSlowBackpropInput: Size
of out_backprop doesn't match computed: actual = 62, computed =
63spatial_dim: 2 input: 500 filter: 16 output: 62 stride: 8 dilation: 1
[[Node: deconv_layer/conv2d_transpose_2 =
Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1],
padding="SAME", strides=[1, 1, 8, 8], use_cudnn_on_gpu=true,
_device="/job:localhost/replica:0/task:0/device:GPU:0"]
(deconv_layer/conv2d_transpose_2-0-VecPermuteNHWCToNCHW-
LayoutOptimizer/_1961, deconv_layer/deconv3/kernel/read,
deconv_layer/Add_1)]]
[[Node: losses/_2091 = _Recv[client_terminated=false,
recv_device="/job:localhost/replica:0/task:0/device:CPU:0",
send_device="/job:localhost/replica:0/task:0/device:GPU:0",
send_device_incarnation=1, tensor_name="edge_4480_losses",
tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]
()]]
Code:
with tf.variable_scope("deconv_layer"):
deconv_shape1 = block2.get_shape()
W_t1 = deconv_utils.weight_variable([4, 4, deconv_shape1[3].value, 2048],
name="deconv1/kernel")
b_t1 = deconv_utils.bias_variable([deconv_shape1[3].value],
name="deconv1/biases")
deconv_t1 = deconv_utils.conv2d_transpose_strided(block4, W_t1, b_t1,
output_shape=tf.shape(block2))
fuse1 = tf.add(deconv_t1, block2)
print("deconv_t1: ", deconv_t1.shape)
print("fuse_1: ", fuse1.shape)
tf.identity(fuse1, name="fuse1")
deconv_shape2 = block1.get_shape()
W_t2 = deconv_utils.weight_variable([4, 4, deconv_shape2[3].value,
deconv_shape1[3].value], name="deconv2/kernel")
b_t2 = deconv_utils.bias_variable([deconv_shape2[3].value],
name="deconv2/biases")
deconv_t2 = deconv_utils.conv2d_transpose_strided(fuse1, W_t2, b_t2,
output_shape=tf.shape(block1))
fuse2 = tf.add(deconv_t2, block1)
print("deconv_t2: ", deconv_t2.shape)
print("fuse2: ", fuse2.shape)
tf.identity(fuse2, name="fuse2")
shape = tf.shape(features)
deconv_shape3 = tf.stack([shape[0], shape[1], shape[2], num_classes])
W_t3 = deconv_utils.weight_variable([16, 16, num_classes,
deconv_shape2[3].value], name="deconv3/kernel")
b_t3 = deconv_utils.bias_variable([num_classes], name="deconv3/biases")
deconv_t3 = deconv_utils.conv2d_transpose_strided(fuse2, W_t3, b_t3,
output_shape=deconv_shape3, stride=8)
print("deconv_t3: ", deconv_t3.shape)
The version with out custom functions is here:
with tf.variable_scope("deconv_layer"):
deconv1_shape = block2.get_shape()
shape1 = [4, 4, deconv1_shape[3].value, 2048]
deconv1_kernel = tf.Variable(initial_value=tf.truncated_normal(shape1,
stddev=0.02),
trainable=True,
name="deconv1/kernel")
deconv1 = tf.nn.conv2d_transpose(value=block4,
filter=deconv1_kernel,
# output_shape=[BATCH_SIZE,
tf.shape(block2)[1], tf.shape(block2)[2], 512],
output_shape=tf.shape(block2),
strides=[1, 2, 2, 1],
padding='SAME',
data_format='NHWC'
)
print('deconv1', deconv1.shape)
fuse1 = tf.add(deconv1, block2) # fuse1 = pool4 + deconv2(pool5)
tf.identity(fuse1, name="fuse1")
deconv2_shape = block1.get_shape()
shape2 = [4, 4, deconv2_shape[3].value, deconv1_shape[3].value]
deconv2_kernel = tf.Variable(initial_value=tf.truncated_normal(shape2,
stddev=0.02),
trainable=True,
name="deconv2/kernel")
deconv2 = tf.nn.conv2d_transpose(value=fuse1,
filter=deconv2_kernel,
output_shape=tf.shape(block1),
strides=[1, 2, 2, 1],
padding='SAME',
data_format='NHWC'
)
print('deconv2', deconv2.shape)
fuse2 = tf.add(deconv2, block1)
tf.identity(fuse2, name="fuse2")
deconv3_shape = tf.stack([tf.shape(features)[0], tf.shape(features)[1],
tf.shape(features)[2], num_classes])
shape3 = [16, 16, num_classes, deconv2_shape[3].value]
deconv_final_kernel = tf.Variable(initial_value=tf.truncated_normal(shape3, stddev=0.02),
trainable=True,
name="deconv3/kernel")
seg_logits = tf.nn.conv2d_transpose(value=fuse2,
filter=deconv_final_kernel,
output_shape=deconv3_shape,
strides=[1, 8, 8, 1],
padding='SAME',
data_format='NHWC')
The conv Net and Deconv Net in FCN, which are built by different structures, are maybe not consistent with each other. In this case, the conv net use conv with padding='VALID', while the deconv net uses all conv_transpose with padding='SAME. Thus the shapes are not the same, which causes the problem above.
This is because of your stride > 1. The calculation can not be correct at all time. This GitHub post explains it.
I had similar issue while trying to replicate pytorch's transposeconv2d function in tensorflow. I was trying to do padding on input before passing to the conv2d_transpose() function and doing padding again on the deconvolved output. This was the reason why graph was initialized properly but there was error in calculating the gradients. I solved the error by removing all manual paddings and changing padding="SAME" inside the function. I guess this is handeled internally in the function. Correct me if I am wrong. I don't know how much this affects the actual output.
I am new to tensorflow, and I think I got the right answer, but I am missing something minimal, that I cant find online. I hope someone send me a reference or leads me to what I am missing.
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
batch_size = 128
test_size = 256
def init_weights(shape):
return tf.Variable(tf.random_normal(shape, stddev=0.01))
def model(X, w, w2, w3, w4, w_o, p_keep_conv, p_keep_hidden):
l1a = tf.nn.relu(tf.nn.conv2d(X, w, # l1a shape=(?, 28, 28, 32)
strides=[1, 1, 1, 1], padding='SAME'))
l1 = tf.nn.max_pool(l1a, ksize=[1, 2, 2, 1], # l1 shape=(?, 14, 14, 32)
strides=[1, 2, 2, 1], padding='SAME')
l1 = tf.nn.dropout(l1, p_keep_conv)
l2a = tf.nn.relu(tf.nn.conv2d(l1, w2, # l2a shape=(?, 14, 14, 64)
strides=[1, 1, 1, 1], padding='SAME'))
l2 = tf.nn.max_pool(l2a, ksize=[1, 2, 2, 1], # l2 shape=(?, 7, 7, 64)
strides=[1, 2, 2, 1], padding='SAME')
l2 = tf.nn.dropout(l2, p_keep_conv)
l3a = tf.nn.relu(tf.nn.conv2d(l2, w3, # l3a shape=(?, 7, 7, 128)
strides=[1, 1, 1, 1], padding='SAME'))
l3 = tf.nn.max_pool(l3a, ksize=[1, 2, 2, 1], # l3 shape=(?, 4, 4, 128)
strides=[1, 2, 2, 1], padding='SAME')
l3 = tf.reshape(l3, [-1, w4.get_shape().as_list()[0]]) # reshape to (?, 2048)
l3 = tf.nn.dropout(l3, p_keep_conv)
l4 = tf.nn.relu(tf.matmul(l3, w4))
l4 = tf.nn.dropout(l4, p_keep_hidden)
pyx = tf.matmul(l4, w_o)
return pyx
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
trX, trY, teX, teY = mnist.train.images, mnist.train.labels, mnist.test.images, mnist.test.labels
trX = trX.reshape(-1, 28, 28, 1) # 28x28x1 input img
teX = teX.reshape(-1, 28, 28, 1) # 28x28x1 input img
X = tf.placeholder("float", [None, 28, 28, 1])
Y = tf.placeholder("float", [None, 10])
w = init_weights([3, 3, 1, 32]) # 3x3x1 conv, 32 outputs
w2 = init_weights([3, 3, 32, 64]) # 3x3x32 conv, 64 outputs
w3 = init_weights([3, 3, 64, 128]) # 3x3x32 conv, 128 outputs
w4 = init_weights([128 * 4 * 4, 625]) # FC 128 * 4 * 4 inputs, 625 outputs
w_o = init_weights([625, 10]) # FC 625 inputs, 10 outputs (labels)
p_keep_conv = tf.placeholder("float")
p_keep_hidden = tf.placeholder("float")
py_x = model(X, w, w2, w3, w4, w_o, p_keep_conv, p_keep_hidden)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=py_x, labels=Y))
train_op = tf.train.RMSPropOptimizer(0.001, 0.9).minimize(cost)
predict_op = tf.argmax(py_x, 1)
# Launch the graph in a session
saver = tf.train.Saver()
with tf.Session() as sess:
# you need to initialize all variables
tf.global_variables_initializer().run()
for i in range(100):
training_batch = zip(range(0, len(trX), batch_size),
range(batch_size, len(trX)+1, batch_size))
for start, end in training_batch:
sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end],
p_keep_conv: 0.8, p_keep_hidden: 0.5})
test_indices = np.arange(len(teX)) # Get A Test Batch
np.random.shuffle(test_indices)
test_indices = test_indices[0:test_size]
print(i, np.mean(np.argmax(teY[test_indices], axis=1) ==
sess.run(predict_op, feed_dict={X: teX[test_indices],
Y: teY[test_indices],
p_keep_conv: 1.0,
p_keep_hidden: 1.0})))
save_path = saver.save(sess, "tmp/model.ckpt")
print("Model saved in file: %s" % save_path)
After all of this, now I am trying to predict a single image from this array just as an example (I know its not a proper test), to give me the class using:
with tf.Session() as sess:
# Restore variables from disk.
saver.restore(sess, "tmp/model.ckpt")
print "...Model Loaded..."
prediction=tf.argmax(predict_op,1)
print prediction.eval(feed_dict={X: teX[2].reshape(1,28,28,1)}, session=sess)
But im getting this error:
InvalidArgumentError: You must feed a value for placeholder tensor
'Placeholder_3' with dtype
This previous problem has been solved by adding p_keep_conv: 1.0, p_keep_hidden: 1.0 to the dict.
After this another issue appeared:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-91-3e9ead14a8b3> in <module>()
4 print "...Model Loaded..."
5 prediction=tf.argmax(predict_op,1)
----> 6 classification = sess.run(tf.argmax(predict_op, 1), feed_dict={X: teX[3].reshape(1,28,28,1),p_keep_conv: 1.0,p_keep_hidden: 1.0})
7
....
InvalidArgumentError: Expected dimension in the range [-1, 1), but got 1
[[Node: ArgMax_21 = ArgMax[T=DT_INT64, Tidx=DT_INT32, output_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"](ArgMax/_37, ArgMax_21/dimension)]]
I'm summing up what we said in the comments in this answer.
Placeholder error:
Your prediction.eval() call has a feed_dict that doesn't contain a value for p_keep_conv and p_keep_hidden. Note that, since you don't provide a name=... argument whe defining your placholders, they get the default name Placeholder_N which is what the error message shows. It's a good practice to always give a meaningful name to variables, constants and placeholders in order to make debugging easier.
Argmax expected dimension:
tf.argmax's definition states:
axis: A Tensor. Must be one of the following types: int32, int64.
int32, 0 <= axis < rank(input). Describes which axis of the input
Tensor to reduce across.
It seems, then, that the only way to run argmax on the last axis of the tensor is by giving it axis=-1, because of the "strictly less than" sign in the definition of the function (I don't understand why they made this design choice).
I'm using tensor flow to process color images with a convolutional neural network. A code snippet is below.
My code runs so I think I got the number of channels right. My question is, how do I correctly order the rgb data? Is it in the form rgbrgbrgb or would it be rrrgggbbb? Presently I am using the latter. Thanks. Any help would be appreciated.
c_output = 2
c_input = 784 * 3
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
self.c_x = tf.placeholder(tf.float32, shape=[None, c_input])
self.c_y_ = tf.placeholder(tf.float32, shape=[None, c_output])
self.W_conv1 = weight_variable([5, 5, 3, 32])
self.b_conv1 = bias_variable([32])
self.x_image = tf.reshape(self.c_x, [-1, 28, 28 , 3])
self.h_conv1 = tf.nn.relu(conv2d(self.x_image, self.W_conv1) + self.b_conv1)
self.h_pool1 = max_pool_2x2(self.h_conv1)
self.W_conv2 = weight_variable([5, 5, 32, 64])
self.b_conv2 = bias_variable([64])
self.h_conv2 = tf.nn.relu(conv2d(self.h_pool1, self.W_conv2) + self.b_conv2)
self.h_pool2 = max_pool_2x2(self.h_conv2)
self.W_fc1 = weight_variable([7 * 7 * 64, 1024])
self.b_fc1 = bias_variable([1024])
self.h_pool2_flat = tf.reshape(self.h_pool2, [-1, 7 * 7 * 64 ])
self.h_fc1 = tf.nn.relu(tf.matmul(self.h_pool2_flat, self.W_fc1) + self.b_fc1)
self.keep_prob = tf.placeholder(tf.float32)
self.h_fc1_drop = tf.nn.dropout(self.h_fc1, self.keep_prob)
self.W_fc2 = weight_variable([1024, c_output])
self.b_fc2 = bias_variable([c_output])
self.y_conv = tf.matmul(self.h_fc1_drop, self.W_fc2) + self.b_fc2
self.c_cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(self.y_conv, self.c_y_))
self.c_train_step = tf.train.AdamOptimizer(1e-4).minimize(self.c_cross_entropy)
self.c_correct_prediction = tf.equal(tf.argmax(self.y_conv, 1), tf.argmax(self.c_y_, 1))
self.c_accuracy = tf.reduce_mean(tf.cast(self.c_correct_prediction, tf.float32))
TL;DR: With your current program, the in-memory layout of the data should be should be R-G-B-R-G-B-R-G-B-R-G-B...
I assume from this line that you are passing in RGB images with 28x28 pixels:
self.x_image = tf.reshape(self.c_x, [-1, 28, 28, 3])
We can call the dimensions of self.x_image are "batch", "height", "width", and "channel". This matches the default data format for tf.nn.conv_2d() and tf.nn.max_pool().
In TensorFlow, the in-memory representation of a tensor is row-major order (or "C" ordering, because that is the representation of arrays in the C programming language). Essentially this means that the rightmost dimension is the fastest changing, and the elements of the tensor are packed together in memory in the following order (where ? stands for the unknown batch size, minus 1):
[0, 0, 0, 0]
[0, 0, 0, 1]
[0, 0, 0, 2]
[0, 0, 1, 0]
...
[?, 27, 27, 1]
[?, 27, 27, 2]
Therefore your program probably isn't interpreting the image data correctly. There are at least two options:
Reshape your data to match its true order ("batch", "channels", "height", "width"):
self.x_image = tf.reshape(self.c_x, [-1, 3, 28, 28])
In fact, this format is sometimes more efficient for convolutions. You can instruct tf.nn.conv2d() and tf.nn.max_pool() to use it without transposing by passing the optional argument data_format="NCHW", but you will also need to change the shape of your bias variables to match.
Transpose your image data to match the result of your program using tf.transpose():
self.x_image = tf.transpose(tf.reshape(self.c_x, [-1, 3, 28, 28]), [0, 2, 3, 1])
I'm trying to apply the expert portion of the tutorial to my own data but I keep running into dimension errors. Here's the code leading up to the error.
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
W_conv1 = weight_variable([1, 8, 1, 4])
b_conv1 = bias_variable([4])
x_image = tf.reshape(tf_in, [-1,2,8,1])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
And then when I try to run this command:
W_conv2 = weight_variable([1, 4, 4, 8])
b_conv2 = bias_variable([8])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
I get the following errors:
ValueError Traceback (most recent call last)
<ipython-input-41-7ab0d7765f8c> in <module>()
3
4 h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
----> 5 h_pool2 = max_pool_2x2(h_conv2)
ValueError: ('filter must not be larger than the input: ', 'Filter: [', Dimension(2), 'x', Dimension(2), '] ', 'Input: [', Dimension(1), 'x', Dimension(4), '] ')
Just for some background information, the data that I'm dealing with is a CSV file where each row contains 10 features and 1 empty column that can be a 1 or a 0. What I'm trying to get is a probability in the empty column that the column will equal a 1.
You have to shape the input so it is compatible with both the training tensor and the output. If you input is length 1, your output should be length 1 (length is substituted for dimension).
When you're dealing with-
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 1, 1, 1],
strides=[1, 1, 1, 1], padding='SAME')
Notice how I changed the strides and the ksize to [1, 1, 1, 1]. This will match an output to a 1 dimensional input and prevent errors down the road.
When you're defining your weight variable (see code below)-
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
you're going to have to make the first 2 numbers conform to the feature tensor that you are using to train your model, the last two numbers will be the dimension of the predicted output (same as the dimension of the input).
W_conv1 = weight_variable([1, 10, 1, 1])
b_conv1 = bias_variable([1])
Notice the [1, 10, in the beginning which signifies that the feature tensor is going to be a 1x10 feature tensor; the last two numbers 1, 1] correspond to the dimensions of the input and output tensors/predictors.
When you reshape your x_foo tensor (I call it x_ [x prime]), you, for whatever reason, have to define it like so-
x_ = tf.reshape(x, [-1,1,10,1])
Notice the 1 and 10 in the middle- ...1,10,.... Once again, these numbers correspond to the dimension of your feature tensor.
For every bias variable, you choose the final number of the previously defined variable. For example, if W_conv1 = weight_variable([1, 10, 1, 1]) appears like so, you take the final number and put that into your bias variable so it can match the dimensions of the input. This is done like so- b_conv1 = bias_variable([1]).
If you need any more explanation please comment below.
The dimensions you are using for the filter are not matching the output of the hidden layer.
Let me see if I understood you: your input is composed of 8 features, and you want to reshape it into a 2x4 matrix, right?
The weights you created with weight_variable([1, 8, 1, 4]) expect a 1x8 input, in one channel, and produce a 1x8 output in 4 channels (or hidden units). The filter you are using sweeps the input in 2x2 squares. However, since the result of the weights is 1x8, they won't match.
You should reshape the input as
x_image = tf.reshape(tf_in, [-1,2,4,1])
Now, your input is actually 2x4 instead of 1x8. Then you need to change the weight shape to (2, 4, 1, hidden_units) to deal with a 2x4 output. It will also produce a 2x4 output, and the 2x2 filter now can be applied.
After that, the filter will match the output of the weights. Also note that you will have to change the shape of your second weight matrix to weight_variable([2, 4, hidden_units, hidden2_units])