Hi there i'm trying to build a simple rnn with 11 inputs and 2 outputs
X=tf.placeholder(tf.float32,[None,n_steps,n_inputs])
y=tf.placeholder(tf.int32,[None,n_steps,n_outputs])
I know the rnn excepts an input in the shape of [batch_size,n_steps,n_inputs] so that's why i have shaped my placeholders like this
However when i run the code i get an error
ValueError: Shape must be rank 2 but is rank 3 for 'in_top_k/InTopKV2' (op: 'InTopKV2') with input shapes: [1,270,2], [1,270,2], [].
The error seems to originate here : correct = tf.nn.in_top_k(logits,tf.reshape(y,[1,n_steps,n_outputs]),1)
I have tried reshaping the logits, squeezing the logits, expanding the y dimensions, but nothing seems to work.
One difference that i have noticed is that when i squeeze the logits with
tf.squeeze(logits)
The error now says
ValueError: Shape must be rank 1 but is rank 3
That is the only 'progress' that i have been able to make, any help would be appreciated.
p.s go easy on me this is my first question ever
You have to reshape the inputs as 2D tensors, then you can reshape the result back to the desired shape:
logits_res = tf.reshape(logits, (-1, n_outputs))
y_res = tf.reshape(y, (-1, n_outputs))
correct_res = tf.nn.in_top_k(logits_res, y_res, 1)
correct = tf.reshape(correct_res, (-1, n_steps))
Related
I am trying to denoise an image with a pre-trained model I loaded as "model". I am getting an error as a result of the dimensions being different. Here is the code I have:
path_clean = r"clean.png"
clean = load_img(path_clean)
path_noisy = r"noise.png"
noisy = load_img(path_noisy)
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=3e-4),
loss=tf.keras.losses.mean_squared_error,
metrics=[tf.keras.metrics.mean_absolute_error])
history = model.fit(img_to_array(noisy), img_to_array(clean), epochs=50)
Here is the error I get, calling from the "history" line:
ValueError: Exception encountered when calling layer "concatenate" (type Concatenate).
Dimension 1 in both shapes must be equal, but are 113 and 114. Shapes are [?,113,1] and [?,114,2]. for '{{node model/concatenate/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](model/conv2d_6/Relu, model/up_sampling2d/resize/ResizeNearestNeighbor, model/concatenate/concat/axis)' with input shapes: [?,113,1,128], [?,114,2,128], [] and with computed input tensors: input[2] = <3>.
Call arguments received:
• inputs=['tf.Tensor(shape=(None, 113, 1, 128), dtype=float32)', 'tf.Tensor(shape=(None, 114, 2, 128), dtype=float32)']
What does it mean that one is 113 and one is 114? When I print the shapes of each image using this:
print(img_to_array(clean).shape)
print(img_to_array(noisy).shape)
I get this:
(500, 500, 3)
(500, 500, 3)
So the dimensions should be the same, right? Thanks for your help.
The error has to do with a certain layer within the network not managing to align the inputs that you give it. The number you see are different because the input data undergoes a series of transformations and then it arrives at this layer and everything breaks down.
Try reading the documentation for this pre-trained model to understand what its constraints are - maybe it performs some reshaping magic and expects a certain shape as input.
When you load the model, you should also be able to inspect the graph structure to understand what happens to the input up until this concatenation.
The issue is that your model relies on a certain image input size (e.g., most likely a multiple of 32). So make sure that the input width and height of the images are divisible by 32.
First of all, I know there are tons of questions similar like this; I've tried to do what the answers suggest, but seems like I do not know how to solve it. I have a Keras Functional API model:
lstm_input = keras.layers.Input(shape=(1,4), name='lstm_input')
x = keras.layers.LSTM(50, name='lstm_0')(lstm_input)
x = keras.layers.Dropout(0.2, name='lstm_dropout_0')(x)
x = keras.layers.Dense(64, name='dense_0')(x)
x = keras.layers.Activation('sigmoid', name='sigmoid_0')(x)
x = keras.layers.Dense(1, name='dense_1')(x)
output = keras.layers.Activation('linear', name='linear_output')(x)
model = keras.Model(inputs=lstm_input, outputs=output)
adam = keras.optimizers.Adam(lr=0.0005)
model.compile(optimizer=adam, loss='mse')
And when I try to fit it, it jumps this error:
ValueError: Error when checking input: expected lstm_input to have 3 dimensions, but got array with shape (4, 1)
This is my call to fit:
model.fit(X_aux['X_i'], X[i+1, 0])
# X_aux['X_i'].shape = (4, ) -- it's a numpy array
I've tried np.reshape([X_aux['X_i1']], (4,1)), where its new shape is (4, 1) but it does not work. How can I solve this?
Make sure your input_shape of X_aux['X-i'] is 3 dimensional.
The input of any RNN-based layer must be 3 dimensional where each axis is corresponded to batch_size, time_step, and feature dimension respectively.
The reason why reshaping to (4, 1) wouldn't help is that the reshaped tensor is still 2 dimension. You need 3.
Make sure you define batch_size, time_step, and feature dimension correctly and reshape X_aux['X-i'] and retrain the model again.
I'm currently working with the TensorFlow Addons SpatialPyramidPooling2D layer for image classification and I got the following error when I tried to fit the model.
ValueError: Dimensions must be equal, but are 8 and 20 for '{{node MatMul}} = BatchMatMulV2[T=DT_FLOAT, adj_x=false, adj_y=false](feature, transpose_1)' with input shapes: [?,20,8], [8,20,?]
I doubt that it's something to do with the output shape of the model. The last layer is supposed to be (None,<number_of_classes>) but I got (None,<number_of_channels>,<number_of_classes>). Because the output of SpatialPyraidPooling2D is a 3D tensor.
I tried to solve it by adding a Flatten layer right after SpatialPyramidPooling2D but it ends up the softmax layer giving me an error as below
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 1280 but received input with shape [None, 25600]
If you want output of shape (None, 8), I suggest you add a 1D pooling layer after the pyramid pooling thing.
import tensorflow as tf
x = tf.random.uniform((10, 20, 8), dtype=tf.float32)
pool = tf.keras.layers.GlobalAveragePooling1D()
print(pool(x).shape)
TensorShape([10, 8])
Does the keras functional API impose that the number of batch elements in models output equals to the number of elements in their input? For instance, the code bellow raises an exception: ValueError: Mismatch between expected batch size and model output batch size. Output shape = (1, 1), expected output shape = shape (2, 1):
d = 2
input_ = Input(shape=(d, ))
output_ = keras.layers.Lambda(lambda x: tf.reduce_sum(x, keepdims=True))(input_)
model = keras.Model(name='model', inputs=input_, outputs=output_)
If I set keepdims to false, another exception is raised: ValueError: zero-dimensional arrays cannot be concatenated, which makes sense as the model expects a batch of elements, each of which being a 2d array.
Tensorflow Keras functional API doesn't necessarily impose that the number of batch elements in model output equals to the number of elements in their input.
Tensorflow handles this by leaving the index 0 of the Output shape to None, for it to be batch_size agnostic.
But in this case, when model.summary() is printed, you can observe that the last layer output shape is fixed to (1,1) rather than (None,1).
Which will work normally if you will only feed it data with batch_size = 1, but will raise an error when the data fed have batch_size != 1.
More specifically: Mismatch between expected batch size and model output batch size. Output shape = (1, 1), expected output shape = shape (BATCH_SIZE, 1).
Expected output shape (BATCH_SIZE,1) is the output shape of your data.
Output shape (1, 1) is the output shape of the model.
I'm building an RNN LSTM network to classify texts based on the writers' age (binary classification - young / adult).
Seems like the network does not learn and suddenly starts overfitting:
Red: train
Blue: validation
One possibility could be that the data representation is not good enough. I just sorted the unique words by their frequency and gave them indices. E.g.:
unknown -> 0
the -> 1
a -> 2
. -> 3
to -> 4
So I'm trying to replace that with word embedding.
I saw a couple of examples but I'm not able to implement it in my code. Most of the examples look like this:
embedding = tf.Variable(tf.random_uniform([vocab_size, hidden_size], -1, 1))
inputs = tf.nn.embedding_lookup(embedding, input_data)
Does this mean we're building a layer that learns the embedding? I thought that one should download some Word2Vec or Glove and just use that.
Anyway let's say I want to build this embedding layer...
If I use these 2 lines in my code I get an error:
TypeError: Value passed to parameter 'indices' has DataType float32 not in list of allowed values: int32, int64
So I guess I have to change the input_data type to int32. So I do that (it's all indices after all), and I get this:
TypeError: inputs must be a sequence
I tried wrapping inputs (argument to tf.contrib.rnn.static_rnn) with a list: [inputs] as suggested in this answer, but that produced another error:
ValueError: Input size (dimension 0 of inputs) must be accessible via
shape inference, but saw value None.
Update:
I was unstacking the tensor x before passing it to embedding_lookup. I moved the unstacking after the embedding.
Updated code:
MIN_TOKENS = 10
MAX_TOKENS = 30
x = tf.placeholder("int32", [None, MAX_TOKENS, 1])
y = tf.placeholder("float", [None, N_CLASSES]) # 0.0 / 1.0
...
seqlen = tf.placeholder(tf.int32, [None]) #list of each sequence length*
embedding = tf.Variable(tf.random_uniform([VOCAB_SIZE, HIDDEN_SIZE], -1, 1))
inputs = tf.nn.embedding_lookup(embedding, x) #x is the text after converting to indices
inputs = tf.unstack(inputs, MAX_POST_LENGTH, 1)
outputs, states = tf.contrib.rnn.static_rnn(lstm_cell, inputs, dtype=tf.float32, sequence_length=seqlen) #---> Produces error
*seqlen: I zero-padded the sequences so all of them have the same list size, but since the actual size differ, I prepared a list describing the length without the padding.
New error:
ValueError: Input 0 of layer basic_lstm_cell_1 is incompatible with
the layer: expected ndim=2, found ndim=3. Full shape received: [None,
1, 64]
64 is the size of each hidden layer.
It's obvious that I have a problem with the dimensions... How can I make the inputs fit the network after embedding?
From the tf.nn.static_rnn , we can see the inputs arguments to be:
A length T list of inputs, each a Tensor of shape [batch_size, input_size]
So your code should be something like:
x = tf.placeholder("int32", [None, MAX_TOKENS])
...
inputs = tf.unstack(inputs, axis=1)
tf.squeeze is a method that removes dimensions of size 1 from the tensor. If the end goal is to have the input shape as [None,64], then put a line similar to inputs = tf.squeeze(inputs) and that would fix your problem.