I'm using keras, and when I try model.fit it throws an error because the X_Train and Y_Train inputs have incompatible shapes.
The data I have is a system of 10 inputs and 1 output. And I'm using 9 iterations of the data as a test, so I have a list of 9 vectors with shape [10, 1] so, understandable, X_Train.shape = [9, 10, 1]. My output is a list of 9 values which makes Y_Train.shape = [9,1]. Yet I get this error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [9,1] vs. [9,10,1]
I asume the correct shape of the Y_Train vector must be [9, 1, 1] but cannot find a way to shape it so.
Based on this I have two questions: Is [9, 1, 1] the correct expected shape according to my description of the problem? and how do I make it conform to that expected shape?
The input shape that passes through a computation graph in keras is of the type:
(?, x.shape[1], x.shape[2], ....) #As seen in model.summary()
The first ? is the channel for passing your samples (rows in your dataset). You can pass them in batches, so thats something that you define while fitting the model itself.
When setting the shape of the layers however, you set it as
(x.shape[1], x.shape[2], ....)
Keras automatically adds the first channel at the start for the batches.
So, if each row in your dataset is a 1-D array of length 10. Then,
## For keras functional API
inp = Input((10,))
## For keras sequential API
model = Sequential([
Dense(32, input_shape=(10,))
])
If you are working with say a 3-D dataset, where each 'row' or sample in your dataset is a 2-D array of (10,10) shape:
## For keras functional API
inp = Input((10,10))
## For keras sequential API
model = Sequential([
Dense(32, input_shape=(10,10))
])
Specific to your question, since you have a list of 9 arrays of the shape (10,1). You should simply ignore the 9, since that is what gets pass on the first channel as (?, 10, 1). So define your input shapes as just (10,) or (10,1)
Related
My input is a array of 64 integers.
model = Sequential()
model.add( Input(shape=(68,), name="input"))
model.add(Conv1D(64, 2, activation="relu", padding="same", name="convLayer"))
I have 10,000 of these arrays in my training set. And I supposed to be specifying this in order for conv1D to work?
I am getting the dreaded
ValueError: Input 0 of layer convLayer is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: [None, 68]
error and I really don't understand what I need to do.
Don't let the name confuse you. The layer tf.keras.layers.Conv1D needs the following shape: (time_steps, features). If your dataset is made of 10,000 samples with each sample having 64 values, then your data has the shape (10000, 64), which is not directly applicable to the tf.keras.layers.Conv1D layer. You are missing the time_steps dimension. What you can do is use the tf.keras.layers.RepeatVector, which repeats your array input n times, in the example 5. This way your Conv1D layer gets an input of the shape (5, 64). Check out the documentation for more information:
time_steps = 5
model = tf.keras.Sequential()
model.add(tf.keras.layers.Input(shape=(64,), name="input"))
model.add(tf.keras.layers.RepeatVector(time_steps))
model.add(tf.keras.layers.Conv1D(64, 2, activation="relu", padding="same", name="convLayer"))
As a side note, you should ask yourself if using a tf.keras.layers.Conv1D layer is the right option for your use case. This layer is usually used for NLP and other time series tasks. For example, in sentence classification, each word in a sentence is usually mapped to a high-dimensional word vector representation, as seen in the image. This results in data with the shape (time_steps, features).
If you want to use character one hot encoded embeddings it would look something like this:
This is a simple example of one single sample with the shape (10, 10) --> 10 characters along the time series dimension and 10 features. It should help you understand the tutorial I mentioned a bit better.
The Conv1D layer does temporal convolution, that is, along the first dimension (not the batch dimension of course), so you should put something like this:
time_steps = 5
model = tf.keras.Sequential()
model.add(tf.keras.layers.Input(shape=(time_steps, 64), name="input"))
model.add(tf.keras.layers.Conv1D(64, 2, activation="relu", padding="same", name="convLayer"))
You will need to slice your data into time_steps temporal slices to feed the network.
However, if your arrays don't have a temporal structure, then conv1D is not the layer you are looking for.
I have 9 features, one output variable i.e. to be predicted, window size is 5
code works very well without "TimeDistributed" command
MODEL INPUT SHAPE: feature_tensor.shape=(1649, 5, 9) MODEL OUTPUT
SHAPE: y_train.shape= (1649,)
Thats my Code:
#Build the network model
act_fn='relu'
modelq = Sequential()
modelq.add(TimeDistributed(Conv1D(filters=105, kernel_size=2, activation=act_fn, input_shape=(None, feature_tensor.shape[1],feature_tensor.shape[2]))))
modelq.add(TimeDistributed(AveragePooling1D(pool_size=1)))
modelq.add(TimeDistributed(Flatten()))
modelq.add(LSTM(50))
modelq.add(Dense(64, activation=act_fn))
modelq.add(Dense(1))
#Compile the model
modelq.compile(optimizer='adam', loss='mean_squared_error')
modelq.fit(feature_tensor, y_train ,batch_size=1, epochs=epoch_count)
THE ERROR STATEMENT IS :
ValueError: Input 0 of layer conv1d is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: (5, 9)
I feel like there is some thing wrong with dimensionality of "feature_tensor" during "Model FITTING" i.e last command... But I don't know what's wrong with it :(
Your intuition is right, the problem is the dimensionality of tensor_feature. If you take a look in the documentation of TimeDistributed you see an example with images and Conv2d layers. There the the input has to have the following shape: batch_size, time steps, x_dim, y_dim, channels. Since you use time-series you need: batch_size, time steps, 1, features. E.g. you can reshape your data by numpy:
feature_tensor = np.reshape(feature_tensor, (-1, 5, 1, 9))
However, I am not sure if it useful to combine Conv1D with TimeDistributed, since in that case you apply the convolution only on the features and not on temporal contiguous values, where a 1d Convolution should be applied.
I have loaded images to train my model on recognizing one feature in those images.
Xtrain is a numpy ndarray of shape (1380,200,200,3 ) containing 1380 images sized 200 by 200pixels in RGB format
Ytrain has targets. shape (1380,2)
When I train my model (model.fit(Xtrain,Ytrain)) I seem to get a value error on everyone of the layers. As if the input was both Xtrain then Ytrain...
ValueError: Error when checking target: expected batch_normalization_24 to have 4 dimensions, but got array with shape (1380, 2)
Image:
The shape of Keras's batch normalizer layer's output is the same as its input. Since you have only two labels, your final layer in a Sequential model should generate two outputs. You can consider adding a Dense layer like:
model.add(Dense(2), activation='relu')
I also recommend to check your model's architecture using print(model.summary()) and make sure that inputs and outputs match with your dataset and vice versa.
I'm trying to assign one of two classes (positive/nagative) to audio using CNN with Keras. My model should accept varied lengths of input (frames) in which each frame contains 41 features but I struggle with the input size. Bear in mind that I haven't acquired full dataset so I just mocked some meaningless data just to check if network works at all.
According to documentation https://keras.io/layers/convolutional/ and my best understanding Conv1D can tackle varied lengths if first element of input_shape tuple is None. Shape of variable containing input data X_train.shape is (4, 497, 41).
data = pd.read_csv('output_file.csv', sep=';')
featureCount = data.values.shape[1]
#mocks because full data is not available yet
Y_train = np.asarray([1, 0, 1, 0])
X_train = np.asarray(
[np.array(data.values, copy=True), np.array(data.values, copy=True), np.array(data.values, copy=True),
np.array(data.values, copy=True)])
# variable length with 41 features
model = keras.models.Sequential()
model.add(keras.layers.Conv1D(100, 5, activation='relu', input_shape=(None, featureCount)))
model.add(keras.layers.GlobalMaxPooling1D())
model.add(keras.layers.Dense(10, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
model.summary()
model.fit(X_train, Y_train, epochs=10, verbose=False, validation_data=(np.array(data.values, copy=True), [1]))
This code produces error
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (497, 41). So it appears like the first dimension was cut out as it contains training samples (it seems correct to me) what bothered me is the required dimensionality, why is it 3?
After searching for the answer I stumbled onto Dimension of shape in conv1D and followed it by adding last dimension (using X_train = np.expand_dims(X_train, axis=3)) that contains only single digit but I ended up with another, similar error:
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (4, 497, 41, 1) now it seems that first dimension that previously was treated as sample "list" is now part of actual data.
I also tried fiddling with input_shape parameter but to no avail and using Reshape layer but ended up fighting with size the
What should I do to satisfy required shape? How to prepare data for processing?
I have been trying to perform regression using tflearn and my own dataset.
Using tflearn I have been trying to implement a convolutional network based off an example using the MNIST dataset. Instead of using the MNIST dataset I have tried replacing the training and test data with my own. My data is read in from a csv file and is a different shape to the MNIST data. I have 255 features which represent a 15*15 grid and a target value. In the example I replaced the lines 24-30 with (and included import numpy as np):
#read in train and test csv's where there are 255 features (15*15) and a target
csvTrain = np.genfromtxt('train.csv', delimiter=",")
X = np.array(csvTrain[:, :225]) #225, 15
Y = csvTrain[:,225]
csvTest = np.genfromtxt('test.csv', delimiter=",")
testX = np.array(csvTest[:, :225])
testY = csvTest[:,225]
#reshape features for each instance in to 15*15, targets are just a single number
X = X.reshape([-1,15,15,1])
testX = testX.reshape([-1,15,15,1])
## Building convolutional network
network = input_data(shape=[None, 15, 15, 1], name='input')
I get the following error:
ValueError: Cannot feed value of shape (64,) for Tensor u'target/Y:0',
which has shape '(?, 10)'
I have tried various combinations and have seen a similar question in stackoverflow but have not had success. The example in this page does not work for me and throws a similar error and I do not understand the answer provided or those provided by similar questions.
How do I use my own data?
Short answer
In the line 41 of the MNIST example, you also have to change the output size 10 to 1 in network = fully_connected(network, 10, activation='softmax') to network = fully_connected(network, 1, activation='linear'). Note that you can remove the final softmax.
Looking at your code, it seems you have a target value Y, which means using the L2 loss with mean_square (you will find here all the losses available):
regression(network, optimizer='adam', learning_rate=0.01,
loss='mean_square', name='target')
Also, reshape Y and Y_test to have shape (batch_size, 1).
Long answer: How to analyse the error and find the bug
Here is how to analyse the error:
The error is Cannot feed value ... for Tensor 'target/Y', which means it comes from the feed_dict argument Y.
Again, according to the error, you try to feed an Y value of shape (64,) whereas the network expect a shape (?, 10).
It expects a shape (batch_size, 10), because originally it's a network for MNIST (10 classes)
We now want to change the expected value of the network for Y.
in the code, we see that the last layer fully_connected(network, 10, activation='softmax') is returning an output of size 10
We change that to an output of size 1 without softmax: fully_connected(network, 1, activation='linear')
In the end, it was not a bug, but a wrong model architecture.