I'm developing a neural network architecture that takes a vector as the input and outputs another different vector.
The error message I´m getting is the following:
ValueError: Dimensions must be equal, but are 384 and 96 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](sequential/cu_dnnlstm_1/transpose_1, IteratorGetNext:1)' with input shapes: [8,384,4], [8,96,4].
With regards to the code, I have declared the following training and validation vectors which shapes are:
x_train.shape = (96, 384, 21)
y_train.shape = (96, 96, 4)
x_val.shape = (25, 384, 21)
y_val.shape = (25, 96, 4)
Being "x" the input vector, and "y" the output vector, or the label of "x".
The architecture of the neural network is the following:
model = Sequential()
model.add(CuDNNLSTM(96,input_shape=(x_train.shape[1],x_train.shape[2]),return_sequences=True))
model.add(CuDNNLSTM(4,input_shape=(y_train.shape[1],y_train.shape[2]),return_sequences=True))
model.compile(optimizer=optimizer, loss='mse', metrics='mse')
Finally the fit of the neural network is done in the following lines:
# Fit the model
epochs = 10
batch_size = 8
train_steps = x_train.shape[0] // batch_size
valid_steps = x_val.shape[0] // batch_size
with tf.device("GPU:1"):
history = model.fit(x_train, y_train,
validation_data=(x_val, y_val),
epochs = epochs,
batch_size = batch_size,
steps_per_epoch = train_steps,
validation_steps = valid_steps,
callbacks=callbacks)
I dont understand where is the error, since in this dummy architecture both shapes match both the input and the output, I have seen that people usually make the output of the neural network a dense layer, but I would like it to be just the whole vector of "y" and not just a point.
Thanks in advance.
I have tried modifying the shapes that the LSTM layers use, doesnt work.
Related
My train set has 10 columns including a target column that I'm trying to predict, while my test set (dataframe_test) has 9 columns. When I run the code I receive this error:
Input 0 of layer "Hidden1" is incompatible with the layer: expected axis -1 of input shape to have value 10, but received input with shape (None, 9)
Call arguments received:
• inputs=tf.Tensor(shape=(None, 9), dtype=float64)
• training=False
• mask=None**
My model looks like this:
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(units=10,
activation='relu',
kernel_regularizer=tf.keras.regularizers.l2(l=0.01),
name='Hidden1'))
model.add(tf.keras.layers.Dense(units=6,
activation='relu',
kernel_regularizer=tf.keras.regularizers.l2(l=0.01),
name='Hidden2'))
model.add(tf.keras.layers.Dense(units=1,
name='Output'))
my_learning_rate = 0.3
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=my_learning_rate),
loss="categorical_crossentropy",
metrics='accuracy')
epochs = 10
batch_size = 32
history = model.fit(train, y_train, epochs = epochs, batch_size = batch_size)
epochs = history.epoch
print(epochs)
score = model.predict(dataframe_test)
You must to split your train set in a 9 columns input matrix x_train = train[:, :10] and a single column training target matrix y_train = train[:, 10].reshape((-1, 1)).
Try using sigmoid
input_size=len(X.columns)
model.add(Dense(10,activation='sigmoid', input_shape=(input_size,)))
model.add(Dense(10,activation='relu'))
model.add(Dense(10,activation='relu'))
model.add(Dense(1))
After running my model for one epoch it crashed with following error message:
InvalidArgumentError: Specified a list with shape [60,9] from a tensor with shape [56,9]
[[{{node TensorArrayUnstack/TensorListFromTensor}}]]
[[sequential_7/lstm_17/PartitionedCall]] [Op:__inference_train_function_29986]
This happened after I changed the LSTM Layer to stateful=True and had to pass
the batch_input_shape Argument instead of the input_shape
Below is my code, I'm sure it has something to do with the shape of my data:
test_split = 0.2
history_points = 60
n = int(histories.shape[0] * test_split)
histories_train = histories[:n]
y_train = next_values_normalized[:n]
histories_test = histories[n:]
y_test = next_values_normalized[n:]
next_values_test = next_values[n:]
print(histories_train.shape)
print(y_train.shape)
-->(1421, 60, 9)
-->(1421, 1)
# model architecture
´´´model = Sequential()
model.add(LSTM(units=128, stateful=True,return_sequences=True, batch_input_shape=(60,history_points, 9)))
model.add(LSTM(units=64,stateful=True,return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=32))
model.add(Dropout(0.2))
model.add(Dense(20))
ADAM=keras.optimizers.Adam(0.0005, beta_1=0.9, beta_2=0.999, amsgrad=False)
model.compile(loss='mean_squared_error', optimizer=ADAM)
model.fit(x=histories_train, y=y_train, batch_size=batchsize, epochs=50, shuffle=False, validation_split=0.2,verbose=1)
´´´
For a stateful LSTM, the batch size should be chosen in a way, so that the number of samples is divisible by the batch size. See also here:
Keras: What if the size of data is not divisible by batch_size?
In your case, considering that you take 20% from your training data as a validation set, you have 1136 samples remaining. So you should choose a batch size by which 1136 is divisible.
Additionally, you could for example remove some samples or reuse samples to be able to choose various batch sizes.
I'm trying to build a multi-output keras model starting from a working single output model. Keras however, is complaining about tensors dimensions.
The single output Model:
This GRU model is training and predicting fine:
timesteps = 250
features = 2
input_tensor = Input(shape=(timesteps, features), name="input")
conv = Conv1D(filters=128, kernel_size=6,use_bias=True)(input_tensor)
b = BatchNormalization()(conv)
s_gru, states = GRU(256, return_sequences=True, return_state=True, name="gru_1")(b)
biases = keras.initializers.Constant(value=88.15)
out = Dense(1, activation='linear', name="output")(s_gru)
model = Model(inputs=input_tensor, outputs=out)
My numpy arrays are:
train_x # shape:(7110, 250, 2)
train_y # shape: (7110, 250, 1)
If fit the model with the following code and everything is fine:
model.fit(train_x, train_y,batch_size=128, epochs=10, verbose=1)
The Problem:
I want to use a slightly modified version of the network that outputs also the GRU states:
input_tensor = Input(shape=(timesteps, features), name="input")
conv = Conv1D(filters=128, kernel_size=6,use_bias=True)(input_tensor)
b = BatchNormalization()(conv)
s_gru, states = GRU(256, return_sequences=True, return_state=True, name="gru_1")(b)
biases = keras.initializers.Constant(value=88.15)
out = Dense(1, activation='linear', name="output")(s_gru)
model = Model(inputs=input_tensor, outputs=[out, states]) # multi output
#fit the model but with a list of numpy array as y
model.compile(optimizer=optimizer, loss='mae', loss_weights=[0.5, 0.5])
history = model.fit(train_x, [train_y,train_y], batch_size=128, epochs=10, callbacks=[])
This training fails and keras is complaining about the target dimensions:
ValueError: Error when checking target: expected gru_1 to have 2 dimensions, but got array with shape (7110, 250, 1)
I'm using Keras 2.3.0 and Tensorflow 2.0.
What am I missing here?
The dimensions of the second output and the second element in the outputs list should be of similar shape. In this case, states would be of shape (7110, 256), which can't really be compared to the train_y shape (which will be of shape (7110, 250, 1) as noted in the first code block. Make sure the outputs can be compared with a similar shape.
I have received multiple diffrent ValueErrors when trying to solve following by changing many parameters.
It is a time series problem, I have data from 60 shops, 215 items, 1034 days. I have splitted 973 days for train and 61 for test.:
train_x = train_x.reshape((60, 973, 215))
test_x = test_x.reshape((60, 61, 215))
train_y = train_y.reshape((60, 973, 215))
test_y = test_y.reshape((60, 61, 215))
My model:
model = Sequential()
model.add(LSTM(100, input_shape=(train_x.shape[1], train_x.shape[2]),
return_sequences='true'))
model.add(Dense(215))
model.compile(loss='mean_squared_error', optimizer='adam', metrics=
['accuracy'])
history = model.fit(train_x, train_y, epochs=10,
validation_data=(test_x, test_y), verbose=2, shuffle=False)
ValueError: Error when checking input: expected lstm_1_input to have
shape (973, 215) but got array with shape (61, 215)
You've split your data with respect to timesteps as opposed to the samples. You need to decide on what are your samples in the first instance. For the sake of the answer I will assume these are along the first axis (assuming the data has been framed as a supervised time-series problem).
The input_size in LSTM expects the shape of (timesteps, data_dim) as explained here, and these dimensions must remain the same for each batch. In your example, samples from training and testing have different dimensions. The batch size can differ (unless specified with batch_size parameter).
Your data should be split between training and testing along the first axis. Here is an analogous example from Keras tutorials:
from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np
data_dim = 16
timesteps = 8
num_classes = 10
# expected input data shape: (batch_size, timesteps, data_dim)
model = Sequential()
model.add(LSTM(32, return_sequences=True,
input_shape=(timesteps, data_dim))) # returns a sequence of vectors of dimension 32
model.add(LSTM(32, return_sequences=True)) # returns a sequence of vectors of dimension 32
model.add(LSTM(32)) # return a single vector of dimension 32
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# Generate dummy training data
x_train = np.random.random((1000, timesteps, data_dim))
y_train = np.random.random((1000, num_classes))
# Generate dummy validation data
x_val = np.random.random((100, timesteps, data_dim))
y_val = np.random.random((100, num_classes))
model.fit(x_train, y_train,
batch_size=64, epochs=5,
validation_data=(x_val, y_val))
You will notice that timesteps is the same for training and testing data and x_train.shape[1] == x_val.shape[1]. It is the number of samples that differs along the first axis x_train.shape[0] is 1000 and x_val.shape[0] is 100.
I want to implement Recurrent Neural network with GRU using Keras in python. I have problem in running code and I change variables more and more but it doesn't work. Do you have an idea for solve it?
inputs = 42 #number of columns input
num_hidden =50 #number of neurons in the layer
outputs = 1 #number of columns output
num_epochs = 50
batch_size = 1000
learning_rate = 0.05
#train (125973, 42) 125973 Rows and 42 Features
#Labels (125973,1) is True Results
model = tf.contrib.keras.models.Sequential()
fv=tf.contrib.keras.layers.GRU
model.add(fv(units=42, activation='tanh', input_shape= (1000,42),return_sequences=True)) #i want to send Batches to train
#model.add(tf.keras.layers.Dropout(0.15)) # Dropout overfitting
#model.add(fv((1,42),activation='tanh', return_sequences=True))
#model.add(Dropout(0.2)) # Dropout overfitting
model.add(fv(42, activation='tanh'))
model.add(tf.keras.layers.Dropout(0.15)) # Dropout overfitting
model.add(tf.keras.layers.Dense(1000,activation='softsign'))
#model.add(tf.keras.layers.Activation("softsign"))
start = time.time()
# sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
# model.compile(loss="mse", optimizer=sgd)
model.compile(loss="mse", optimizer="Adam")
inp = np.array(train)
oup = np.array(labels)
X_tr = inp[:batch_size].reshape(-1, batch_size, inputs)
model.fit(X_tr,labels,epochs=20, batch_size=batch_size)
However I get the following error:
ValueError: Error when checking target: expected dense to have shape (1000,) but got array with shape (1,)
Here, you have mentioned input vector shape to be 1000.
model.add(fv(units=42, activation='tanh', input_shape= (1000,42),return_sequences=True)) #i want to send Batches to train
However, shape of your training data (X_tr) is 1-D
Check your X_tr variable and have same dimension for input layer.
If you read the error carefully you would realize there is a shape mismatch between the shapes of labels you provide, which is (None, 1), and the shape of output of model, which is (None, 1):
ValueError: Error when checking target: <--- This means the output shapes
expected dense to have shape (1000,) <--- output shape of model
but got array with shape (1,) <--- the shape of labels you give when training
Therefore you need to make them consistent. You just need to change the number of units in the last layer to 1 since there is one output per input sample:
model.add(tf.keras.layers.Dense(1, activation='softsign')) # 1 unit in the output