We Keep Coding

I am trying Time series forecasting using CNN , LSTM and MLP. But when i use TimeDistributed with CNN it gives dimensionality error, during fitting

I have 9 features, one output variable i.e. to be predicted, window size is 5
code works very well without "TimeDistributed" command
MODEL INPUT SHAPE: feature_tensor.shape=(1649, 5, 9) MODEL OUTPUT
SHAPE: y_train.shape= (1649,)
Thats my Code:
#Build the network model
act_fn='relu'
modelq = Sequential()
modelq.add(TimeDistributed(Conv1D(filters=105, kernel_size=2, activation=act_fn, input_shape=(None, feature_tensor.shape[1],feature_tensor.shape[2]))))
modelq.add(TimeDistributed(AveragePooling1D(pool_size=1)))
modelq.add(TimeDistributed(Flatten()))
modelq.add(LSTM(50))
modelq.add(Dense(64, activation=act_fn))
modelq.add(Dense(1))
#Compile the model
modelq.compile(optimizer='adam', loss='mean_squared_error')
modelq.fit(feature_tensor, y_train ,batch_size=1, epochs=epoch_count)
THE ERROR STATEMENT IS :
ValueError: Input 0 of layer conv1d is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: (5, 9)
I feel like there is some thing wrong with dimensionality of "feature_tensor" during "Model FITTING" i.e last command... But I don't know what's wrong with it :(

Your intuition is right, the problem is the dimensionality of tensor_feature. If you take a look in the documentation of TimeDistributed you see an example with images and Conv2d layers. There the the input has to have the following shape: batch_size, time steps, x_dim, y_dim, channels. Since you use time-series you need: batch_size, time steps, 1, features. E.g. you can reshape your data by numpy:
feature_tensor = np.reshape(feature_tensor, (-1, 5, 1, 9))
However, I am not sure if it useful to combine Conv1D with TimeDistributed, since in that case you apply the convolution only on the features and not on temporal contiguous values, where a 1d Convolution should be applied.

Shape mismatch, 2D Input & 2D Labels

I want to create a neural network, that -easy speaking- creates an image out of an image (greyscale)
I have successfully created a dataset of 3200 examples of input and output (label) images.
(I know the dataset should be larger but that is not the problem right now)
The input [Xin] has the size (3200, 50, 30), since it is 50*30 pixels
The output [yout] has the size of (3200, 30, 20) since it is 30*20 pixels
I want to try out a fully connected network (later on a CNN)
The built of the fully connected model looks like that:
# 5 Create Model
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(256, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(30*20, activation=tf.nn.relu))
#compile the model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# 6 Train the model
model.fit(Xin, yout, epochs=1) #train the model
After that I get the following error:
ValueError: Shape mismatch: The shape of labels (received (19200,)) should equal the shape of logits except for the last dimension (received (32, 600)).
I already tried to flatten yout:
youtflat = yout.transpose(1,0,2).reshape(-1,yout.shape[1]*yout.shape[2])
but this resulted in the same error

It appears you're flattening your labels (yout) completely, i.e., you're losing batch dimension. If your original yout has a shape of (3200, 30, 20) you should reshape it to have a shape of (3200, 30*20) which equals (3200, 600):
yout = numpy.reshape((3200, 600))
Then it should work
NOTE
The suggested fix however only removes the error. I see many problems with your method though. For the task you're trying to perform (getting an image as output), you cannot use sparse_categorical_crossentropy as loss and accuracy as metrics. You should use 'mse' or 'mae' instead.

Python, Keras, RNN: cannot copy sequence with size 7 to array axis with dimension 5

When I Run this code:
model = Sequential([
LSTM(300, activation='tanh', input_shape=data.shape, recurrent_activation='sigmoid'),
Dropout(0.2),
Dense(4, activation='linear')
])
model.compile(optimizer='adam', loss='mse')
history = model.fit_generator(training, epochs=22, steps_per_epoch=1)
This error pops up:
ValueError: cannot copy sequence with size 7 to array axis with dimension 5
I don't know how to handle that. Yesterday, the code worked perfectly. In other threads I've found; the error might be related to np.vstack. This is the thread I'm writing about.
I've tried to change the data to a np.array but it did not work. Can anyone help?
This might be useful:
training = TimeseriesGenerator(data, data, length=7, batch_size=6)
TimeseriesGenerator is from keras.preprocessing.sequence

You need input numpy arrays into Keras' TimeSeries generator, not directly DataFrames, you can do data.values for pass numpy only.

Error when checking input in first layer of Keras Conv1D

I'm trying to assign one of two classes (positive/nagative) to audio using CNN with Keras. My model should accept varied lengths of input (frames) in which each frame contains 41 features but I struggle with the input size. Bear in mind that I haven't acquired full dataset so I just mocked some meaningless data just to check if network works at all.
According to documentation https://keras.io/layers/convolutional/ and my best understanding Conv1D can tackle varied lengths if first element of input_shape tuple is None. Shape of variable containing input data X_train.shape is (4, 497, 41).
data = pd.read_csv('output_file.csv', sep=';')
featureCount = data.values.shape[1]
#mocks because full data is not available yet
Y_train = np.asarray([1, 0, 1, 0])
X_train = np.asarray(
[np.array(data.values, copy=True), np.array(data.values, copy=True), np.array(data.values, copy=True),
np.array(data.values, copy=True)])
# variable length with 41 features
model = keras.models.Sequential()
model.add(keras.layers.Conv1D(100, 5, activation='relu', input_shape=(None, featureCount)))
model.add(keras.layers.GlobalMaxPooling1D())
model.add(keras.layers.Dense(10, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
model.summary()
model.fit(X_train, Y_train, epochs=10, verbose=False, validation_data=(np.array(data.values, copy=True), [1]))
This code produces error
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (497, 41). So it appears like the first dimension was cut out as it contains training samples (it seems correct to me) what bothered me is the required dimensionality, why is it 3?
After searching for the answer I stumbled onto Dimension of shape in conv1D and followed it by adding last dimension (using X_train = np.expand_dims(X_train, axis=3)) that contains only single digit but I ended up with another, similar error:
ValueError: Error when checking input: expected conv1d_input to have 3 dimensions, but got array with shape (4, 497, 41, 1) now it seems that first dimension that previously was treated as sample "list" is now part of actual data.
I also tried fiddling with input_shape parameter but to no avail and using Reshape layer but ended up fighting with size the
What should I do to satisfy required shape? How to prepare data for processing?

What is the timestep in Keras' LSTM?

I have some troubles with the LSTM implementation in Keras.
My training set is structured as follow:
number of sequences: 5358
the length of each sequence is 300
each element of the sequence is a vector of 54 features
I'm unsure on how to shape the input for a stateful LSTM.
Following this tutorial: http://philipperemy.github.io/keras-stateful-lstm/, I've created the subsequences (in my case there are 1452018 subsequences with a window_size = 30).
What is the best option to reshape the data for a stateful LSTM's input?
What means the timestep of the input in this case? And why?
Is the batch_size related to the timestep?

I'm unsure on how to shape the input for a stateful LSTM.
LSTM(100, statefull=True)
But before using stateful LSTM ask yourself do I really need statefull LSTM? See here and here for more details.
What is the best option to reshape the data for a stateful LSTM's
input?
It really depends on the problem on hands. However, I think you do not need reshaping just feed data directly into Keras:
input_layer = Input(shape=(300, 54))
What means the timestep of the input in this case? And why?
In your example timestamp is 300. See here for further details on timestamp. In the following picture, we have 5 timestamps that we feed them into the LSTM network.
Is the batch_size related to the timestep?
No, it has nothing to do with batch_size. More details on batch_size can be found here.
Here is simple code based on the description that you provide. It might give you some intuition:
import numpy as np
from tensorflow.python.keras import Input, Model
from tensorflow.python.keras.layers import LSTM
from tensorflow.python.layers.core import Dense
x_train = np.zeros(shape=(5358, 300, 54))
y_train = np.zeros(shape=(5358, 1))
input_layer = Input(shape=(300, 54))
lstm = LSTM(100)(input_layer)
dense1 = Dense(20, activation='relu')(lstm)
dense2 = Dense(1, activation='sigmoid')(dense1)
model = Model(inputs=input_layer, ouputs=dense2)
model.compile("adam", loss='binary_crossentropy')
model.fit(x_train, y_train, batch_size=512)