We Keep Coding

LSTM Inputs Confusion

I've been trying to understand LSTM inputs for a while now and I think I understand but I keep getting confused on how to implement them.
This is what I think, please correct me if I am wrong.
When specifying an LSTM, you specify the number of cells and the input shape (I've been having issues with the input shape). The Number of cells specifies how many cells should look at the data given and does not affect the required input shape. The input shape (When Stateful) goes by batch size, Timesteps in a batch, and features in a time step. A stateful LSTM retains it's Internal States until reset. Is this right?
If so I'm having so much confusion trying to specify the input shape for my network. This is because I'm trying to upgrade a current network and I cant figure out how and where to specify the input shape without an error.
The way I'm trying to upgrade it is initially I have a CNN going to a dense layer. I'm trying to change it such that it adds an LSTM that takes the CNN's flattened 1D output as one batch and one time step with features dependent on the size of the CNN's Output. Then concatenates its output with the CNN's output (The LSTM's Input) then feeds into the dense layer. Thus it now behaves like LSTM with a skip connection. The issue that I cant seem to understand is when and how to specify the LSTM layer's Input_shape as it has NO INPUT_SHAPE parameter for the functional API? Or maybe I'm just super confused, Everyone uses different API's going over different examples and it gets super confusing what is and isn't specified and how.
Thank you, even if you just help with one of the two parts.
TLDR:
Do I understand LSTM Parameters correctly?
How and when do I specify LSTM Input_shapes if at all?

LSTM units argument means dimensions of LSTM matrices and output shape.
With Functional API you can specify input shape for the very first layer only. If your LSTM layer follows CNN - then its input shape is determined automatically as CNN output.

How to build an end-to-end NLP RNN classification model?

I have trained a NLP model with RNN on keras to classify tweets with word embeddings (Stanford GloVe) used as a feature selection method. I would like to apply this model trained onto new tweets extracted. However, this error appears when I try to apply the model to new data.
ValueError: Error when checking input: expected input_1 to have shape (22,) but got array with shape (51,)
Then I realised that the model trained is expecting an input with a 22-input vector (the max tweet length in the training set tweets). On the other hand, the new dataset I would like to apply the model to has a 51-input vector (the max tweet length in the new dataset tweets).
In attempt to tackle this, I increased the size of the vector when training the model to 51 so both would be balanced. A new error pops up:
InvalidArgumentError: indices[29,45] = 5870 is not in [0, 2489)
Thus, I decided to try to apply the model back on the training dataset to see if it was possible in the first place with the original parameters and model. It was not and a very similar error appeared.
InvalidArgumentError: indices[23,11] = 2489 is not in [0, 2489)
In this case, how can I export an end-to-end NLP RNN classification model to apply on new unseen data? (FYI: I was able to successfully to do this for Logistic Regression with TF-IDF used as a feature selection. There just seems to be a lot of issues with the Word Embeddings.)
===========
UPDATE:
I was able to solve this issue by pickling not only the model, but also variables such as the max_len, texttotensor_instance and tokenizer. When applying the model to new data, I will have to use the same variables generated from the training data (instead of redefining them with the new data).

Your error is because the maximum number of words in your training data exceeds the max in the Embeddings layer (aka. input_dim).
It seems that the input_dim param. in your Embeddings layer is set to 2489, where you have words in your dataset tokenized and mapped to a higher value (5870).
Also don't forget to add one to the maximum # of words when you set this in the Embedding layer (input_dim=max_number_of_words+1). If you're interested to know why check this question: Keras embedding layer masking. Why does input_dim need to be |vocabulary| + 2?

What does input_shape,input_dim and units indicate or mean while adding layers in a Keras?

Im new to keras and i was wondering if I could do some work regarding text classification using neural networks.
So i went ahead and got a data set regarding spam or ham and I vectorized the data using tfidf and converted the labels to a numpy array using the to_categorical() and managed to split my data into train and test each of which is a numpy array having around 7k columns.
This the code i used.
model.add(Dense(8,input_dim=7082,activation='relu'))
model.add(Dense(8,input_dim=7082))
model.add(Dense(2,activation='softmax'))
model.compile(loss="categorical_crossentropy",optimizer="adam",metrics=['accuracy'])
I dont know if im doing something totally wrong. Could someone point me in the right direction as to what i should change.
The error thrown:
Error when checking input: expected dense_35_input to have 2 dimensions, but got array with shape ()

Dense layers doesn't seem to have any input_dim parameter according to the documentation.
input_shape is a tuple and must be used in the first layer of your model. It refers to the shape of the input data.
units refers to the dimension of the output space, that is the shape of each output element processed by the dense layer.
In your case, if your input data has is of dimensionality 7082, this should work:
model.add(Dense(8,input_shape=(7082,),activation='relu'))
model.add(Dense(2,activation='softmax'))
model.compile(loss="categorical_crossentropy",optimizer="adam",metrics=['accuracy'])

I could not use my keras model for do a prediction because of one hot encoding size error

I just trained my data by using keras.Before adding model i just one hot encoded the entire data for improving accuracy.
My model gives more than 90% accuracy while training.
After trained my neural network i have tried to predict another same type data by using the trained model.When i was trying to predict it gives below error.
ValueError: Error when checking input: expected lstm_1_input to have shape (10, 133) but
got array with shape (10, 119)
I know ofcourse it was happen because of One hot encoding size.
so friends please tell me if there is any other method to predict the data?
Many thanks in advance

You need to be attentive at two different things.
Keras only accepts batch_predictions. For example, if you have an image with size (1024,1024,3), i.e. a RGB image, you cannot feed the image as it is to a neural network. One needs to simulate the batch index. So practically, when you feed an image to your network it corresponds to a batch with one size. People usually use np.expand_dims(axis = 0), this adds the batch axis. The same is valid for LSTM predictions.
While the first paragraph was written for you to understand this issue of feeding test data, w.r.t to your actual error, it is clear that you do not feed the test data in the same way you feed your training data.
What you should to is to see the exact shape of the data before you use .fit() or .fit_generator(). Print it, and feed the exact input shape to your neural network when testing. Also, do not forget about the batch_size trick!
In your case, your test data needs to have 133 the second dimension, not 119. Make sure that your small amount of data has 133 value on the second dimension; after this it will definitely work.

Tensorflow: Array activation1, which is an input to the Div operator producing the output array dropout/div, is lacking min/max data

I'm using tensorflow 1.8.0rc1. I'm trying to save a very simple NN model to tflite format, with weight quantization, following this documentation: https://www.tensorflow.org/performance/quantization.
However, when converting with toco, I receive this error:
Array Relu, which is an input to the Div operator producing the output array dropout/div, is lacking min/max data, which is necessary for quantization. Either target a non-quantized output format, or change the input graph to contain min/max information, or pass --default_ranges_min= and --default_ranges_max= if you do not care about the accuracy of results.\n"
And this is the graph:
At some point it was not complaining about RELU, but Assign operations (fixed that no idea how), and if I remove the RELU layers it complains about the Add layers. Any idea what's going on?
EDIT:
Just realized that between dropout_1 and activation2 (see picture) there's an act_quant node that must be the fake quantization of activation2 (a RELU). This is not happening in the first layer, between dropout and activation1. I guess this the problem? According to the tensorflow quantization tutorial (attached before) the scripts described there should rewrite the graph with all the necessary information for toco to quantize the weights.