We Keep Coding

Train a model with 2 stacked models in it keras

I have the folowing models that i want to train (See image below):
The model has an input of 20. The model A has an input of 10 (the first 10 elements of the initial input), the model B has an input of 10 (the last 10 elements of the initial input) finally the input of the model C is the concatenation of the output of the models A and B.
How can I train this 3 models at the same time in Keras? Can I merge it in one big model? (I only have data to train the big model)

Can I merge it in one big model?
Yes!
How can I train this 3 models at the same time in Keras?
I will give you pointers:
Use functional APIs. Want to know how it is different from sequential? Look here
Use concatenate layer - Reference

Lets assume that you have your three models defined, and named model_A, model_B and model_C. You can now define you complete model somewhat like this (I did not check the exact code):
def complete_model(model_A, model_B, model_C):
input_1 = layers.Input(shape=(10,))
input_2 = layers.Input(shape=(10,))
model_A_output = model_A(input_1)
model_B_output = model_B(input_2)
concatenated = tf.concat([model_A_output, model_B_output], axis=-1)
model_C_output = model_C(concatenated)
model = Model(inputs=[input_1, input_2], outputs=model_C_output)
model.compile(loss=losses.MSE)
model.summary()
return model
This requires you to give two-dimensional inputs, so you have to do some numpy slicing to preprocess your inputs.
If you still want your one-dimensional inputs, you can just define a single input layer with shape (20,) and then use the tf.split function to split it in half and feed it into the next networks.

What is the meaning of the result of model.predict() function for semantic segmentation?

I use Segmentation Models library for multi-class (in my case 4 class) semantic segmentation. The model (UNet with 'resnet34' backbone) is trained with 3000 RGB (224x224x3) images. The accuracy is around 92.80%.
1) Why model.predict() function requires (1,224,224,3) shaped array as input ? I didn't find the answer even in the Keras documentation. Actually, below code is working, I have no problem with it but I want to understand the reason.
predictions = model.predict( test_image.reshape(-1,224,224,3) );
2) predictions is a (1,224,224,3) shaped numpy array. Its data type is float32 and contains some floating numbers. What is the meaning of the numbers inside this array? How can I visualize them? I mean, I assumed that the result array will contain one of 4 class label (from 0 to 3) for every pixel, and then I will apply the color map for each class. In other words, the result should have been a prediction map, but I didn't get it. To understand better what I mean about prediction map, please visit the Jeremy Jordan's blog about semantic segmentation.
result = predictions[0]
plt.imshow(result) # import matplotlib.pyplot as plt
3) What I finally want to do is like Github: mrgloom - Semantic Segmentation Categorical Crossentropy Example did in visualy_inspect_result function.

1) Image input shape in your deep neural network architecture is (224,224,3), so width=height=224 and 3 color channels. And you need an additionnal dimension in case you want to give more than one image at a time to your model. So (1,224,224,3) or (something, 224,224,3).
2) According to the doc of Segementation models repo, you can specify the number of classes you want as output model = Unet('resnet34', classes=4, activation='softmax'). Thus if you reshape your labelled image to have a shape (1,224,224,4). The last dimension is a mask channel indicating with a 0 or 1 if pixel i,j belongs to class k. Then you can predict and access to each output mask
masked = model.predict(np.array([im])[0]
mask_class0 = masked[:,:,0]
mask_class1 = masked[:,:,1]
3) Then using matplotlib you will be able to plot semantic segmentation or using scikit-image : color.label2rgb function

How to get output with maximum probability from the all the predicted outputs from dense layer?

I trained a neural network for sign language recognition. Here's my output layer model.add(Dense(units=26,activation="softmax"))
Now I'm getting probability for all 26 alphabets. Somehow I'm getting 99% accuracy when I test this model accuracy = model.evaluate(x=test_X,y=test_Y,batch_size=32). I'm new at this. I can't understand how this code works and I'm missing something major here. How to get a 1D list having just the predicted alphabet in it?

To get probabilities you need to do something like this:
prediction = model.predict(test_X)
probs = prediction.max(1)
But it is important to remember that softmax doesn't exactly provide probabilities of each class.

To get outputs with maximum probability in a single list, run:
np.argmax(model.predict(x_test),axis=1)

Supposing alphabet is a list with all alphabet symbols alphabet = ['a', 'b', ...]
pred = model.predict(test_X)
pred_ind = pred.max(1)
pred_alphabet = [alphabet[ind] for ind in pred_ind]
will give you the list with predicted symbols.

In neural networks first layer is for the input image you have. Let's say your image is 32x32 pixels. In that case you would have 32x32x3 nodes in the input layer. This 3 comes for the RGA color scheme. Then depending on your design and model you should use appropriate number of hidden input layers. At most scenarios we use 2 hidden input layers. Then the final layer is for the number of distinct classes you have. Let's say you're going to identify 26 distinct signs. Then you will have 26 nodes in the final layer.
model.evaluate(x=test_X,y=test_Y,batch_size=32)
I think here you're trying to make predictions on your test data set. At first you may have separated your data set into train and test sets. Here test_X stands for the images in test set. test_Y stands for corresponding labels. You're trying to evaluate your network by taking 32 images at a time. That's the meaning of batch_size=32.
I think this information might helpful for you to understand what you're doing. But your question is not clear. Please refer the below tutorial. That might helpful for you.
https://www.pyimagesearch.com/2018/09/10/keras-tutorial-how-to-get-started-with-keras-deep-learning-and-python/

Multiple Input types in a keras Neural Network

As an example, I'd like to train a neural network to predict the location of a picture(longitude, latitude) with the image, temperature, humidity and time of year as inputs into the model.
My question is, what is the best way to add this addition information to a cnn? Should I just merge the numeric inputs with the cnn in the last dense layer or at the beginning? Should I encode the numeric values (temperature, humidity and time of year)?
Any information, resources, sources would be greatly appreciated, thanks in advance.

You can process numeric inputs separately and merge them afterwards before making the final prediction:
# Your usual CNN whatever it may be
img_in = Input(shape=(width, height, channels))
img_features = SomeCNN(...)(img_in)
# Your usual MLP model
aux_in = Input(shape=(3,))
aux_features = Dense(24, activation='relu')(aux_in)
# Possibly add more hidden layers, then merge
merged = concatenate([img_features, aux_features])
# create last layer.
out = Dense(num_locations, activation='softmax')(merged)
# build model
model = Model([img_in, aux_in], out)
model.compile(loss='categorical_crossentropy', ...)
Essentially, you treat them as separate inputs and learn useful features that combined allow your model to predict. How you encode numeric inputs really depends on their type.
For continuous inputs like temperature you can normalize between -1, 1 for discrete inputs one-hot is very often. Here is a quick guide.

If you want to predict basis on those four features then i would suggest go with cnn + rnn
so feed the image to cnn and take the logits after that make a sequence like
logits=np.array(output).flatten()
[[logits] , [temperature], [humidity] , [time_of_year]] and feed it to
rnn , Rnn will treat it like a sequence input.

Trend "Predictor" in Python?

I'm currently working with data frames (in pandas) that have 2 columns: the first column is some numeric quantitative data, like weight, amount of money spent on some day, GPA, etc., and the second column are date values, i.e. the date on which the corresponding column 1 entry was added on.
I was wondering, is there a way to "predict" what the next value after time X is going to be in Python? E.g. if I have 100 weight entries spanning over 2-3 months (not all entries have the same time difference, so 1 entry could be during Day 3, the next Day 5, and the next Day 10), and wanted to "predict" what my next entry after 1 month, is there a way to do that?
I think this has something to do with Time Series Analysis, but my statistical background isn't very strong, so I don't know if that's the right approach. If it is, how could I apply it to my data frames (i.e. which packages)? Would there be any significance to the value it potentially returns, or would it be meaningless in the context of what I'm working with? Thank you.

For predicting time-series data, I feel the best choice would be a LSTM, which is a type of recurrent neural network, which are well suited for time-series regression.
If you don't want to dive deep into the backend of neural networks, I suggest using the Keras library, which is a wrapper for the Tensorflow framework.
Lets say you have a 1-D array of values and you want to predict the next value. Code in Keras could look like:
#start off by building the training data, let arr = the list of values
X = []
y = []
for i in range(len(arr)-100-1):
X.append(arr[i:i+100]) #get prev 100 values for the X
y.append(arr[i+100]) # predict next value for Y
Since an LSTM takes a 3-D input, we want to reshape our X data to have 3 dimensions:
import numpy as np
X = np.array(X)
X = X.reshape(len(X), len(X[0]), 1)
Now X is in the form (samples, timesteps, features)
Here we can build a neural network using keras:
from keras.models import Sequential
from keras.layers import Dense, LSTM
model = Sequential()
model.add(LSTM(input_shape = (len(X[0], 1)) #input 3-D timeseries data
model.add(Dense(1)) #output 1-D vector of predicted values
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, y)
And viola, you can use your model to predict the next values in your data

Statsmodels is a python module that provides one of the "most famous" methods in time series forecasting (Arima).
An example can be seen in the following link : https://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/
Other methods for time series forecasting are available in some libraries, like support vector regression, Holt-Winters and Simple Exponential Smoothing.
Spark-ts (https://github.com/sryza/spark-timeseries) is one time series library that supports Python , and provides methods like Arima, Holt-Winters and Exponential Weighted Moving Average.
Libsvm (https://github.com/cjlin1/libsvm) provides support vector regression methods.