Classification using neuronal networks - python

I have built up a NN for classification, but when trying to compile I get problems with the dimensions of input and output:
from keras.models import Sequential
from keras.layers import Dense
# data splited into input (X) and output (y) variables
model = Sequential()
model.add(Dense(12, input_dim=456, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(8, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
Here are the dimensions of my y and X
print(y.shape, X.shape)
(8000, 1) (8000, 456, 3)
I have 8000 sub sets which contain 456 particles(x,y,z);
and I have labels which are in y ranging from 0 to 7; this is also why my output layer has 8 nodes.
But when I fit with
model.fit(X, y, epochs=15, batch_size=10)
I do not get why this error occurs:
ValueError: Error when checking input: expected dense_26_input to have 2 dimensions, but got array with shape (8000, 456, 3)
Any suggestions?


To answer your question, you can achieve what you want by doing :
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
model = Sequential()
model.add(Dense(12, input_shape=(456,3), activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(8, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
Edit:
I think what you're looking for is that type of architecture :
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Flatten
model = Sequential()
model.add(Dense(12, input_shape=(456,3), activation='relu'))
model.add(Flatten())
model.add(Dense(8, activation='relu'))
model.add(Dense(8, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
So that it only output the 8 labels

Related

I tried this code for LSTM and GRU. But the GRU is not working

I am not sure where i am wrong in this code. My goal is to train my dataset for binary classification using LSTM and GRU.
[the output comes with module wrapper and GRU not executing please check the image][1]
#BUILD THE MODEL
top_words = 10000
embedding_vecor_length = 32
model = Sequential()
model.add(Embedding(top_words, embedding_vecor_length, input_length=X.shape[1]))
#model.add(Dropout(0.2))
model.add(GRU(100,dropout=0.2, recurrent_dropout=0.2, return_sequences=True))
model.add(LSTM(100,dropout=0.2, recurrent_dropout=0.2))
#model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='Adam',
metrics=['accuracy'])
print(model.summary())
model.summary()
```
[1]: https://i.stack.imgur.com/14pyl.jpg

Keras producing model with no accuracy

I have the following code for training a model based off of some numbers:
from numpy import loadtxt
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from time import sleep
dataset = loadtxt("data.csv", delimiter=",")
X = dataset[:,0:2]
y = dataset[:,2]
model = Sequential()
model.add(Dense(196, input_dim=2, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, y, epochs=600, batch_size=10)
_, accuracy = model.evaluate(X, y)
print('Accuracy: %.2f' % (accuracy*100))
For reference, here is some of the data it is being presented with:
433,866,1299,1732
421,842,1263,1684
443,886,1329,1772
142,284,426,568
437,874,1311,1748
455,910,1365,1820
172,344,516,688
219,438,657,876
101,202,303,404
289,578,867,1156
110,220,330,440
421,842,1263,1684
472,944,1416,1888
121,242,363,484
215,430,645,860
134,268,402,536
488,976,1464,1952
467,934,1401,1868
418,836,1254,1672
134,268,402,536
241,482,723,964
116,232,348,464
395,790,1185,1580
438,876,1314,1752
396,792,1188,1584
57,114,171,228
218,436,654,872
372,744,1116,1488
305,610,915,1220
462,924,1386,1848
455,910,1365,1820
42,84,126,168
347,694,1041,1388
394,788,1182,1576
184,368,552,736
302,604,906,1208
326,652,978,1304
333,666,999,1332
335,670,1005,1340
176,352,528,704
168,336,504,672
62,124,186,248
26,52,78,104
335,670,1005,1340
(The first three numbers should be inputs, and the last one an output)
The Keras program keeps training but only warrants an accuracy of 0. What am I doing wrong?

Like discussed in comments, this is a regression problem (not classification), so we can use, for example, mse (mean squared errors) as a loss function, and change activation of the last layer to linear:
X = dataset[:,0:3]
y = dataset[:,3]
model = Sequential()
model.add(Dense(196, input_dim=3, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mse', optimizer='adam')
model.fit(X, y, epochs=600, batch_size=10)

Problem with reducing the loss of a Neural Network

I have a quite large data and a binary classification problem, which I want to train with neural network, I used more than 10 combination for my NN structure varying from 3 layers to 20, I also tried to over-fit my model on a smaller sample for the purpose of debugging, but my loss does not reduce at all!! It get stuck on 0.4 after number of epochs every times with every combinations and every sample sizes! But the strange thing is that the accuracy also does not reduce and it's around 0.8 which is not very bad!
As I'm new in working with NNs, any suggestion for solving the problem?
I use keras sequential model libraries.
here is my designed neural network:
#Normalizing the data
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X = sc.fit_transform(X)
print(X)
from keras import utils
y = utils.to_categorical(y)
print(y)
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size = 0.9)
#Dependencies
import keras
from keras.models import Sequential
from keras.layers import Dense
# Neural network
model = Sequential()
model.add(Dense(24, input_dim=25, activation='relu'))
model.add(Dense(22, activation='relu'))
model.add(Dense(20, activation='sigmoid'))
model.add(Dense(18, activation='selu'))
model.add(Dense(18, activation='sigmoid'))
model.add(Dense(16, activation='relu'))
model.add(Dense(16, activation='sigmoid'))
model.add(Dense(14, activation='tanh'))
model.add(Dense(12, activation='relu'))
model.add(Dense(10, activation='sigmoid'))
model.add(Dense(8, activation='sigmoid'))
model.add(Dense(8, activation='selu'))
model.add(Dense(7, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(4, activation='sigmoid'))
model.add(Dense(2, activation='sigmoid'))
model.add(Dense(2, activation='sigmoid'))
opt = keras.optimizers.Adam(learning_rate=0.01)
model.compile(loss='categorical_crossentropy', optimizer = opt , metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=5000, batch_size=1000)

How to apply model.fit() function over an CNN-LSTM model?

I am trying to use this to classify the images into two categories. Also I applied model.fit() function but its showing error.
ValueError: A target array with shape (90, 1) was passed for an output of shape (None, 10) while using as loss binary_crossentropy. This loss expects targets to have the same shape as the output.
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, LSTM
import pickle
import numpy as np
X = np.array(pickle.load(open("X.pickle","rb")))
Y = np.array(pickle.load(open("Y.pickle","rb")))
#scaling our image data
X = X/255.0
model = Sequential()
model.add(Conv2D(64 ,(3,3), input_shape = (300,300,1)))
# model.add(MaxPooling2D(pool_size = (2,2)))
model.add(tf.keras.layers.Reshape((16, 16*512)))
model.add(LSTM(128, activation='relu', return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)
model.compile(loss='binary_crossentropy', optimizer=opt,
metrics=['accuracy'])
# model.summary()
model.fit(X, Y, batch_size=32, epochs = 2, validation_split=0.1)

If your problem is categorical, your issue is that you are using binary_crossentropy instead of categorical_crossentropy; ensure that you do have a categorical instead of a binary classification problem.
Also, please note that if your labels are in simple integer format like [1,2,3,4...] and not one-hot-encoded, your loss_function should be sparse_categorical_crossentropy, not categorical_crossentropy.
If you do have a binary classification problem, like said in the error of the above ensure that:
Loss is binary_crossentroy + Dense(1,activation='sigmoid')
Loss is categorical_crossentropy + Dense(2,activation='softmax')

Keras Model for Molecular Activity

I am experimenting with the Merku molecular activity challenge and I have created the train and test dataset.
The shape of the data is the following:
x_train.shape=(1452, 4306)
y_train.shape=(1452, 1)
x_test.shape=(363, 4306)
y_test.shape=(363, 1)
I have used the Dense layer for defining the model as follows:
model = Sequential()
model.add(Dense(100, activation="relu", input_shape=(4306,)))
model.add(Dense(50, activation="relu"))
model.add(Dense(25, activation="relu"))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1))
# Compile the model
model.compile(
loss='categorical_crossentropy',
optimizer="adam",
)
model.summary()
# Train the model
model.fit(
x_train,
y_train,
batch_size=300,
epochs=900,
validation_data=(x_test, y_test),
shuffle=True
)
While trying the above code, the following error occurred:
ValueError: Input 0 is incompatible with layer flatten_23: expected min_ndim=3, found ndim=2
How can I resolve this error?

Just remove the flatten layer:
model = Sequential()
model.add(Dense(100, activation="relu", input_shape=(4306,)))
model.add(Dense(50, activation="relu"))
model.add(Dense(25, activation="relu"))
model.add(Dropout(0.25))
model.add(Dense(1))
The data sent to sequential layers is essentially 1-D (ignoring the batch column) so there's nothing to flatten. The data entering the flatten layer is already 1D.
EDIT -- for regression:
Categorical crossentropy is not an appropriate cost function for regression, you need to use mean-square error, which is commonly used for all regression tasks:
model.compile(
loss='mse',
optimizer="adam",
)

Categories