Keras model's validation accuracy is not changing, and when I try to run the model it's just outputting same messy numbers. As input I'm giving it flatten 5x4 array of bounding boxes, and the output are keypresses (W,A,S,D).
I'm using it to avoid detected objects for autopilot.
I tried changing activation, layers in model, learning rate and so on, but nothing helped.
Any help would be appreciated.
from tensorflow import keras
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, Input
from tensorflow.keras.callbacks import TensorBoard
data = np.load("training_data/data.npz",allow_pickle=True)
X = data['arr_2']
X = keras.utils.normalize(X,axis=1)
Y = data['arr_1']
# Creating model for object processing
model2 = Sequential()
# Model for object processing
# Inputs
model2.add(Input(shape=(20)))
# Hidden layer
model2.add(Dense(16,activation="relu"))
# Outputs
model2.add(Dense(4,activation="softmax"))
model2.compile(loss='binary_crossentropy',
optimizer="adam",
metrics=['accuracy'])
model2.fit(X,Y,batch_size=256, epochs=20, validation_split=0.1)
Related
I'm trying to search Hyperparameters for a model using Keras Tuner, but I'm getting this error when I run the code: "RuntimeError: Model-building function did not return a valid Keras Model instance, found < keras.engine.sequential.Sequential object at 0x000001E9C2903F28 >"
I've searched on Internet but didn't found anything that could help, also I've followed the tutorial in the Keras Tuner gitHub page (https://github.com/keras-team/keras-tuner), but it dind't work either.
Here is my code:
class MyHyperModel(HyperModel):
def __init__(self, num_classes):
self.num_classes = num_classes
def build(self, hp):
model=Sequential()
model.add(Dense(units=hp.Int('units_0', 30, 900, step=30),
activation=hp.Choice('act_0', ['relu', 'tanh']),
input_dim=12))
for i in range(hp.Int('layers', 3, 9)):
model.add(Dense(units=hp.Int('units_' + str(i), 30, 900, step=30),
activation=hp.Choice('act_' + str(i), ['relu', 'tanh'])))
model.add(Dense(6, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer=hp.Choice('optimizer', ['adam', 'sgd']),
metrics=['categorical_accuracy'])
return model
hypermodel = MyHyperModel(num_classes=6)
tuner = kt.tuners.bayesian.BayesianOptimization(
hypermodel,
objective='val_accuracy',
max_trials=5,
executions_per_trial=3,
seed=(np.random.seed(1)),
directory='Tests',
project_name='test')
tuner.search_space_summary()
tuner.search(data[:200], labels[:200],
verbose=2,
epochs=3,
validation_data=(data[200:], labels[200:]))
models = tuner.get_best_models(num_models=2).summary()
tuner.get_best_hyperparameters()[0].values
tuner.results_summary()
data is an list of 300 vector with 12 values and on labels there are 6 classes which was converted to tensor with the function tensorflow.convert_to_tensor().
I appreciate any help.
If you import module members from keras, you must import from tensorflow.keras instead of keras. For example, if you write:
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout
from keras.optimizers import Adam
Then change them to:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Dropout
from tensorflow.keras.optimizers import Adam
I know what's wrong and is not the code, my model have 6 neurons on the last layer and I've used the loss as 'categorical_crossentropy', but this only works when the labels are 0 and 1, so I've changed the loss to 'sparse_categorical_crossentropy' and metrics to 'accuracy' and it worked.
Thanks everyone for the reply, I appreciate the help.
Hi I followed a forum from Github about removing the last layers of a pre-trained model. However it's not working for me; perhaps I did something wrong
I'm following this and here's my code. I thought all I had to do was model.layers[-2].output but it's telling me AttributeError: 'Tensor' object has no attribute 'summary'
import tensorflow as tf
import tensorflow.keras as keras
import tensorflow.keras.backend as K
import numpy as np
from tensorflow.keras.layers import Dense, Input, Layer
from tensorflow.keras.models import Model
from tensorflow.keras.applications.inception_v3 import InceptionV3
model = InceptionV3()
print(model.summary())
modele = model.layers[-2].output
print(modele.summary())
modele variable is only a layer. You have to do:
model = InceptionV3()
print(model.summary())
output = model.layers[-2].output
modele = Model(inputs = model.input, outputs = output)
print(modele.summary())
I have a code to train MNIST dataset to work on the street view house number project, but when I run the code I have acc = 0,1
Import libraries and modules
import numpy as np
np.random.seed(123) # for reproducibility
from keras import backend as K
K.set_image_dim_ordering('th')
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Convolution2D, MaxPooling2D
from keras.utils import np_utils
from keras.datasets import mnist
from keras.models import load_model
from keras.utils import CustomObjectScope
from keras.initializers import glorot_uniform
4. Load pre-shuffled MNIST data into train and test sets
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
X_train = keras.utils.normalize(X_train,axis=1)
X_test = keras.utils.normalize(X_test, axis=1)
7. Define model architecture
model = Sequential()
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(10, activation='softmax'))
8. Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
9. Fit model on training data
model.fit(np.array(X_train), np.array(Y_train), batch_size=32, epochs=3,verbose=1)
In step 4, are you normalizing the data correctly? If I recall correctly X_train has shape batch, width, height. I don't really know what you would want to normalize, but axis=1 doesn't seem it's supposed to be there. I think you should the normalization.
If you still have low accuracy, try training more epochs than 3. 3 epochs are not that many.
There are several reasons why you have an accuracy like that.
Your data is not normalized correctly.
You are trying to do image recognition with 3 dense layers in 3 epochs, which will not work.
There is no optimization in your code.
Look at the https://keras.io/examples/mnist_cnn/ . It is the Keras documentation on working with the MNIST data using neural network.
regarding input shapes – have been using LSTM for a while and didn’t have any problems with it but now I tried 1D convolutional layers for speeding up processing and now I run into trouble – can you see what the problem is with the following? (Dummy data used here)
I get an error for the fitting:
ValueError: Error when checking target: expected dense_17 to have 2
dimensions, but got array with shape (400, 20, 2)
I cannot see what is wrong here?!
Code is shown below
#load packages
import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, GRU,
TimeDistributed
from keras.layers import Conv1D, MaxPooling1D, Flatten,
GlobalAveragePooling1D
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import np_utils
nfeat, kernel, timeStep, length, fs = 36, 8, 20, 100, 100
#data (dummy)
data = np.random.rand(length*fs,nfeat)
classes = 0*data[:,0]
classes[:int(length/2*fs)] = 1
#make correct input shape (batch, timestep, feature)
X = np.asarray([data[i*timeStep:(i + 1)*timeStep,:] for i in
range(0,length * fs // timeStep)])
#classes
Y = np.asarray([classes[i*timeStep:(i + 1)*timeStep] for i in
range(0,length * fs // timeStep)])
#split into training and test set
from sklearn.model_selection import train_test_split
trainX, testX, trainY, testY =
train_test_split(X,Y,test_size=0.2,random_state=0)
#one-hot-encoding
trainY_OHC = np_utils.to_categorical(trainY)
trainY_OHC.shape, trainX.shape
#set up model with simple 1D convnet
model = Sequential()
model.add(Conv1D(8,10,activation=’relu’,input_shape=(timeStep,nfeat)))
model.add(MaxPooling1D(3))
model.add(Flatten())
model.add(Dense(10,activation=’tanh’))
model.add(Dense(2,activation=’softmax’))
model.summary()
#compile model
model.compile(loss=’mse’,optimizer=’Adam’ ,metrics=[‘accuracy’])
#train model
model.fit(trainX,trainY_OHC,epochs=5,batch_size=4,
validation_split=0.2)
I am trying to design a Bi-Directional LSTM model and I want to concatenate features after Max pooling and Average pooling layers.
I have this for my model:
from keras.layers import Dense, Embedding
from keras.layers.recurrent import LSTM
from keras.layers import Bidirectional
from keras.models import Sequential
from keras.layers.core import Dropout
from features import train,embedding_matrix,words
from keras.layers import concatenate,AveragePooling1D,GlobalMaxPooling1D
model=Sequential()
model.add(Embedding(words,300,input_length=train.shape[1],weights=[embedding_matrix]))
model.add(Bidirectional(LSTM(20,activation='tanh',kernel_initializer='glorot_uniform',recurrent_dropout = 0.2, dropout = 0.2,return_sequences=True)))
model.add(concatenate([GlobalMaxPooling1D(),AveragePooling1D()]))
model.add(Dropout(0.2))
model.add(Dense(2, activation='softmax'))
print model.summary()
But I am having:
ValueError: Layer concatenate_1 was called with an input that isn't a symbolic tensor which is because I believe the concatenating layer. As I am not adding the pooling in the model.
Can I add two layers in the same model? or Should I define two separate models and then add pooling layers in each of them?
The trick here is to use a graph model instead of a sequential model.
Before we get started, I assume
your network expects a 2D input tensor of shape (B=batch_size, N=num_of_words), where N is the longest sample length of your training data. (In case you have unequal length samples, you should use keras.preprocessing.sequence.pad_sequences to achieve equal length samples)
your vocabulary size is V (probably is 300 if I understand correctly)
your embedding layer encodes each word to a feature of F dimension, i.e. your embedding layer's weight matrix is VxF.
from keras.layers import Dense, Embedding, Input, Concatenate, Lambda
from keras.layers.recurrent import LSTM
from keras.layers import Bidirectional
from keras.models import Model
from keras.layers.core import Dropout
from keras import backend as BKN
from keras.layers import concatenate,AveragePooling1D,GlobalMaxPooling1D
words = Input( shape=(N,))
f = Embedding(input_dim=V,output_dim=F)( words )
f = Bidirectional(LSTM(20,activation='tanh',
kernel_initializer='glorot_uniform',
recurrent_dropout = 0.2,
dropout = 0.2,return_sequences=True))(f)
gpf = GlobalMaxPooling1D()(f)
gpf = Lambda( lambda t : BKN.expand_dims(t, axis=1) )(gpf)
apf = AveragePooling1D( pool_size=2 )(f)
pf = Concatenate(axis=1)([gpf, apf])
pf = Dropout(0.2)( pf )
pred = Dense(2, activation='softmax')(pf) # <-- make sure this is correct
model = Model( input=words, output=pred )
Finally, I fail to find that keras Embedding layer supports syntax like weights=[embedding_matrix].