I am testing the code below.
#%matplotlib inline
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.model_selection import cross_validate
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegressionCV
iris = sns.load_dataset("iris")
iris.head()
sns.pairplot(iris, hue='species')
X = iris.values[:, 0:4]
y = iris.values[:, 4]
train_X, test_X, train_y, test_y = train_test_split(X, y, train_size=0.5, random_state=0)
lr = LogisticRegressionCV()
lr.fit(train_X, train_y)
pred_y = lr.predict(test_X)
print("Test fraction correct (Accuracy) = {:.2f}".format(lr.score(test_X, test_y)))
# Test fraction correct (Accuracy) = 0.93
import keras
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.utils import np_utils
train_y_ohe = pd.get_dummies(train_y)
test_y_ohe = pd.get_dummies(test_y)
model = Sequential()
model.add(Dense(16, input_shape=(4,)))
model.add(Activation('sigmoid'))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
loss, accuracy = model.evaluate(test_X, test_y_ohe, show_accuracy=True, verbose=0)
print("Test fraction correct (Accuracy) = {:.2f}".format(accuracy))
Everything works fine until the next-to-last-line of code.
When I try to run this:
loss, accuracy = model.evaluate(test_X, test_y_ohe, show_accuracy=True, verbose=0)
I get this error:
TypeError: evaluate() got an unexpected keyword argument 'show_accuracy'
I did a bit of research, and found that 'show_accuracy=True' may have been depreciated a short time ago. Is there some other way of doing this now? How can I evaluate, and print, the accuracy of the model?
I found the code sample here:
https://blog.fastforwardlabs.com/2016/02/24/hello-world-in-keras-or-scikit-learn-versus.html
The show_accuracy argument is deprecated in new versions of keras,remove this argument from model.evaluate() and use instead metrics=['accuracy'] in model.compile()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit model
train_y_ohe = pd.get_dummies(train_y)
model.fit(train_X, train_y_ohe,epochs=1000,batch_size=20)
loss, accuracy = model.evaluate(test_X, test_y_ohe, verbose=0)
print("Test fraction correct (Accuracy) = {:.2f}".format(accuracy))
#Test fraction correct (Accuracy) = 0.97
Related
I am really new with deep learning. I want to do a task which asks: Evaluate the model on the test data and compute the mean squared error between the predicted concrete strength and the actual concrete strength. You can use the mean_squared_error function from Scikit-learn.
here is my code:
import pandas as pd
from tensorflow.python.keras import Sequential
from tensorflow.python.keras.layers import Dense
from sklearn.model_selection import train_test_split
concrete_data = pd.read_csv('https://cocl.us/concrete_data')
n_cols = concrete_data.shape[1]
model = Sequential()
model.add(Dense(units=10, activation='relu', input_shape=(n_cols-1,)))
model.compile(loss='mean_squared_error',
optimizer='adam')
y = concrete_data.Cement
x = concrete_data.drop('Cement', axis=1)
xTrain, xTest, yTrain, yTest = train_test_split(x, y, test_size = 0.3)
model.fit(xTrain, yTrain, epochs=50)
and now to evaluate mean square error I wrote this :
from sklearn.metrics import mean_squared_error
predicted_y = model.predict(xTest)
mean_squared_error(yTest, predicted_y)
and I got this error:
y_true and y_pred have different number of output (1!=10)
my predicted_y shape is : (309, 10)
I googled it and I really couldn't find an answer to solve this problem. I don't know what is wrong with my code.
Your y_test data shape is (N, 1) but because you put 10 neurons in output layer, your model makes 10 different predictions which is the error.
You need to change the number of neurons in the output layer to 1 or add a new output layer which has only 1 neuron.
The below code probably works for you.
import pandas as pd
from tensorflow.python.keras import Sequential
from tensorflow.python.keras.layers import Dense
from sklearn.model_selection import train_test_split
concrete_data = pd.read_csv('https://cocl.us/concrete_data')
n_cols = concrete_data.shape[1]
model = Sequential()
model.add(Dense(units=10, activation='relu', input_shape=(n_cols-1,)))
model.add(Dense(units=1))
model.compile(loss='mean_squared_error',
optimizer='adam')
y = concrete_data.Cement
x = concrete_data.drop('Cement', axis=1)
xTrain, xTest, yTrain, yTest = train_test_split(x, y, test_size = 0.3)
model.fit(xTrain, yTrain, epochs=50)
Actually, what you are trying to check is the mean_squared_error of y_test and the predicted_y
You have to check what your model predict on x_test, which is the prediction :
predicted_y = model.predict(x_test)
Then you can calculate the error:
mean_squared_error(y_test, predicted_y)
y_pred = model.predict(x_test).sum(axis=1)
Try this, it worked for me
So I'm trying to train a neural network and at no point does it appear that pickle is even being used, so I'm somewhat confused. Here's the details:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
X.shape
(149, 8)
y.shape
(149,)
# define base model
def baseline_model():
# create model
model = Sequential()
model.add(Dense(149, input_dim=149, kernel_initializer='normal', activation='relu'))
model.add(Dense(75))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
estimator = KerasRegressor(build_fn=baseline_model(), epochs=100, batch_size=5, verbose=2)
kfold = KFold(n_splits=3)
results = cross_val_score(estimator, X, y, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
Which is where the problem arises...
TypeError: can't pickle _thread._local objects
What am I doing wrong?
You need to pass a callable function as build_fn in KerasRegressor. Removing the rounds brackets should make it works.
Following your code, chage:
estimator = KerasRegressor(build_fn=baseline_model(), epochs=100, batch_size=5, verbose=2)
into:
estimator = KerasRegressor(build_fn=baseline_model, epochs=100, batch_size=5, verbose=2)
so I used deeplearning to improve my model accuracy, but when I check with bayesian classifier I got 91.67% accuracy
then I check with deep learning, but it doesn't improve max I get 91.67%
I have to improve my accuracy, I want to try using Tuning, but I don't know how
My dataset has 3 class
So please help me, at least I get 92% accuracy
import pandas
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
# load dataset
dataframe = pandas.read_csv("pca_aug.csv", header=None)
dataset = dataframe.values
X = dataset[:,0:300].astype(float)
Y = dataset[:,300]
xtrain,xtest,ytrain,ytest= train_test_split(X,Y,test_size=0.4,random_state=0)
# encode class values as integers
def konversi(Y):
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
# convert integers to dummy variables (i.e. one hot encoded)
dummy_y = np_utils.to_categorical(encoded_Y)
return dummy_y
ytrain_dummy= konversi(ytrain)
ytest_dummy= konversi(ytest)
# create model
model = Sequential()
model.add(Dense(1000, input_dim=300, activation='relu'))
model.add(Dense(3, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
nepochs = 20
nbatch = 5
model.fit(xtrain, ytrain_dummy, epochs=nepochs, batch_size=nbatch)
_, accuracy = model.evaluate(xtest, ytest_dummy)
print('Accuracy: %.2f' % (accuracy*100))
You can't set a limit on the accuracy, before training the model, it will introduce biases to the results. The performance metric depends on the data, noise present in the data, training, and the model.
You can use a hyperparameter searching library like keras-tuner.
import kerastuner as kt
from tensorflow import keras
def build_model(hp):
...
return model
tuner = kt.RandomSearch(
build_model,
objective='val_loss',
max_trials=5)
tuner.search(x_train, y_train, epochs=5, validation_data=(x_val, y_val))
best_model = tuner.get_best_models()[0]
So basically, I am working on this bullet optimization program. I wish to study how different ballistics parameters such as weight, length, and mass affect a ballistics coefficient. However, my training accuracy is 0, although there is loss and val_loss. I've read similar Stackoverflow posts regarding this, but none have helped me so far. Perhaps I just didn't do them right; I am referencing https://stackoverflow.com/a/63513872/12349188
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.utils import shuffle
df = pd.read_csv('Bullet Optimization\ShootForum Bullet DB_2.csv')
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
dataset = df.values
X = dataset[:,0:12]
X = np.asarray(X).astype(np.float32)
y = dataset[:,13]
y = np.asarray(y).astype(np.float32)
X_train, X_val_and_test, y_train, y_val_and_test = train_test_split(X, y, test_size=0.3, shuffle=True)
X_val, X_test, y_val, y_test = train_test_split(X_val_and_test, y_val_and_test, test_size=0.5)
from keras.models import Sequential
from keras.layers import Dense, BatchNormalization
model = Sequential(
[
#2430 is the shape of X_train
#BatchNormalization(axis=-1, momentum = 0.1),
Dense(32, activation='relu'),
Dense(32, activation='relu'),
Dense(1,activation='softmax'),
]
)
model.compile(optimizer='sgd', loss='binary_crossentropy', metrics=['accuracy'])
history = model.fit(X_train, y_train, batch_size=64, epochs=100, validation_data=(X_val, y_val))
Did I do something wrong in my code? I know some python but I just kind of built upon the tutorials for my own purposes.
There are several problems in your code.
First this line:
Dense(1,activation='softmax')
This line will cause the output 1 every time. So even if you are making classification, your accuracy would be 50% if you had 2 classes. Softmax outputs' sum will be equal to one. So using it with one neuron does not make sense.
You need to change your loss and metric as this is a regression.
loss='mse', metrics=['mse']
Also your output neuron should be linear which means does not need any activation function. It should be like:
Dense(1)
I am new to KerasRegressor. I am trying to run neural net following this codes:
import tensorflow as tf
import numpy as np
from sklearn import datasets, linear_model
from sklearn.model_selection import cross_val_score, KFold
from keras.models import Sequential
from sklearn.metrics import accuracy_score
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
seed = 1
def baseline_model():
model = Sequential()
model.add(Dense(10, input_dim=10, activation='relu'))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
return model
estimator = KerasRegressor(build_fn=baseline_model, nb_epoch=100, batch_size=100, verbose=False)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(estimator, X_train,y_train, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
estimator.fit(X_train, y_train)
prediction = estimator.predict(X_test)
Then I stumbled the following errors:
ValueError: Input 0 of layer sequential_45 is incompatible with the layer: expected axis -1 of input shape to have value 10 but received input with shape [None, 39]
UPDATE: I fixed the error, it was in input_dim, which should be change to X_train.shape[-1]. However, I have a very large error, even after I try normalized X_train:
Results: -1674844.52 (3109620.06) MSE
I wonder how I should solve this? Thanks!