We Keep Coding

AttributeError: 'function' object has no attribute 'fit'

I'm just starting with deep learning and python and I'm already stuck with this error when I try to train the model.
I thought it would be an easy starting project to get together the basic building blocks, but I obviously haven't grasped some basics..
My goal is to train a model on a data set of 5 column values '1ex','2ex','3ex','4ex','5ex' and predict sequences of 5 values.
I'm reading the dataset from a csv file i generated and it's displaying as expected.
Can you help me understanding what I'm missing out?
Thank you very much as always.
This is the code I've written so far:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import os
from sklearn.preprocessing import MinMaxScaler
from collections import deque
import random
# load the data set
df = pd.read_csv('DataSet.csv',delimiter=',',usecols=['Wheel','Date','1ex','2ex','3ex','4ex','5ex'])
# divide it into portions
times = sorted(df.index.values) # get the times
last_10pct = sorted(df.index.values)[-int(0.1*len(times))] # get the last 10% of the times
last_20pct = sorted(df.index.values)[-int(0.2*len(times))] # get the last 20% of the times
test_df = df[(df.index >= last_10pct)]
validation_df = df[(df.index >= last_20pct) & (df.index < last_10pct)]
train_df = df[(df.index < last_20pct)] # now the train_df is all the data up to the last 20%
# drop 'Date' column
train_df.drop(columns=["Date"], inplace=True)
validation_df.drop(columns=["Date"], inplace=True)
test_df.drop(columns=["Date"], inplace=True)
# the model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
# from tensorflow.keras.layers import LSTM
from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
# define base model
def baseline_model():
# scale = StandardScaler()
# create model
model = Sequential()
model.add(Dense(5, input_dim=5, kernel_initializer='normal', activation='relu'))
model.add(Dense(15, kernel_initializer='normal', activation='relu'))
model.add(Dense(15, kernel_initializer='normal', activation='relu'))
model.add(Dense(5, kernel_initializer='normal', activation = softmax))
# Compile model
# model.compile(loss='mean_absolute_error', optimizer='adam')
model.compile(loss='mean_squared_error', optimizer='adam')
# model.fit(train_df, epochs = 5)
return model
# train the model
baseline_model.fit(train_df, batch_size=1, epochs=200, verbose=1)

If you write baseline_model, it returns the function, not the result.
Therefore baseline_model.fit can't be called because 'function' object has no attribute 'fit'
You must execute the function to get its result, using parentheses - baseline_model() - and then fit will be performed on the result. ;)
tl;dr:
baseline_model.fit( -> baseline_model().fit(

Keras estimator predict says that input is misshapen

I'm getting a numpy shape error when I use the predict function of a Keras estimator. I build, evaluate, and then retrain the model using the following code:
import pandas as pd
import sqlalchemy as sqla
import numpy
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from keras.utils.np_utils import to_categorical
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
# Connect to to the DB and retrieve the iris table
con = sqla.create_engine('postgresql://tristan:sebens#db:5432/tristan')
con.connect()
table_name = "iris"
schema = "public"
iris = pd.read_sql_table(table_name, con, schema=schema)
iris.head()
iris_ds = iris.values # Convert the table to a numpy array
X = iris_ds[:, 0:4].astype(float) # Slice the descriptive features into a numpy array
Y = iris_ds[:, 4] # Slice the labels away as their own numpy array
# The labels are encoded as strings, so we need to encode them
# as numbers that can be output by an ANN
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
# convert integers to dummy variables (i.e. one hot encoded)
dummy_y = to_categorical(encoded_Y)
# define baseline model
def baseline_model():
# create model
model = Sequential()
model.add(Dense(8, input_dim=4, activation='relu'))
model.add(Dense(3, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
seed = 7
# Train the model:
# First we define the model as a classifier. This will affect the process used to train it
estimator = KerasClassifier(build_fn=baseline_model, epochs=200, batch_size=5, verbose=0)
# Honestly not totally sure what this is, but it has to do with splitting the training/evaluation data in
# a way that gives us a more realistic metric of the model's accuracy
kfold = KFold(n_splits=10, shuffle=True, random_state=seed)
# Now that we have our classifier and our data pipeline defined, we can begin the training process
results = cross_val_score(estimator, X, dummy_y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
# If we like our accuracy, then we can train the model for real
# Evaluating the model actually evaluates a clone of the model, so now we need to train the model again
estimator.fit(X, dummy_y)
And this is where the trouble is. I try to make a test prediction:
# Let's make a test prediction with our model
x = X[0]
estimator.predict(x)
And I get an input shape error:
ValueError: Error when checking input: expected dense_21_input to have shape (4,) but got array with shape (1,)
I'm at a loss. How can the input have the wrong shape if it's literally a member of the training dataset?

Replicating sklearn.neural_network.MPLRegressor with Keras

I'm currently working with sklearn.neural_network.MPLRegressor but looking for more fexibility in the definition of the neural network, I tried Keras.
I create the following dataset
import numpy as np
x = np.linspace(-0.5,5,2000)
y = np.tanh(x)
y=y.reshape(-1, )
x=x.reshape(-1, 1)
and implement the trivial neural network with sklearn
from sklearn.neural_network import MLPRegressor
model = MLPRegressor(hidden_layer_sizes=(1,),activation='tanh',solver='lbfgs',verbose=True,validation_fraction = 0.1)
model.fit(x,y)
which is solved with a high degree of precision MSE 7.252229995056021e-10
I replicate the same nn with Keras
from keras.layers import Input, Dense
from keras.models import Model
import keras
from keras.wrappers.scikit_learn import KerasRegressor
keras.initializers.RandomUniform()
visible = Input(shape=(1,))
hidden1 = Dense(1, activation='tanh')(visible)
output = Dense(1,activation="linear")(hidden1)
model = Model(inputs=visible, outputs=output)
model.compile(optimizer='adam',metrics=['mean_squared_error'],loss =['mean_squared_error'] )
model.fit(x, y,validation_split=0.1)
but I obtain MSE 0.02130996414874087 (the result doesn't change a lot using different optimizers).
Since the nn can be solved analytically, I would expect a higher degree of precision.
Does anyone know the reason behind such difference, if any (apart from the different optimizer)?
Thanks!

Keras multi-class classification process taking a long time

I started learning how to use Keras. I have a raw file that encodes ASCII values of characters in a sentence with a corresponding product name. For example, abcd toothpaste cream would be classified as Toothpaste. The first two lines (out of ~150,000 lines) of the code is shown below. The file is also available for download here (this link will last two months from today).
12,15,11,31,30,15,0,26,28,15,29,30,19,17,15,0,19,24,30,15,28,24,11,30,0,18,19,17,19,15,24,15,0,35,0,12,15,22,22,15,36,11,0,12,15,22,22,15,36,11,0,16,28,11,17,11,24,13,19,11,29,0,16,15,23,15,24,19,24,11,29,0,11,36,36,15,14,19,24,15,0,11,36,36,15,14,19,24,15,11,22,11,19,11,0,26,15,28,16,31,23,15,0,16,15,23,15,24,19,24,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,Body Care Other
12,15,19,15,28,29,14,25,28,16,0,30,18,11,19,22,11,24,14,0,13,25,0,22,30,14,0,29,21,19,24,13,11,28,15,0,26,28,15,26,11,28,11,30,19,25,24,29,0,16,11,13,19,11,22,0,13,22,15,11,24,29,15,28,29,0,24,19,32,15,11,0,16,11,13,19,11,22,0,13,22,15,11,24,29,15,28,29,0,26,28,25,14,31,13,30,29,0,24,19,32,15,11,0,23,11,21,15,0,31,26,0,13,22,15,11,28,0,23,19,13,15,22,22,11,28,0,33,11,30,15,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,Skin Care Other
I am following a blog post where it uses a simple deep learning Keras model to do multi-class classification. I changed the configuration of the neural network to 243 inputs --> [100 hidden nodes] --> 67 outputs (because I have 67 classes to classify). The code is below:
import numpy
import pandas
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
def baseline_model():
model = Sequential()
# I changed it to 243 inputs --> [100 hidden nodes] --> 67 outputs (because I have 67 classes to classify)
model.add(Dense(100, input_dim=X_len, activation='relu'))
model.add(Dense(Y_cnt, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
seed = 7
numpy.random.seed(seed)
# load dataset
dataframe = pandas.read_csv("./input/raw_mappings.csv", header=None)
dataset = dataframe.values
X_len = len(dataset[0,:-1])
X = dataset[:,0:X_len].astype(float)
Y = dataset[:,X_len]
Y_cnt = len(numpy.unique(Y))
# encode class values as integers
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
# convert integers to dummy variables (i.e. one hot encoded)
dummy_y = np_utils.to_categorical(encoded_Y)
estimator = KerasClassifier(build_fn=baseline_model, epochs=200, batch_size=5, verbose=0)
kfold = KFold(n_splits=10, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X, dummy_y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
But it never seems to finish when I ran it on my desktop computer for more than 12 hours. I'm starting to think there is almost nothing going on. Is there something that I'm doing wrong with either the configuration of the neural network or the problem I'm trying to solve (meaning, maybe Sequential model is not the right way to go for classifying >60 classes?).
Any pointer or tip would be greatly appreciated. Thank you.

LSTM implementation in keras Using specific dataset

I am trying to understand how LSTM RNNs work and how they can be implemented in Keras in order to be able to solve a binary classification problem. My code and the dataset i use are visible below. When i compilr the code i get an error TypeError: __init__() got multiple values for keyword argument 'input_dim', Can anybody help?
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers.embeddings import Embedding
from keras.layers import Dense
from sklearn.cross_validation import train_test_split
import numpy
from sklearn.preprocessing import StandardScaler # data normalization
seed = 7
numpy.random.seed(seed)
dataset = numpy.loadtxt("sorted output.csv", delimiter=",")
X = dataset[:,0:4]
scaler = StandardScaler(copy=True, with_mean=True, with_std=True ) #data normalization
X = scaler.fit_transform(X) #data normalization
Y = dataset[:4]
# split into 67% for train and 33% for test
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, random_state=seed)
# create model
model = Sequential()
model.add(Embedding(12,input_dim=4,init='uniform',activation='relu'))
model.add(Dense(4, init='uniform', activation='relu'))
model.add(LSTM(100))
model.add(Dense(1, init='uniform', activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X_train, y_train, validation_data=(X_test,y_test), nb_epoch=150, batch_size=10)

Looks like two separate questions here.
Regarding how to use LSTMs / Keras, there are some good tutorials around. Try this one which also describes a binary classification problem. If you have a specific issue or area that you don't understand, let me know.
Regarding the file opening issue, perhaps the whitespace in the filename is causing an issue. Check out this answer to see if it helps.

This is in fact a case where the error message you are getting is perfectly to-the-point. (I wish this would always be the case with Python and Keras...)
Keras' Embedding layer constructor has this signature:
keras.layers.embeddings.Embedding(input_dim, output_dim, ...)
However, you are constructing it using:
Embedding(12,input_dim=4,...)
So figure out which is the input and output dimension, respectively, and fix your parameter order and names. Based on the table you included in the question, I'm guessing 4 is your input dimension and 12 is your output dimension; then it'd be Embedding(input_dim=4, output_dim=12, ...).