We Keep Coding

Fitting the MNIST dataset to neural network giving “NotImplementedError”

I'm new to PyTorch and I'm using the classic MNIST dataset for image classification.While fitting the model, I'm getting the error :
NotImplementedError: uint8
I'm using fastai's library's class as a wrapper for all training and validation data and a very basic single-layer neural network. The code I'm using is as follows :
from keras.datasets import mnist
import matplotlib.pyplot as plt
from fastai.metrics import *
from fastai.model import *
from fastai.dataset import *
import torch.nn as nn
(x_train, y_train), (x_valid, y_valid) = mnist.load_data()
net = nn.Sequential(
nn.Linear(784,10),
nn.Softmax()).cuda()
md = ImageClassifierData.from_arrays('/data/mnist',
(x_train,y_train),
(x_valid, y_valid))
loss = nn.NLLLoss()
metrics = [accuracy]
opt=optim.SGD(net.parameters(), 1e-1, momentum=0.9, weight_decay=1e-3)
fit(net, md, n_epochs=3, crit=loss, opt=opt, metrics=metrics)
Can someone tell what is this error about and it's solution?

Keras multi-class classification process taking a long time

I started learning how to use Keras. I have a raw file that encodes ASCII values of characters in a sentence with a corresponding product name. For example, abcd toothpaste cream would be classified as Toothpaste. The first two lines (out of ~150,000 lines) of the code is shown below. The file is also available for download here (this link will last two months from today).
12,15,11,31,30,15,0,26,28,15,29,30,19,17,15,0,19,24,30,15,28,24,11,30,0,18,19,17,19,15,24,15,0,35,0,12,15,22,22,15,36,11,0,12,15,22,22,15,36,11,0,16,28,11,17,11,24,13,19,11,29,0,16,15,23,15,24,19,24,11,29,0,11,36,36,15,14,19,24,15,0,11,36,36,15,14,19,24,15,11,22,11,19,11,0,26,15,28,16,31,23,15,0,16,15,23,15,24,19,24,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,Body Care Other
12,15,19,15,28,29,14,25,28,16,0,30,18,11,19,22,11,24,14,0,13,25,0,22,30,14,0,29,21,19,24,13,11,28,15,0,26,28,15,26,11,28,11,30,19,25,24,29,0,16,11,13,19,11,22,0,13,22,15,11,24,29,15,28,29,0,24,19,32,15,11,0,16,11,13,19,11,22,0,13,22,15,11,24,29,15,28,29,0,26,28,25,14,31,13,30,29,0,24,19,32,15,11,0,23,11,21,15,0,31,26,0,13,22,15,11,28,0,23,19,13,15,22,22,11,28,0,33,11,30,15,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,Skin Care Other
I am following a blog post where it uses a simple deep learning Keras model to do multi-class classification. I changed the configuration of the neural network to 243 inputs --> [100 hidden nodes] --> 67 outputs (because I have 67 classes to classify). The code is below:
import numpy
import pandas
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
def baseline_model():
model = Sequential()
# I changed it to 243 inputs --> [100 hidden nodes] --> 67 outputs (because I have 67 classes to classify)
model.add(Dense(100, input_dim=X_len, activation='relu'))
model.add(Dense(Y_cnt, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
seed = 7
numpy.random.seed(seed)
# load dataset
dataframe = pandas.read_csv("./input/raw_mappings.csv", header=None)
dataset = dataframe.values
X_len = len(dataset[0,:-1])
X = dataset[:,0:X_len].astype(float)
Y = dataset[:,X_len]
Y_cnt = len(numpy.unique(Y))
# encode class values as integers
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
# convert integers to dummy variables (i.e. one hot encoded)
dummy_y = np_utils.to_categorical(encoded_Y)
estimator = KerasClassifier(build_fn=baseline_model, epochs=200, batch_size=5, verbose=0)
kfold = KFold(n_splits=10, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X, dummy_y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
But it never seems to finish when I ran it on my desktop computer for more than 12 hours. I'm starting to think there is almost nothing going on. Is there something that I'm doing wrong with either the configuration of the neural network or the problem I'm trying to solve (meaning, maybe Sequential model is not the right way to go for classifying >60 classes?).
Any pointer or tip would be greatly appreciated. Thank you.

Getting error in Keras while loading the model from load_model()

I am a beginner in Keras and I am writing a simple program for MNIST but when I tried to load the model I am getting this error:
ValueError: You are trying to load a weight file containing 2 layers into a model with 0 layers.
This is my code:
import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import np_utils
#fixing random number seed
np.random.seed(7)
(X_train, Y_train),(X_test, Y_test) = mnist.load_data("D:\MY CODE PROJECT\CNN\datasets\mnist.npz")
num_pixel = X_train.shape[1] * X_train.shape[2]
#converting image to vector
X_train = X_train.reshape(X_train.shape[0],num_pixel).astype('float32')
X_test = X_test.reshape(X_test.shape[0],num_pixel).astype('float32')
# Normalizing Input from 0-255 to 0-1
X_train = X_train/255
X_test = X_test/255
#As output is multiclass so change output labels to 'ONE-HOT' ecodings Form
Y_train = np_utils.to_categorical(Y_train)
Y_test = np_utils.to_categorical(Y_test)
#defining simple Neural Network with one hidden layer
num_classes = Y_test.shape[1]
#creating model
model = Sequential()
model.add(Dense(num_pixel,activation = 'relu',kernel_initializer='normal'))
model.add(Dense(num_classes, kernel_initializer='normal',activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
#Fitting the model
model.fit(X_train,Y_train,batch_size=200,epochs=10,verbose=2,validation_data=(X_test,Y_test))
scores = model.evaluate(X_test,Y_test,verbose=0)
#Printing Error
print("baseline Error: %f" %(100-scores[1]*100))
model.save('mnist_nn_keras.h5')
del model
model = load_model('mnist_nn_keras.h5')
Can anyone explain what's wrong in the code? I am using Keras version 2.2.0.

you need to add input_shape to your model while adding layer instance. read the documentation for add function it's clearly talking about error details.
Please see below screenshot.

Conv1D does not update the weights. (All zero) and Test outputs are always same which is equal to Last layer Weights?

I would like to use 1D CNN to predict next day solar energy. The time series data resolution is one hour and the length is one year. I am training the model with the data of day 1 to predict day 2. xtrain = day1, ytrain = day2, xtest = day3 to predict day4.
24 hour data input -> CNN -> 24 hour output
I have trained the data for 10 days (samples); then predicted 4 days in advance, the problem is that CNN always gives the same output value whatever the input is. Then, I checked the weights, only output layer weights are nonzero. Also, the output value does not change with different set of input.
The link for the data: https://mega.nz/#!NpoTzIBJ!U5l8ToQgcJ6xif2tMjIrXuace3skhrtwLEdeoWe_FkM
The image of weight matrix:
The graph of Predicted Values:
Code:
import pandas
from pandas import Series
from pandas import DataFrame
import keras
from keras.callbacks import ModelCheckpoint
from sklearn.metrics import r2_score
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, regularizers, initializers
from keras.layers import Conv1D, MaxPooling1D, Flatten, AveragePooling1D
from keras.activations import *
from keras.losses import *
from keras.optimizers import *
from keras.utils import plot_model
from keras.models import Sequential
from keras.utils import plot_model
import numpy as np
import matplotlib.pyplot as plt
from sklearn import preprocessing
from numpy import zeros, newaxis
from keras.callbacks import EarlyStopping
import sklearn.metrics
from sklearn.metrics import mean_squared_error
from math import sqrt
data = pandas.read_csv("meas.csv", header=0)
dataset = data.values[:,1]
all_pred_data = []
ytest_all = []
model = Sequential()
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Flatten())
model.add(Dense(42,activation='tanh',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))
model.add(Dense(24, activation='linear',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))
model.compile(loss='mse',
optimizer=keras.optimizers.Adam(),
metrics=['mae','accuracy'])
for i in range(0,10-1,1):
xtrain = dataset[24*(i+1)-24:24*(i+1)]
ytrain = dataset[24*(i+1):24*(i+2)]
xtrain = xtrain.reshape(1,24,1)
ytrain = ytrain.reshape(1,24)
model.fit(xtrain,ytrain,epochs=500,verbose=2)
# TEST
for i in range(20,25-1,1):
xtest = dataset[24*(i+1):24*(i+2)] #(i+1):(i+6+1)
ytest = dataset[24*(i+2):24*(i+2)+24]
xtest = xtest.reshape(1,xtrain.shape[1], 1)
pred_data = np.round(model.predict(xtest),3)
pred_data_transpose = pred_data.transpose()
all_pred_data.extend(pred_data_transpose)
ytest_all.extend(np.round(ytest,3))

If the weight went all to 0 they will never change because the gradient will always be 0, to avoid this try to normalize your data which make it easier to train the network, try also to use batch_normalization if normalizing the input data wasn't enough

Multi-class classification using keras

I am developing a neural network in order to classify with classes pre-calculated with k-means.
Dataset looks like:
50,12500,2,1,5
50,8500,2,1,15
50,6000,2,1,9
50,8500,2,1,15
Where resulting row is the last row.
Here is the code on Python with Keras I am trying to get working:
import numpy
import pandas
from keras.models import Sequential
from keras.layers import Dense,Dropout
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# load dataset
dataset = numpy.genfromtxt ('../r-calculations/k-means/output16.csv', delimiter=",")
X = dataset[:,0:4].astype(float)
Y = dataset[:,4]
print(Y[0])
Y = np_utils.to_categorical(Y)
model = Sequential()
model.add(Dense(5, activation='tanh', input_dim=4))
#model.add(Dropout(0.25))
model.add(Dense(10, activation='tanh'))
#model.add(Dropout(0.25))
model.add(Dense(10, activation='relu'))
#model.add(Dropout(0.25))
model.add(Dense(17, activation='softmax'))
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(X,Y, epochs=10, batch_size=10)
#print( model.predict(numpy.array([2,36,2,5,2384,1,2,4,3,1,1,4,33,3,1,1,2,1,1,1]).reshape((1,20))) )
#print( model.predict(numpy.array(X[0]).reshape((1,4))) )
#print( model.predict(numpy.array(X[1]).reshape((1,4))) )
#print( model.predict(numpy.array(X[2]).reshape((1,4))) )
result = model.predict(numpy.array(X[0]).reshape((1,4)))
for res in result[0]:
print res
If I get it right, now I am getting a probability for each class as an output. How can I retrieve labels back after I have called "to_categorical" on it?
Is there a way to get a class number, instead of probability for each class?
For now it does not seem to be working right, big loss ~2, accuracy ~0.29 and I cannot make it to converge. What am I doing wrong?
UPDATE Mar 19
So far I have solved my problem, I changed my model a lot of times and finally found working configuration.

If you want the class instead of the probability you could call numpy argmax at your predictions.
Or use the convenient call predict_classes instead of predict
result = model.predict_classes(numpy.array(X[0]).reshape((1,4)))
As for your result, you could try running a few extra epochs, but it is hard to say what is wrong. Could be your training data quality, bad initialization, not having enough data, bad model (i'd use only relu activations).