First project with pytorch and I got stuck trying to convert an MNIST label 'int' into a torch 'Variable'. Debugger says it has no dimension?!
# numpy mnist data
X_train, Y_train = read_data("training")
X_test , Y_test = read_data("testing")
arr = np.zeros(5)
for i in range(5):
# in your training loop:
costs_ = 0
for k in range(10000):
optimizer.zero_grad() # zero the gradient buffers
a = torch.from_numpy(np.expand_dims(X_train[k].flatten(), axis=0)).float()
b = torch.from_numpy(np.array(Y_train[k], dtype=np.float)).float()
input = Variable(a)
output = net(input)
target = Variable(b) # PROBLEM!!
loss = criterion(output, target)
loss.backward()
optimizer.step() # Does the update
costs_ += loss.data.numpy()
arr[i] = costs_
print(i)
Thrown error is: "RuntimeError: input and target have different number of elements: input[1 x 1] has 1 elements, while target[] has 0 elements at /b/wheel/pytorch-src/torch/lib/THNN/generic/MSECriterion.c:12"
The error is telling you exactly what is happening. Your target variable is empty.
Edit (after the comment below):
if Y_train[k] = 5, then np.array(Y_train[k], dtype=np.float).shape = (), and in turn Variable(b) becomes a tensor with no dimension.
In order to fix this you will need to pass a list to np.array() and not a integer or a float.
Like this:
b = torch.from_numpy(np.array([Y_train[k]], dtype=np.float)).float()
Related
I want to get the accuracy of my train section of my neuronal network
But i get this error:
correct += (prediction.argmax(1) == y).type(torch.float).item()
ValueError: only one element tensors can be converted to Python scalars
With this code :
def train_loop(dataloader, model, optimizer):
model.train()
size = len(dataloader.dataset)
correct = 0, 0
l_loss = 0
for batch, (X, y) in enumerate(dataloader):
prediction = model(X)
loss = cross_entropy(prediction, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
correct += (prediction.argmax(1) == y).type(torch.float).sum().item()
loss, current = loss.item(), batch * len(X)
l_loss = loss
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
correct /= size
accu = 100 * correct
train_loss.append(l_loss)
train_accu.append(accu)
print(f"Accuracy: {accu:>0.1f}%")
I don't understand why it is not working becaus in my test section it work perfektly fine with execly the same code line.
item function is used to convert a one-element tensor to a standard python number as stated in the here. Please try to make sure that the result of the sum() is only a one-element tensor before using item().
x = torch.tensor([1.0,2.0]) # a tensor contains 2 elements
x.item()
error message: ValueError: only one element tensors can be converted to Python scalars
Try to use this:
prediction = prediction.argmax(1)
correct = prediction.eq(y)
correct = correct.sum()
print(correct) # to check if it is a one value tensor
correct_sum += correct.item()
Further to my previous post,
LSTM-CNN to classify sequences of images,
I got further and I am now running the below code:
loop += 1
if loop % 50 == 0:
# calculate Accuracy
correct = 0
total = 0
# Iterate through the test dataset
for j in range(iter):
sequences = x_test[bs:bs + batch_size, :]
labels_t = y_test[bs:bs + batch_size]
test_images = dataset.load_images(sequences)
bs += batch_size
test_images = (torch.from_numpy(test_images)).view(-1, 4, 784)
print('test_image_shape {} - iter: {}'.format(test_images.shape, j))
labels_t = torch.from_numpy(labels_t)
labels_t = torch.argmax(labels_t, dim=1)
test_outputs = model(test_images).float()
pred_y = torch.max(test_outputs, 1)[1].data.numpy().squeeze()
total += labels_t.size(0)
correct += (pred_y == labels_t).float()
accuracy = 100 * (correct / total)
print('Iteration:{}. Loss:{}. Accuracy{}:'.format(loop, loss.item(), accuracy))
However it is giving me the following error:
correct += (pred_y == labels_t).float()
AttributeError: 'bool' object has no attribute 'float'
correct is always resulting in 0 and so is accuracy.
I am suspecting that it has to do with a certain package or requirement, because I searched and it seems that it should work. Any idea what could be missing here?
That's because pred_y is that numpy.ndarray and labels_t is torch.tensor.
If you compare two variables as torch.tensor, you can get float value
but the variables' type is not equal.
so python said tow variables are not the same type(so return False)
you use below code for convert torch.tensor to numpy.ndarray
pred_y = torch.max(test_outputs, 1)[1].data.numpy().squeeze()
I think you can implement like this.
pred_y = torch.max(test_outputs, 1)[1].squeeze()
Please try this one!
I am still grappling with PyTorch, having played with Keras for a while (which feels a lot more intuitive).
Anyway - I have the nn.linear model code below, which works fine for just one input feature, where:
inputDim = 1
I am now trying to expand the same code to include 2 features, and so I have included another column in my feature dataframe and also set:
inputDim = 2
However, when I run the code, I get the dreaded error:
RuntimeError: mat1 dim 1 must match mat2 dim 0
This error references line 63, which is:
outputs = model(inputs)
I have gone through several other posts here relating to this dimensionality error, but I still can't see what is wrong with my code. Any help would be appreciated.
The full code looks like this:
import numpy as np
import pandas as pd
import torch
from torch.autograd import Variable
import matplotlib.pyplot as plt
device = 'cuda' if torch.cuda.is_available() else 'cpu'
df = pd.read_csv('Adjusted Close - BAC-UBS-WFC.csv')
x = df[['BAC', 'UBS']]
y = df['WFC']
# number_of_features = x.shape[1]
# print(number_of_features)
x_train = np.array(x, dtype=np.float32)
x_train = x_train.reshape(-1, 1)
y_train = np.array(y, dtype=np.float32)
y_train = y_train.reshape(-1, 1)
class linearRegression(torch.nn.Module):
def __init__(self, inputSize, outputSize):
super(linearRegression, self).__init__()
self.linear = torch.nn.Linear(inputSize, outputSize)
def forward(self, x):
out = self.linear(x)
return out
inputDim = 2
outputDim = 1
learningRate = 0.01
epochs = 500
# Model instantiation
torch.manual_seed(42)
model = linearRegression(inputDim, outputDim)
if torch.cuda.is_available(): model.cuda()
criterion = torch.nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learningRate)
# Model training
loss_series = []
for epoch in range(epochs):
# Converting inputs and labels to Variable
inputs = Variable(torch.from_numpy(x_train).cuda())
labels = Variable(torch.from_numpy(y_train).cuda())
# Clear gradient buffers because we don't want any gradient from previous epoch to carry forward, dont want to cummulate gradients
optimizer.zero_grad()
# get output from the model, given the inputs
outputs = model(inputs)
# get loss for the predicted output
loss = criterion(outputs, labels)
loss_series.append(loss.item())
print(loss)
# get gradients w.r.t to parameters
loss.backward()
# update parameters
optimizer.step()
print('epoch {}, loss {}'.format(epoch, loss.item()))
# Calculate predictions on training data
with torch.no_grad(): # we don't need gradients in the testing phase
predicted = model(Variable(torch.from_numpy(x_train).cuda())).cpu().data.numpy()
General advice: For errors with dimension, it usually helps to print out dimensions at each step of the computation.
Most likely in this specific case, you have made mistake in reshaping the input with this x_train = x_train.reshape(-1, 1)
Your input is (N,1) but NN expects (N,2).
I'm learning how to use pytorch and I was able to get a grasp on the overall process of construction and execution of ML models. However, what I am not able to grasp is how to "format" or "reshape" the data before executing the model. I keep getting errors like:
RuntimeError: size mismatch, m1: [1 x 700], m2: [1 x 1] at c:\programdata\miniconda3\conda-bld\pytorch_1524543037166\work\aten\src\th\generic/THTensorMath.c:2033
Or,
Expected object of type Variable[torch.DoubleTensor] but found type Variable[torch.FloatTensor] for argument #1 ‘mat2’
So, I have a csv file named "train.csv" with attributes called 'x' and 'y' and there are 700 samples in it, I want to perform a simple linear regression on the data, and I parse data from it using pandas, how do I format or reshape the data such that it will execute smoothly? How does pytorch iterate through input data?
The recent code i executed is:
import torch
import torch.nn as nn
from torch.autograd import Variable
import pandas as pd
class Linear_Reg(nn.Module):
def __init__(self, inp_sz, out_sz):
super(Linear_Reg, self).__init__()
self.linear = nn.Linear(inp_sz, out_sz)
def forward(self, x):
out = self.linear(x)
return out
train = pd.read_csv('C:\\Users\\hgstr\\Jupyter_Files\\Data_Sets\\linear_regression\\train.csv')
test = pd.read_csv('C:\\Users\\hgstr\\Jupyter_Files\\Data_Sets\\linear_regression\\test.csv')
x_train = torch.Tensor(train['x'])
y_train = torch.Tensor(train['y'])
x_test = torch.Tensor(test['x'])
y_test = torch.Tensor(test['y'])
x_train = torch.Tensor(x_train)
x_train = x_train.view(1,-1)
#================================
input_sz = 1;
output_sz = 1
epochs = 60
learning_rate = 0.001
#================================
model = Linear_Reg(input_sz, output_sz)
crit = nn.MSELoss()
opt = torch.optim.SGD(model.parameters(), learning_rate)
for e in range(epochs):
opt.zero_grad()
out = model(x_train)
loss = crit(out, y_train)
loss.backward()
opt.step()
print('epoch {}, loss {}'.format(e,loss.data[0]))
And it gave out the following:
RuntimeError: size mismatch, m1: [1 x 700], m2: [1 x 1] at c:\programdata\miniconda3\conda-bld\pytorch_1524543037166\work\aten\src\th\generic/THTensorMath.c:2033
Solutions?
According to the error, I believe that your data is not correctly formatted. The tensor should be in the form [700, 2] (batch x data) and yours is [1, 700] (data x batch). This makes the model 'think' that you are adding only one entry as training with 700 features instead of 700 entries with only 1 feature.
Reshaping the x_train variable should make the code work. Just remove the line x_train = x_train.view(1,-1).
Regarding the second error, it can be that after reading the .csv into a variable its type is Double (due to pd.read_csv) while in pytorch by default Tensors are created as floats. I think that casting your input data before feeding it to the model should be enough: model(x_train.float()) or specifying it in the Tensor creation part x_train = torch.FloatTensor(train['x']). Note that you should cast all the Tensors that are not Floats.
edit: This piece of code works for me
import torch
import torch.nn as nn
import pandas as pd
class Linear_Reg(nn.Module):
def __init__(self, inp_sz, out_sz):
super(Linear_Reg, self).__init__()
self.linear = nn.Linear(inp_sz, out_sz)
def forward(self, x):
out = self.linear(x)
return out
train = pd.read_csv('yourpath')
test = pd.read_csv('yourpath')
x_train = torch.Tensor(train['x']).to(torch.float).view(700, 1)
y_train = torch.Tensor(train['y']).to(torch.float).view(700, 1)
x_test = torch.Tensor(test['x']).to(torch.float).view(300, 1)
y_test = torch.Tensor(test['y']).to(torch.float).view(300, 1)
# ================================
input_sz = 1;
output_sz = 1
epochs = 60
learning_rate = 0.001
# ================================
model = Linear_Reg(input_sz, output_sz)
crit = nn.MSELoss()
opt = torch.optim.SGD(model.parameters(), learning_rate)
for e in range(epochs):
opt.zero_grad()
out = model(x_train)
loss = crit(out, y_train)
loss.backward()
opt.step()
print('epoch {}, loss {}'.format(e, loss.data[0]))
I asked a previous question about the same code here where and how to put the filename in this tensorflow code?
Not sure if I should merge that into this question or leave it as is.
The following code is from Sirajology's git hub. I have not found a super straight forward tutorial on how to get one's own .csv file into a simple tensorflow neural network so my hope is this thread might provide that instruction for future searchers.
The code is as follows
import tensorflow.python.platform
import numpy as np
import tensorflow as tf
# Global variables.
NUM_LABELS = 2 # The number of labels.
BATCH_SIZE = 5 # The number of training examples to use per training step.
# Define the flags useable from the command line.
tf.app.flags.DEFINE_string('train', None,
'File containing the training data (labels & features).')
tf.app.flags.DEFINE_string('test', None,
'File containing the test data (labels & features).')
tf.app.flags.DEFINE_integer('num_epochs', 1,
'Number of examples to separate from the training '
'data for the validation set.')
tf.app.flags.DEFINE_boolean('verbose', False, 'Produce verbose output.')
FLAGS = tf.app.flags.FLAGS
# Extract numpy representations of the labels and features given rows consisting of:
# label, feat_0, feat_1, ..., feat_n
def extract_data(filename):
# Arrays to hold the labels and feature vectors.
labels = []
fvecs = []
# Iterate over the rows, splitting the label from the features. Convert labels
# to integers and features to floats.
for line in file(filename):
row = line.split(",")
labels.append(int(row[0]))
fvecs.append([float(x) for x in row[1:]])
# Convert the array of float arrays into a numpy float matrix.
fvecs_np = np.matrix(fvecs).astype(np.float32)
# Convert the array of int labels into a numpy array.
labels_np = np.array(labels).astype(dtype=np.uint8)
# Convert the int numpy array into a one-hot matrix.
labels_onehot = (np.arange(NUM_LABELS) == labels_np[:, None]).astype(np.float32)
# Return a pair of the feature matrix and the one-hot label matrix.
return fvecs_np,labels_onehot
def main(argv=None):
# Be verbose?
verbose = FLAGS.verbose
# Get the data.
train_data_filename = FLAGS.train
test_data_filename = FLAGS.test
# Extract it into numpy matrices.
train_data,train_labels = extract_data(train_data_filename)
test_data, test_labels = extract_data(test_data_filename)
# Get the shape of the training data.
train_size,num_features = train_data.shape
# Get the number of epochs for training.
num_epochs = FLAGS.num_epochs
# This is where training samples and labels are fed to the graph.
# These placeholder nodes will be fed a batch of training data at each
# training step using the {feed_dict} argument to the Run() call below.
x = tf.placeholder("float", shape=[None, num_features])
y_ = tf.placeholder("float", shape=[None, NUM_LABELS])
# For the test data, hold the entire dataset in one constant node.
test_data_node = tf.constant(test_data)
# Define and initialize the network.
# These are the weights that inform how much each feature contributes to
# the classification.
W = tf.Variable(tf.zeros([num_features,NUM_LABELS]))
b = tf.Variable(tf.zeros([NUM_LABELS]))
y = tf.nn.softmax(tf.matmul(x,W) + b)
# Optimization.
cross_entropy = -tf.reduce_sum(y_*tf.log(y))
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
# Evaluation.
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
# Create a local session to run this computation.
with tf.Session() as s:
# Run all the initializers to prepare the trainable parameters.
tf.initialize_all_variables().run()
if verbose:
print ('Initialized!')
print
print ('Training.')
# Iterate and train.
for step in xrange(num_epochs * train_size // BATCH_SIZE):
if verbose:
print (step,)
offset = (step * BATCH_SIZE) % train_size
batch_data = train_data[offset:(offset + BATCH_SIZE), :]
batch_labels = train_labels[offset:(offset + BATCH_SIZE)]
train_step.run(feed_dict={x: batch_data, y_: batch_labels})
if verbose and offset >= train_size-BATCH_SIZE:
print
# Give very detailed output.
if verbose:
print
print ('Weight matrix.')
print (s.run(W))
print
print ('Bias vector.')
print (s.run(b))
print
print ("Applying model to first test instance.")
first = test_data[:1]
print ("Point =", first)
print ("Wx+b = ", s.run(tf.matmul(first,W)+b))
print ("softmax(Wx+b) = ", s.run(tf.nn.softmax(tf.matmul(first,W)+b)))
print
print ("Accuracy:", accuracy.eval(feed_dict={x: test_data, y_: test_labels}))
if __name__ == '__main__':
tf.app.run()
When I run the code from terminal with the following command (windows10 cmd line) python YourScript.py --train FileName.csv --test TestName.csv --num_epochs 5 --verbose True I get these errors. Any help is greatly appreciated!
Error #1
File "softmax.py", line 133, in
tf.app.run()
tf.app.run()
Error #2
File "C:\app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
labels_onehot = (np.arange(NUM_LABELS) == labels_np[:, None]).astype(np.float32)
Error #3
File "softmax.py", line 57, in main
train_data,train_labels = extract_data(train_data_filename)
train_data,train_labels = extract_data(train_data_filename)
test_data, test_labels = extract_data(test_data_filename)
Error #4
File "softmax.py", line 31, in extract_data
for line in file(filename):
NameError: name 'file' is not defined
for line in file(filename):
row = line.split(",")
labels.append(int(row[7]))
fvecs.append([float(x) for x in row[1:6]])
It looks like the problem stems from this line, which uses a built-in function (file()) that is not available in Python 3.5:
for line in file(filename):
Replacing it with the following line should fix the error:
for line in open(filename):