The code below throws the error 'numpy.float64' object is not callable at last_mae = mae(val_scaled_price_client, cv_model). Nevertheless the mae function with the same parameters works just fine when used outside the loop.
loss_history = [1000.00]
for i in range(10000):
# train
iterations = 10
train_auto_encoder(train_latent_customers=train_latentvars,
train_product_customers=train_scaled_price_client,
auto_encoder=auto_model,
iters=iterations,
batch_size=128,
display_step=20)
cv_model = auto_model.predict([val_latentvars, val_scaled_price_client_corrupted])
last_mae = mae(val_scaled_price_client, cv_model)
loss_history.append(last_mae)
if loss_history[-1] < loss_history[-2]:
iterations += 10
else:
break
I declared a function mae in previous cells as follow
# define function to calculate MAE between true and reconstructed values
def mae(y_true, y_pred):
# get non-zero positions
cond = np.not_equal(y_true, 0)
# get number of non-zero elements
num_non_zero = np.sum(cond)
# initialize zer matrix
zero_matrix = np.zeros(shape=y_true.shape)
# replace
predictions_corrected = np.where(cond, y_pred, zero_matrix)
# get rmse
mae = np.sum(np.abs(y_true - predictions_corrected)) / num_non_zero
# return
return(mae)
The problem is in your mae function. In the last but one row you overwrite your function definition of mae with a number. If you call this function once everything is OK. As soon as you call it again (as in the loop), you try to call a number instead of a function, which is impossible.
Just change
mae = np.sum(np.abs(y_true - predictions_corrected)) / num_non_zero
# return
return(mae)
to
return np.sum(np.abs(y_true - predictions_corrected)) / num_non_zero
Python is not Basic or Fortran where you assign the result to the function name to return it :).
Related
I keep getting the error IndexError: tuple index out of range and i am not sure what is happening. My code was working just fine however when i restarted the jupyter notebook i started receiving this error.
this is my code:
X = df.Tweet
y = df.target
from sklearn import linear_model
import pyswarms as ps
# Create an instance of the classifier
classifier = linear_model.LogisticRegression()
# Define objective function
def f_per_particle(m, alpha):
total_features = X.shape[1]
# Get the subset of the features from the binary mask
if np.count_nonzero(m) == 0:
X_subset = X
else:
X_subset = X[:,m==1]
# Perform classification and store performance in P
classifier.fit(X_subset, y)
P = (classifier.predict(X_subset) == y).mean()
# Compute for the objective function
j = (alpha * (1.0 - P)
+ (1.0 - alpha) * (1 - (X_subset.shape[1] / total_features)))
return j
[some more code]
options = {'c1': 0.5, 'c2': 0.5, 'w':0.9, 'k': 30, 'p':2}
# Call instance of PSO
dimensions = X.shape[1] # dimensions should be the number of features
optimizer = ps.discrete.BinaryPSO(n_particles=30, dimensions=dimensions, options=options)
# Perform optimization
cost, pos = optimizer.optimize(f, iters=1000)
i received the following traceback:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-76-bea8cf064cd2> in <module>
2
3 # Call instance of PSO
----> 4 dimensions = X.shape[1] # dimensions should be the number of features
5 optimizer = ps.discrete.BinaryPSO(n_particles=30, dimensions=dimensions, options=options)
6
IndexError: tuple index out of range
It is not absolutely clear, but it seems to me that your df variable might be a Pandas dataframe, and your df.Tweet may be a Pandas series.
In that case, being a series, your X will have only one dimension (so, only the first element of the tuple X.shape, X.shape[0]), instead of two dimensions - reason for the index out of range exception in your code. The two dimensions case occurs only when the variable is a dataframe.
More information: https://www.google.com/amp/s/www.geeksforgeeks.org/python-pandas-series-shape/amp/
I'm going off of https://github.com/cortexlabs/cortex/blob/master/examples/pytorch/text-generator/predictor.py
But if I pass num_samples=5, I get:
generated = torch.cat((generated, next_token.unsqueeze(0)), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Got 5 and 1 in dimension 0
the code is:
def sample_sequence(
model,
length,
context,
num_samples=1,
temperature=1,
top_k=0,
top_p=0.9,
repetition_penalty=1.0,
device="cpu",
):
context = torch.tensor(context, dtype=torch.long, device=device)
context = context.unsqueeze(0).repeat(num_samples, 1)
print('context_size', context.shape)
generated = context
print('context', context)
with torch.no_grad():
for _ in trange(length):
inputs = {"input_ids": generated}
outputs = model(
**inputs
) # Note: we could also use 'past' with GPT-2/Transfo-XL/XLNet/CTRL (cached hidden-states)
next_token_logits = outputs[0][0, -1, :] / (temperature if temperature > 0 else 1.0)
# reptition penalty from CTRL (https://arxiv.org/abs/1909.05858)
for _ in set(generated.view(-1).tolist()):
next_token_logits[_] /= repetition_penalty
filtered_logits = top_k_top_p_filtering(next_token_logits, top_k=top_k, top_p=top_p)
if temperature == 0: # greedy sampling:
next_token = torch.argmax(filtered_logits).unsqueeze(0)
else:
next_token = torch.multinomial(F.softmax(filtered_logits, dim=-1), num_samples=1)
generated = torch.cat((generated, next_token.unsqueeze(0)), dim=1)
return generated
As far as I can see this code doesn't provide multiple samples, but you can adjust it with a some adjustments.
This line uses already multinomial but returns only 1:
next_token = torch.multinomial(F.softmax(filtered_logits, dim=-1), num_samples=1)
change it to:
next_token = torch.multinomial(F.softmax(filtered_logits, dim=-1), num_samples=num_samples)
Now you also need to change the result construction. This concatenates line the next_token with the sentence. You get now num_samples of next_tokens and you need unsqueeze all of them:
generated = torch.cat((generated, next_token.unsqueeze(0)), dim=1)
change it to:
generated = torch.cat((generated, next_token.unsqueeze(1)), dim=1)
The whole function should look like this now:
def sample_sequence(
model,
length,
context,
num_samples=1,
temperature=1,
top_k=0,
top_p=0.9,
repetition_penalty=1.0,
device="cpu",
):
context = torch.tensor(context, dtype=torch.long, device=device)
context = context.unsqueeze(0).repeat(num_samples, 1)
generated = context
with torch.no_grad():
for _ in trange(length):
inputs = {"input_ids": generated}
outputs = model(
**inputs
) # Note: we could also use 'past' with GPT-2/Transfo-XL/XLNet/CTRL (cached hidden-states)
next_token_logits = outputs[0][0, -1, :] / (temperature if temperature > 0 else 1.0)
# reptition penalty from CTRL (https://arxiv.org/abs/1909.05858)
for _ in set(generated.view(-1).tolist()):
next_token_logits[_] /= repetition_penalty
filtered_logits = top_k_top_p_filtering(next_token_logits, top_k=top_k, top_p=top_p)
if temperature == 0: # greedy sampling:
next_token = torch.argmax(filtered_logits).unsqueeze(0)
else:
next_token = torch.multinomial(F.softmax(filtered_logits, dim=-1), num_samples=num_samples)
generated = torch.cat((generated, next_token.unsqueeze(1)), dim=1)
return generated
Last but not least you have to change your tokenizer.decode call to tokenizer.batch_decode as the return value contains now multiple samples:
tokenizer.batch_decode(output.tolist(), clean_up_tokenization_spaces=True, skip_special_tokens=True)
Something you have to think of byt yourself, is what you want to do when there is no valide next_token. Currently you will receive an error message like:
RuntimeError: invalid multinomial distribution (with replacement=False, not enough non-negative category to sample)
Another thing you have to think of, is if their code is even correct. During the few test I have conducted, it felt like that the quality of created sentences decreased with an increasing number of num_samples (i.e. Maybe the quality is better when you use a simple loop to call sample_sequence multiple times?). I haven't worked with GPT2 yet and can't help you here.
I am trying to predict the future value with three inputs. Here I want to forecast the future value according to the three inputs in every one hour. Here g= temperature, p=humidity, c=wind and I want to predict temperature in next hour according to these inputs. That's why here I put n_out is 1, I wrote the code in def class. After that I tried to add that def class value as x,y value. Because I am going to write it as train and test value. But the error came as this. I am going to predict future value using LSTM . After this I don't know how to add this code as train and test into LSTM model. Can anyone help me to solve this problem?
Here I paste my code and csv file.
def change(train,X, n_out=1):
data = train.reshape((train.shape[0]))
x, y = list(), list()
in_start = 0
# step over the entire history one time step at a time
for _ in range(len(data)):
# define the end of the input sequence
in_end = in_start + X
out_end = in_end + n_out
# ensure we have enough data for this instance
if out_end < len(data):
x_input = data[in_start:in_end, 0]
x_input = x_input.reshape((len(x_input), 3))
x.append(x_input)
y.append(data[in_end:out_end, 0])
# move along one time step
in_start += 1
return array(x), array(y)
data= pd.DataFrame(data,columns=['g','p','c'])
data.columns = ['g', 'p', 'c',]
pd.options.display.float_format = '{:,.0f}'.format
data = data.dropna ()
cols=['g', 'p', 'c']
X=data[cols]
x,y = change(data)
The error came as
my csv file:
Aftre edditing the code it gave me this error:
In your definition of the function you have 3 parameters:
train, X and n_out=1
def change(train, X, n_out=1)
when you are calling your function you are providing just 1 argument(data)
x,y = change(data)
how n_out you define as 1, you need to provide x also, or define your function as:
def change(train, n_out=1)
NOTE:
you need to provide X when you are calling your function for example :
x,y = change(data, 1)
or define functrion like :
def change(train, X=1, n_out=1)
I am trying to write a function that properly calculates the entropy of a given dataset. However, I am getting very weird entropy values.
I am following the understanding that all entropy calculations must fall between 0 and 1, yet I am consistently getting values above 2.
Note: I must use log base 2 for this
Can someone explain why am I yielding incorrect entropy results?
The dataset I am testing is the ecoli dataset from the UCI Machine Learning Repository
import numpy
import math
#################### DATA HANDLING LIBRARY ####################
def csv_to_array(file):
# Open the file, and load it in delimiting on the ',' for a comma separated value file
data = open(file, 'r')
data = numpy.loadtxt(data, delimiter=',')
# Loop through the data in the array
for index in range(len(data)):
# Utilize a try catch to try and convert to float, if it can't convert to float, converts to 0
try:
data[index] = [float(x) for x in data[index]]
except Exception:
data[index] = 0
except ValueError:
data[index] = 0
# Return the now type-formatted data
return data
# Function that utilizes the numpy library to randomize the dataset.
def randomize_data(csv):
csv = numpy.random.shuffle(csv)
return csv
# Function to split the data into test, training set, and validation sets
def split_data(csv):
# Call the randomize data function
randomize_data(csv)
# Grab the number of rows and calculate where to split
num_rows = csv.shape[0]
validation_split = int(num_rows * 0.10)
training_split = int(num_rows * 0.72)
testing_split = int(num_rows * 0.18)
# Validation set as the first 10% of the data
validation_set = csv[:validation_split]
# Training set as the next 72
training_set = csv[validation_split:training_split + validation_split]
# Testing set as the last 18
testing_set = csv[training_split + validation_split:]
# Split the data into classes vs actual data
training_cols = training_set.shape[1]
testing_cols = testing_set.shape[1]
validation_cols = validation_set.shape[1]
training_classes = training_set[:, training_cols - 1]
testing_classes = testing_set[:, testing_cols - 1]
validation_classes = validation_set[:, validation_cols - 1]
# Take the sets and remove the last (classification) column
training_set = training_set[:-1]
testing_set = testing_set[:-1]
validation_set = validation_set[:-1]
# Return the datasets
return testing_set, testing_classes, training_set, training_classes, validation_set, validation_classes
#################### DATA HANDLING LIBRARY ####################
# This function returns the list of classes, and their associated weights (i.e. distributions)
# for a given dataset
def class_distribution(dataset):
# Ensure the dataset is a numpy array
dataset = numpy.asarray(dataset)
# Collect # of total rows and columns, using numpy
num_total_rows = dataset.shape[0]
num_columns = dataset.shape[1]
# Create a numpy array of just the classes
classes = dataset[:, num_columns - 1]
# Use numpy.unique to remove duplicates
classes = numpy.unique(classes)
# Create an empty array for the class weights
class_weights = []
# Loop through the classes one by one
for aclass in classes:
# Create storage variables
total = 0
weight = 0
# Now loop through the dataset
for row in dataset:
# If the class of the dataset is equal to the current class you are evaluating, increase the total
if numpy.array_equal(aclass, row[-1]):
total = total + 1
# If not, continue
else:
continue
# Divide the # of occurences by total rows
weight = float((total / num_total_rows))
# Add that weight to the list of class weights
class_weights.append(weight)
# Turn the weights into a numpy array
class_weights = numpy.asarray(class_weights)
# Return the array
return classes, class_weights
# This function returns the entropy for a given dataset
# Can be used across an entire csv, or just for a column of data (feature)
def get_entropy(dataset):
# Set initial entropy
entropy = 0.0
# Determine the classes and their frequencies (weights) of the dataset
classes, class_freq = class_distribution(dataset)
# Utilize numpy's quicksort to test the most occurring class first
numpy.sort(class_freq)
# Determine the max entropy for the dataset
max_entropy = math.log(len(classes), 2)
print("MAX ENTROPY FOR THIS DATASET: ", max_entropy)
# Loop through the frequencies and use given formula to calculate entropy
# For...Each simulates the sequence operator
for freq in class_freq:
entropy += float(-freq * math.log(freq, 2))
# Return the entropy value
return entropy
def main():
ecol = csv_to_array('ecoli.csv')
testing_set, testing_classes, training_set, training_classes, validation_set, validation_classes = split_data(ecol)
entropy = get_entropy(ecol)
print(entropy)
main()
The following function was used to calculate Entropy:
# Function to return Shannon's Entropy
def entropy(attributes, dataset, targetAttr):
freq = {}
entropy = 0.0
index = 0
for item in attributes:
if (targetAttr == item):
break
else:
index = index + 1
index = index - 1
for item in dataset:
if ((item[index]) in freq):
# Increase the index
freq[item[index]] += 1.0
else:
# Initialize it by setting it to 0
freq[item[index]] = 1.0
for freq in freq.values():
entropy = entropy + (-freq / len(dataset)) * math.log(freq / len(dataset), 2)
return entropy
As #MattTimmermans had indicated, entropy's value is actually contingent on the number of classes. For strictly 2 classes, it is contained in the 0 to 1 (inclusive) range. However, for more than 2 classes (which is what was being tested), entropy is calculated with a different formula (converted to Pythonic code above). This post here explains those mathematics and calculations a bit more in detail.
Is there a way to directly retrieve the minimized error after scipy.minimize has converged or must that be directly coded into the cost function?
I can only retrieve the converged to coefficients it seems.
def errorFunction(params,series,loss_function,slen = 12):
alpha, beta, gamma = params
breakUps = int(len(series) / slen)
end = breakUps * slen
test = series[end:]
errors = []
for i in range(2,breakUps+1):
model = HoltWinters(series=series[:i * 12], slen=slen,
alpha=alpha, beta=beta, gamma=gamma, n_preds=len(test))
model.triple_exponential_smoothing()
predictions = model.result[-len(test):]
actual = test
error = loss_function(predictions, actual)
errors.append(error)
return np.mean(np.array(errors))
opt = scipy.optimize.minimize(errorFunction, x0=x,
args=(train, mean_squared_log_error),
method="L-BFGS-B", bounds = ((0, 1), (0, 1), (0, 1))
)
#gets the converged values
optimal values = opt.x
#I would like to know what the error with errorFunction is when using opt.x values, without having to manually run the script again
#Is the minimum error stored somewhere in the returned object opt
From what I understand from the documentation of the function scipy.optimize.minimize, the result is returned as a OptimizeResult object.
From the documentation of this class (here) it has an attribute fun that is the "values of objective function".
So if you do opt.fun, you should obtain the result you are looking for. (There are more values that you can retrieve like the Jacobian opt.jac, the Hessian opt.hess, ... as described in the documentation)