I am new to Neural Networks and I am trying to modify this code RNN-Classifier and instead of using the GRU_step, I would rather use an LSTM.
I added one extra parameter c_prev
def lstm_step(x, h_prev, c_prev, W_xz, W_hz, W_xm, W_hm):
and after applying all the LSTM equations I am returning them both (h and c)
My hidden vector looks like that:
hidden_vector, _ = theano.scan(
lstm_step,
sequences=input_vectors,
outputs_info=initial_hidden_vector,
non_sequences=[W_xz, W_hz, W_xm, W_hm]
)
hidden_vector = hidden_vector[-1]
I get an exception like this and don't understand why it does not see the c_prev as an existant parameter (or how/where can I feed it with some values, so that it's not empty?)
python rnnclassifier.py data/sentiment.train.txt data/sentiment.test.txt
Traceback (most recent call last):
File "rnnclassifier.py", line 167, in <module>
rnn_classifier = RnnClassifier(word2id_len, n_classes)
File "rnnclassifier.py", line 110, in __init__
non_sequences=[W_xz, W_hz, W_xm, W_hm]
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan.py",
line 773, in scan condition, outputs, updates =
scan_utils.get_updates_and_outputs(fn(*args))
TypeError: lstm_step() takes exactly 7 arguments (6 given)
I am new to this topic and would appreciate any help or advice! Thank you.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
I was writing a program to construct a GAN in which generator class takes in a tf module with all initialization required, but when this class is initialized and finally called, i am facing an error of extra arguments being passed (complete error i have posted below)
class Generator(tf.Module):
def __init__(self, noise_size, condition_size, generator_latent_size, cell_type, mean=0, std=1):
super().__init__()
self.noise_size = noise_size
self.condition_size = condition_size
self.generator_latent_size = generator_latent_size
self.mean = mean
self.std = std
if cell_type == "lstm":
self.cond_to_latent = tf.keras.layers.LSTM(generator_latent_size)
else:
self.cond_to_latent = tf.keras.layers.GRU(generator_latent_size)
self.model = tf.keras.Sequential(
tf.keras.layers.InputLayer(input_shape=((generator_latent_size + self.noise_size),)),
tf.keras.layers.Dense(generator_latent_size + self.noise_size),
tf.keras.layers.ReLU(),
tf.keras.layers.Dense(1)
)
def forward(self, noise, condition):
condition = (condition - self.mean) / self.std
condition = condition.view(-1, self.condition_size, 1)
condition = condition.transpose(0, 1)
condition_latent, _ = self.cond_to_latent(condition)
condition_latent = condition_latent[-1]
g_input = tf.concat((condition_latent, noise), dim=1)
output = self.model(g_input)
output = output * self.std + self.mean
return output
def get_noise_size(self):
return self.noise_size
when is generator object is called, i am getting an error in internal method wrapper stating
" Traceback (most recent call last):
File "forgan.py", line 185, in <module>
forgan = ForGAN(opt)
File "forgan.py", line 35, in __init__
std=opt.data_std)
File "C:\Users\mura_ab\PycharmProjects\ForGAN\components.py", line 23, in __init__
tf.keras.layers.Dense(1)
File "C:\Users\mura_ab\Anaconda3\envs\Plygrnd\lib\site-packages\tensorflow_core\python\training\tracking\base.py", line 457, in _method_wrapper
result = method(self, *args, **kwargs)
TypeError: __init__() takes from 1 to 3 positional arguments but 5 were given"
this is the Generator object being initilized:
class ForGAN:
def __init__(self, opt):
self.opt = opt
self.device = tf.device("cuda:0") if tf.test.is_gpu_available() else tf.device("cpu")
print("***** Hyper-parameters *****")
for k, v in vars(opt).items():
print("{}:\t{}".format(k, v))
print("************************")
# Making required directories for logging, plots and models' checkpoints
os.makedirs("./{}/".format(self.opt.dataset), exist_ok=True)
# Defining GAN components
self.generator = Generator(noise_size=opt.noise_size,
condition_size=opt.condition_size,
generator_latent_size=opt.generator_latent_size,
cell_type=opt.cell_type,
mean=opt.data_mean,
std=opt.data_std)
and at last called by
forgan = ForGAN(opt)
Can someone please let me know if there is any workaround method to tackle this problem.
In general, you should pay close attention to what the error message says. Instead of answering to solve your error straight away, I'm gonna show you how I found it in hopes you'll do it on your own with the next error you get.
Reading all the info of the error one by one:
We know the error appears in the creation of your forgan.
File "forgan.py", line 185, in <module>
forgan = ForGAN(opt)
More concretely, we know it has something to do with the initialization of the Generator,
line 35, in __init__
std=opt.data_std)
It seems to be related with an initialization in line 23, which could come from the creation of a Dense layer (or something below the Dense layer), or with the initialization of the Sequential model (after all, line 23 is part of the initialization of the sequential model happening in line 19):
File "C:\Users\mura_ab\PycharmProjects\ForGAN\components.py", line 23, in __init__
tf.keras.layers.Dense(1)
The error goes out of your code.
It is clear by the text in the error that you used 5 arguments to initialize an object somewhere. Seeing step 3, you didn't initialize the Dense layer using 5 arguments. However, you seem to be initializing the Sequential model with 4 arguments... and if you add the hidden self argument always present in Python object initializations, it adds up to 5 arguments! Maybe you're doing something wrong when initializing the Sequential model.
To corroborate it, you should take a look at the official API or some other official guide. Googling it, you can easily get to this API and this guide. In the API you can see that the initialization requires between no extra arguments and 2 extra arguments(1 to 3 including the self). That's exactly what your error says! In the guide you can see examples on how to use it properly. It looks like the layers should be passed inside a container (list or tuple).
Therefore, this should solve the issue (notice the extra square brackets to turn all 5 arguments into a single list):
self.model = tf.keras.Sequential([
tf.keras.layers.InputLayer(input_shape=((generator_latent_size + self.noise_size),)),
tf.keras.layers.Dense(generator_latent_size + self.noise_size),
tf.keras.layers.ReLU(),
tf.keras.layers.Dense(1)
])
I am working on migrating some working Pytorch code I found online (which is a 2D image classification example using the MNIST data; apologies that I lost track of the original source and am unable to find it) to what I need, which is converting a 1D collection of values into a numerical score. I created my own Dataset class. When I call model(), I get an error: RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #2 'mat1' in call to _th_addmm. My first level of confusion is that I can't find any reference to Python even having a Double datatype. And my second is why I get the error--when I put in debug code to show the datatype of mat1 and its elements, I am told that it is a Tensor which claims to be float64. I also wonder why it is expecting a scalar for mat1, which the documentation describes as a matrix/tensor.
The full error dump is
Traceback (most recent call last):
File "mlalan.py", line 174, in <module>
outputs = model(images)
File "/usr/home/adf/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "mlalan.py", line 80, in forward
x = activate(self.fc1(x))
File "/usr/home/adf/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "/usr/home/adf/.local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/usr/home/adf/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 1610, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #2 'mat1' in call to _th_addmm
Some of the key code from my Dataset class is
class RandomDataset(Dataset):
def __init__(self, csv_file, transform=None):
self.data_frame = pd.read_csv(csv_file, dtype=float)
def __getitem__(self, idx):
raw = self.data_frame.values[idx]
sample = raw[0:6], raw[6:8]
return sample
The full source code is at http://8wheels.org/mlalan.py.
By default, in Python, float means float32. However, in Pandas and Numpy, float means float64. I was able to resolve the problem by adding a call to astype as below. The "32" is required for it to work.
raw = self.data_frame.values[idx].astype(np.float32)
Thanks!
Now I can move on to the next crash :-)
I decided to mess with Uber Ludwig again. I wanted to make a simple demo using the python API that learns to add 1 to the input number. I have successfully produced a model, but the issue arises when predicting. I am running on the newest release from github on PopOS 19.10 on CPU TensorFlow.
Thank you for any help.
Edit: I have reproduced the issue on windows as well.
The error is as follows
Traceback (most recent call last):
File "predict.py", line 3, in <module>
x = model.predict({"numberIn":[1]}, return_type='dict')
File "/home/user/.local/lib/python3.7/site-packages/ludwig/api.py", line 914, in predict
gpu_fraction=gpu_fraction,
File "/home/user/.local/lib/python3.7/site-packages/ludwig/api.py", line 772, in _predict
self.model_definition['preprocessing']
File "/home/user/.local/lib/python3.7/site-packages/ludwig/data/preprocessing.py", line 159, in build_data
preprocessing_parameters
File "/home/user/.local/lib/python3.7/site-packages/ludwig/data/preprocessing.py", line 180, in handle_missing_values
dataset_df[feature['name']] = dataset_df[feature['name']].fillna(
AttributeError: 'list' object has no attribute 'fillna'
Here is my prediction script
from ludwig.api import LudwigModel
model = LudwigModel.load("/home/user/Documents/ludwig-test/plus1/results/api_experiment_run_0/model")
x = model.predict({"numberIn":[1]}, return_type='dict')
#x = model.predict({"numberIn":[1]}, return_type=<class 'dict'>) I tried this with no success
print(x)
Here is the contents of my training script.
mydata = {"numberIn":[], "value":[]}
for x in range(10000):
mydata["numberIn"].append(x)
mydata["value"].append(x + 1)
from ludwig.api import LudwigModel
print("Imported Ludwig")
modelobject = LudwigModel(model_definition_file="modeldef.yaml")
stats = modelobject.train(data_dict=mydata)
modelobject.close()
modeldef.yaml
input_features:
-
name: numberIn
type: numerical
output_features:
-
name: value
type: numerical
Solution: Input argument of predict function is not positional and data_dict needs to be specified in this case.
x = modelobject.predict(data_dict=mydictionary)
I got TypeError: expected torch.LongTensor (got torch.cuda.FloatTensor).
How do I convert torch.cuda.FloatTensor to torch.LongTensor?
Traceback (most recent call last):
File "train_v2.py", line 110, in <module>
main()
File "train_v2.py", line 81, in main
model.update(batch)
File "/home/Desktop/squad_vteam/src/model.py", line 131, in update
loss_adv = self.adversarial_loss(batch, loss, self.network.lexicon_encoder.embedding.weight, y)
File "/home/Desktop/squad_vteam/src/model.py", line 94, in adversarial_loss
adv_embedding = torch.LongTensor(adv_embedding)
TypeError: expected torch.LongTensor (got torch.cuda.FloatTensor)
You have a float tensor f and want to convert it to long, you do long_tensor = f.long()
You have cuda tensor i.e data is on gpu and want to move it to cpu you can do cuda_tensor.cpu().
So to convert a torch.cuda.Float tensor A to torch.long do A.long().cpu()
Best practice for Pytorch 0.4.0 is to write device agnostic code: That is, instead of using .cuda() or .cpu() you can simply use .to(torch.device("cpu"))
A = A.to(dtype=torch.long, device=torch.device("cpu"))
Note that .to() is not an "in-place" operation (see, e.g., this answer), thus you need to assign A.to(...) back into A.
If you have a tensor t.
t = t.cpu()
would be the old way.
t = t.to("cpu")
would be the new API.
I have problem. here's my code.
http://colorscripter.com/s/9vc2ryj
And I mistaked. evaluate_classifier(bigram_word_feats) is what I want.
I'm trying to text mining by SVM.
The feature vectors are bigram model.
But I got a problem:
Traceback (most recent call last):
File "C:/Users/LG/Desktop/untitled1/TEST.py", line 184, in <module>
evaluate_classifier(bigram_word_feats)
File "C:/Users/LG/Desktop/untitled1/TEST.py", line 90, in evaluate_classifier
classifier.train(trainfeats)
File "C:\Users\LG\Anaconda3\lib\site-packages\nltk\classify\scikitlearn.py", line 115, in train
X = self._vectorizer.fit_transform(X)
File "C:\Users\LG\Anaconda3\lib\site-packages\sklearn\feature_extraction\dict_vectorizer.py", line 226, in fit_transform
return self._transform(X, fitting=True)
File "C:\Users\LG\Anaconda3\lib\site-packages\sklearn\feature_extraction\dict_vectorizer.py", line 190, in _transform
feature_names.sort()
TypeError: unorderable types: tuple() < str()
Why this happen and how can I solve?
and what's the process of nltk classifier?
give it to my feature word and period? Then it just generate svm model?
Oh and I'm using python 3. Do I need to use python 2?
New answer:
I think the problem is that nltk expects a dict indexed by strings instead of tuples. Can you try to replace the return statement from:
return dict([(ngram, True) for ngram in itertools.chain(words, bigrams)])
to the following:
return dict([('|'.join (ngram), True) for ngram in itertools.chain(words, bigrams)])
Old answer:
`train` methods of Scikit-learn predictors expect two inputs: features and targets. Something like the following (not tested):
negfeats = [featx(f) for f in word_split(negdata)]posfeats = [featx(f) for f in word_split(posdata)]...trainlabels = [-1,] * negcutoff + [+1,] * poscutoffclassifier.train(trainfeats, trainlabels)
In defining trainlabels, I followed your style of using arithmetic operators on lists but I wouldn't do it in my code as it makes it less readable.