I'm trying to implement a slightly different version of SGD with pytorch and test it on some datasets. I need to write a custom optimizer on which to train my model, however I cannot find any guide which explains how to do so. Is anyone able to tell me how I can proceed or suggest me any useful reference?
Thanks in advance.
Related
I would like to implement a cost-sensitive loss function in PyTorch. My two-class training dataset is heavily imbalanced, where 75% of the data are label '0' and only 25% of the data are label '1'.
I am new to PyTorch but my supervisor is adamant that I use it (they have more experience with it).
I found some implementations in Keras, but I am not that strong in coding to be able to port it over to PyTorch.
I have read around to find some resources to create a cost-sensitive loss function.
This paper uses something which I think might work (https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9417097), but I do not understand how the code is implemented despite having access to it here (https://github.com/emadeldeen24/AttnSleep/blob/f993511426900f9fca20594a738bf8bee1116381/utils/util.py).
This website describes the math very detailedly but I do not understand it: https://medium.com/rv-data/how-to-do-cost-sensitive-learning-61848bf4f5e7
Here is an implementation in Keras which I have trouble with converting to PyTorch: https://towardsdatascience.com/fraud-detection-with-cost-sensitive-machine-learning-24b8760d35d9
I also found this implementation in PyTorch, but have trouble with understanding it: https://discuss.pytorch.org/t/dealing-with-imbalanced-datasets-in-pytorch/22596/21
Could you please help me to understand the last link's implementation of the cost-sensitive loss function?
Thank you.
In my previous project, I need to frame an image classification task as a regression problem. I implement the regression model using Tensorflow, with standard Sequential model with a 1 node Dense layer with no activation function as the last layer. In order to measure the performance, I need to use standard classification metrics, such as accuracy and cohen kappa.
However, I can't directly use those metrics because my model is a regression model, so I need to clip and round the output before feeding them to the metrics. I use a workaround by defining my own metric, however that workaround is not practical. Therefore, I'm thinking about contributing to Tensorflow by implementing a custom transformation_function to transform y_pred by a Tensor lambda function before storing them in the __update_state method. After reading the source code, I get doubts regarding this idea. So, I'm asking out to you, fellow Tensorflow user/contributors, what is the best practice of transforming y_pred before feeding it to a metric? Is this functionality already implemented in the newest version?
Thank you!
I started experimenting with gans and on the internet, there are a lot of options the thing I worry about now is which one should I use Keras fit or Keras train_on_batchs which one is the proper way to train the model thanks
According to tensorflow's documentation, the combination of tf.data & model.train_on_batch() is the fastest way to train a model.
You can check more about tf.data in the link below:
https://www.tensorflow.org/guide/data
I have already implemented GANs with keras. My implementations might help you get an idea how to train those models:
https://github.com/kochlisGit/Keras-GAN
So, I've googled prior to asking this, obviously, however, there doesn't seem to be much mention on these modes directly. Tensorflow documentation mentions "test" mode in passing which, upon further reading, didn't make very much sense to me.
From what I've gathered, my best shot at this is that to reduce ram, when your model is in prediction mode, you just use a pretrained model to make some predictions based on your input?
If someone could help with this and help me understand, I would be extremely grateful.
Training refers to the part where your neural network learns. By learning I mean how your model changes it's weights to improve it's performance on a task given a dataset. This is achieved using the backpropogation algorithm.
Predicting, on the other hand, does not involve any learning. It is only to see how well your model performs after it has been trained. There are no changes made to the model when it is in prediction mode.
I am desperately trying to use cohen kappa's metric as either a loss function or an evaluation metric in my keras neural network. I have tried many different implementations of it online and none of them seem to be maintained. In particular due to the fact that tf.contrib no longer exists in tensorflow 2.0. Any help in pointing in me in the right direction of a working implementation will be much appreciated!
When using the tf add-on class found here https://www.tensorflow.org/addons/api_docs/python/tfa/metrics/cohens_kappa.
I keep getting the following error and have no idea how I would go about debugging this.
ValueError: Number of samples in y_true and y_pred are different