Hello Guys, PLEASE I NEED YOUR ADVICE ON THIS.
I have a problem with the IBM Visual Recognition Service, I am creating a weed detection Model using IBM Visual Recognition Service. I have carefully labelled and trained my images across the different classes.
The Model Performs well when I test for unseen images that belong to this two classes(CORN AND CHENOPODIUM ALBUM) as indicated below:
But My Major Problem is when I try to test for plants outside my labelled images, The Model Identifies that as part of of my labelled images with a very high accuracy. (Plantain and Cassava)
What might be the reason for this, and how can I correct this issue..??
So you have trained the model with images of CORN AND CHENOPODIUM ALBUM, and then are testing it with images of Plantain and Cassava, is that right?
The general "best practice" for training any machine learning model, whether a classifier or object detector, is to have your training data match the test data as nearly as possible. This is summarized as "You get what you train for."
It is not always possible, but to the extent that you have knowledge of what the test data will be like then you want to sample your training data from a similar distribution.
Think of yourself like a teacher preparing a student for a test. If you teach them Spanish and then the test is in Italian, the results will not be good.
In this case, to detect Plantain and Cassava, you will need to add Plantain and Cassava examples to your training set.
Related
i'm totally new in NLP and Bert Model.
What im trying to do right now is Sentiment Analysis on Twitter Trending Hashtag ("neg", "neu", "pos") by using DistilBert Model, but the accurazcy was about 50% ( I tried w Label data taken from Kaggle).
So here is my idea:
(1) First, I will Fine-tunning Distilbertmodel (Model 1) with IMDB dataset,
(2) After that since i've got some data took from Twitter post, i will sentiment analysis them my Model 1 and get Result 2.
(3) Then I will refine-tunning Model 1 with the Result 2 and expecting to have Model (3).
Im not really sure this process has any meaning to make the model more accuracy or not.
Thanks for reading my post.
I'm a little skeptical about your first step. Since the IMDB database is different from your target database, I do not think it will positively affect the outcome of your work. Thus, I would suggest fine-tuning it on a dataset like a tweeter or other social media hashtags; however, if you are only focusing on hashtags and do not care about the text, that might work! My little experience with fine-tuning transformers like BART and BERT shows that the dataset that you are working on should be very similar to your actual data. But in general, you can fine-tune a model with different datasets, and if the datasets are structured for one goal, it can improve the model's accuracy.
If you want to fine-tune a sentiment classification head of BERT for classifying tweets, then I'd recommend a different strategy:
IMDB dataset is a different kind of sentiment - the ratings do not really correspond with short post sentiment, unless you want to focus on tweets regarding movies.
using classifier output as input for further training of that classifier is not really a good approach, because, if the classifier made many mistakes while classifying, these will be reflected in the training, and so the errors will deapen. This is basically creating endogenous labels, which will not really improve your real-world classification.
You should consider other ways of obtaining labelled training data. There are a few good examples for twitter:
Twitter datasets on Kaggle - there are plenty of datasets available containing millions of various tweets. Some of those even contain sentiment labels (usually inferred from emoticons, as these were proven to be more accurate than words in predicting sentiment - for explanation see e.g. Frasincar 2013). So that's probably where you should look.
Stocktwits (if youre interested in financial sentiments)- contain posts that authors can label for sentiments, thus are a perfect way of mining labelled data, if stocks/cryptos is what you're looking for.
Another thing is picking a model that's better for your language, I'd recommend this one. It has been pretrained on 80M tweets, so should provide strong improvements. I believe it even contains a sentiment classification head that you can use.
Roberta Twitter Base
Check out the website for that and guidance for loading the model in your code - it's very easy, just use the following code (this is for sentiment classification):
MODEL = "cardiffnlp/twitter-roberta-base-sentiment"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL)
Another benefit of this model is that it has been pretrained from scratch with a vocabulary that contains emojis, meaning it has a deep understanding of them, their typical contexts and co-occurences. This can greatly benefit a social media classification, as many researchers would agree that emojis/emoticons are better predictors of sentiment than normal words.
I created a corpus of 30,000 headlines. I want to predict the sentiments of these headlines using advanced supervised machine learning (deep learning) methods such as RNN, LSTM, or DNN.
My question is: Is it possible to train and test a deep learning model with any labeled datasets such as IMDB movie review, amazon review, or yelp review.
For example, suppose we train and test the IMDB movie reviews dataset with RNN which gives us a 92% f1 score.
Then, can I input my unlabeled dataset (30,000 headlines) and predict their sentiments with this trained and tested model?
The reason for asking this question is that I found many blogs and tutorial with code that uses deep learning methods for sentiment analysis. They use the label dataset and train and test the model and short ....accuracy or f1 score. Nobody goes further and input the unlabeled data and "predict" the sentiment with their model. That is why I am wondering whether it is possible or not.
Thanks for your suggestions and advice.
Good question,
Yes, nothing stops you from testing it against your own dataset. However, this is not how this is supposed to be done:
Consider for example You train a model on Amazon reviews and then you are testing it on Movie reviews. So what's different? The distributions of data are different, this may have a lot of side effects. The choice of words, sentences, metaphors would be different in both the sets of reviews.
For eg. Consider this review in the Life Sciences domain:
"This drug partially cures cancer"
This is most likely to output negative sentiment if you had trained on Amazon Review data because cancer is a negative word in other domains. So, there is a need to train different sentiment classifiers for different domains.
Summary:
Trying to Use Data from the same data source wherever possible.
Train and Predict on the same domain data.
I'm new to Tensorflow and AI, so I'm having trouble researching my question. Either that, or my question hasn't been answered.
I'm trying to make a text classifier to put websites into categories based on their keywords. I have at minimum 5,000 sites and maximum 37,000 sites to train with.
What I'm trying to accomplish is: after the model is trained, I want it to continue to train as it makes predictions about the category a website belongs in.
The keywords that the model is trained on is chosen by clients, so it can always be different than the rest of the websites in its category.
How can I make Tensorflow retrain it's model based on corrections made by me if it's prediction is inaccurate? Basically, to be training for ever.
The key phrase you lack is fine-tuning. This is when you take a model that has finished its customary training (whatever that may be), and needs more work for the application you have in mind. You then give it additional training with new input; when that training has completed (training accuracy plateaus and is close to test accuracy), you then deploy the enhanced model for your purposes.
This is often used in commercial applications -- for instance, when a large predictive model is updated to include the most recent week of customer activity. Another common use is to find a model in a zoo that is trained for something related to the application you want -- perhaps cats v dogs -- and use its recognition of facial features to shorten training for a model to identify two classes of cartoon characters -- perhaps Pokemon v Tiny Toons.
In this latter case, your fine-tuning will almost entirely eliminate what was learned by the last few layers of the model. What you gain is the early-layer abilities to find edges, regions, and features through eyes-nose-mouth combinations. This saves at least 30% of the overall training time.
I have a use case where I have about 300 images out of 300 different items. I need machine learning to detect an item about once a minute.
I've been using Keras with Sequential to detect images but I'm wondering what I should take into consideration when I have 300 labels and only one image per label for learning.
So in short:
1) Can you do machine learning image detection with one learning image per label?
2) Are there any special things I take into consideration?
If this were a special case -- say, one class in 100 was represented by a single training image -- then you might get away with it. However, a unique image per class is asking for trouble.
A neural network learns by iterative correction, figuring out what features and combinations are important, and which are not, in discriminating the classes from one another. Training starts by a chaotic process that has some similarities to research: look at the available data, form hypotheses, and test then against the real world.
In a NN, the "hypotheses" are the various kernels it develops. Each kernel is a pattern to recognize something important to the discrimination process. If you lack enough examples for the model to generalize and discriminate for each class, then you run the risk (actually, you have the likelihood) of the model making a conclusion that is valid for the one input image, but not others in the same class.
For instance, one acquaintance of mine did the canonical cat-or-dog model, using his own photos, showing the pets of his own household and those of a couple of friends. The model trained well, identified cats and dogs with 100% accuracy on the test data, and he brought it into work ...
... where it failed, having an accuracy of about 65% (random guessing is 50%). He did some analysis and found the problem: his friends have indoor cats, but their preferred dog photos were out of doors. Very simply, the model had learned to identify not cats vs dogs, but rather couches and kitchen cabinets vs outdoor foliage. One of the main filters was of large, textured, green areas. Yes, a dog is a large, textured, green being. :-)
The only way your one-shot training would work is if each of your training images was specifically designed to include exactly those features that differentiate this class from the other 299, and no other visual information. Unfortunately, to identify what features those might be, and to provide canonical training photos, you'd have to know in advance what patterns the model needed to pick.
This entirely defeats the use case of deep learning and model training.
If you were to only train on that image once, it probably wouldn't be able to detect it yet. If you train it more, it will probably over fit and only recognize that one image. If that is what you are trying to do then you should make an algorithm to search the screen for that image (it will be more efficient).
1) You'll probably have problems with the generalization of your models because the lack of training set. In other words, your model will not "learn" about that class.
2) It's good to have a better training set in order to create a better model.
Currently I am working for a project to classify a given set of test images into one of the 5 predefined categories. I implemented Logistic Regression with a feature vector of 240 features for each image and trained it using 100 images/ category. The learning accuracy I achieved was ~98% for each category, whereas when tested on validation set consisting of 500 images (100 images/category), only ~57% images were rightly classified.
Please suggest me few libraries/tools which I can use (preferably based on Neural Network) in order to attain higher accuracy.
I tried using a Java based tool, Neurophy (neuroph.sourceforge.net) on windows but, it didn't run as expected.
Edit: The feature vector were already provided for the project. I am also looking for a better feature extraction tool for Images.
You can get help from this paper Image Classification
In My opinion, SVM is relatively better than logistic regression when it comes to multi-class response problems. We use it in e commerce classification of product where there are 1000s of response level and thousands of features.
Based on your tags I assume you would like a python package, scikit-learn has good classification routines: scikit-learn.org.
I have had good success using the WEKA tools, you need to isolate the feature set that you are interested in and then apply a classifier from this library. The examples are very clear. http://weka.wikispaces.com