I am working on an object detector using Tensorflow Object Detection API whereby I downloaded a model from the model zoo repository, specifically ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8 to train my custom dataset on which contains 2500 images split into 70:30 training and testing ratio. I have edited the pipeline.config file accordingly following a tutorial, where I added the label_maps paths and other requirements. I have trained this model for 50,000 steps and monitored the training process on tensorboard which displayed good training progress results. Now the issue occurred when I went to evaluate the model. For some reason, the evaluation results are the following:
What could possibly be the issue, since the loss graphs seem to be good and even the learning rate graphs? Thanks
Related
I am trying to get inference from a pre-trained machine learning object detection model(detr) on to jrdb dataset(https://jrdb.stanford.edu/). I am using the demo code provided by the team that authored the paper. Here is the link to the code: detr_demo
I have changed the model to resnet-101 and the weights file to the best they have reported on the GitHub repo(https://github.com/facebookresearch/detr) for the code. yet, the Average precision score I am getting on the jrdb test dataset is 15.8 AP. while, the best result on the same code using the pre-trained model and weights for another team is 48 AP, as reported on the jrdb leaderboard for 2d detection(https://jrdb.stanford.edu/leaderboards/results/412). What I wish to know is, are there ways to improve inference, without retraining? I have searched on google and most links show to better train the model.
I have trained multiple deep learning object detection models on my own custom dataset of beef cattle. I am planning to compare the performance of the models between them and YOLO Version 3. To do so, I am using legacy/eval.py to get the mAP of the trained model. I have tried to get the mAP for Faster-RCNN ResNet-50 at this moment by running the following codes -
python legacy/eval.py --logtostderr --pipeline_config_path=training/faster_rcnn_resnet50.config checkpoint_dir=training/ --eval_dir=eval/
After running the evaluation on the test dataset, it is showing mAP as >0.99 which is way too high. The same model gained ~0.3 on the COCO dataset. Is it normal to have such a high mAP while evaluating on training dataset?
My system is as follows-
python 3.6
tensorflow 1.5 (gpu)
number of training image ~2800
number of test image ~400
evaluated on test image
I'm currently using a custom version of YOLO v2 from pjreddie.com written with Tensorflow and Keras. I've successfully got the model to start and finish training over 100 epochs with 10000 training images and 2400 testing images which I randomly generated along with the associated JSON files all on some Titan X gpus with CUDA. I only wish to detect two classes. However, after leaving the training going, the loss function decreases but the test accuracy hovers at below 3%. All the images appear to be getting converted to black and white. The model seems to perform reasonably on one of the classes when using the training data, so the model appears overfitted. What can I do to my code to get the model to become accurate?
Okay, so it turned out that YOLOv2 was performing very well on unseen data except that the unseen data has to be the same size of images as the ones it's trained on. Don't feed Yolo with 800x800 images if it's been trained on 400x400and 300x400 images. Also the Keras accuracy measure is meaningless for detection. It might say 2% accuracy and actually be detecting all objects. Passing unseen data of the same size solved the problem.
I already trained a custom person detector which doesn't work well on detecting perons on aerial footage which is because my dataset lacks on aerial images of a person. Can I continue the training using the latest checkpoint and add another dataset (different tfrecord file) for my model or do I have to restart the training using the updated dataset?
I retrained the Inception model to detect persons only since there no other way to remove the other 89 objects from the pretrained model.
You can definitely start from a checkpoint using another dataset. However, it might not be a good idea to train only on a subset of your data due to the tendency of neural nets to forget what they'e already learned (a problem known as catastrophic forgetting). It's probably a better idea to create a new dataset that includes both your old and new data, and either pick up from the checkpoint using those data (similar to how you fine-tuned the Inception model), or start the training process over.
I am trying to use ssd_inception_v2_coco pre-trained model from Tensorflow API by training it with a single class dataset and also applying Transfer Learning. I trained the net for around 20k steps(total loss around 1) and using the checkpoint data, I created the inference_graph.pb and used it in the detection code.
To my surprise, when I tested the net with the training data the graph is not able to detect even 1 out of 11 cases (0/11). I am lost in finding the issue.
What might be the possibile mistake?.
P.S : I am not able to run train.py and eval.py at the same time, due to memory issues. So, I don't have info about precision from tensorboard
Has anyone faced similar kind of issue?