Detectron: Is it possible to generate validation error????

[siddiquemu]

I can generate training loss and accuracy vs iteration to observe the training progress for my own dataset. I am curious to check the validation loss and accuracy also to stop the over fitting. can anyone help me to get validation error during training?

Network use both training and validation set for training. There is no validation performance, right?

[gadcam]

@siddiquemu Looks like this should be a partial answer to your question https://github.com/facebookresearch/Detectron/blob/b3c93df2cecca1139f73d005b9dfcd83ef55c16d/FAQ.md#q-how-do-i-compute-validation-ap-during-training

[ccpocker]

@siddiquemu Could you share your code for visualization train loss and train accurancy? Very Appericate for sharing if you accept

[siddiquemu]

`@ccpocker return training loss and accuracy after each 20 iterations in train.py script

      if (cur_iter % 20 == 0 or cur_iter == cfg.SOLVER.MAX_ITER - 1):
         accuracy,loss = training_stats.accuracy_loss_stats(cur_iter, lr)
         train_loss.append(loss)
         train_accuracy.append(accuracy)
         print('train_loss:',loss)
         print('train_accuracy:',accuracy)
             np.savetxt('/path/train_loss.txt',train_loss,delimiter = ',')
             np.savetxt('/path/train_accuracy.txt',train_accuracy,delimiter  = ',')

`# function in training_stats.py script

def accuracy_loss_stats(self, cur_iter, lr):
      stats = self.GetStats(cur_iter, lr)
      accuracy = stats['accuracy_cls']
      loss = stats['loss']
      return [accuracy,loss]

[ccpocker]

@siddiquemu Very appreciate for your sharing

[RakhithaRR]

How did you manage to set up a custom database other than COCO. I'm also trying to do the same thing, but with no luck. Any help is appreciated.

[siddiquemu]

@RakhithaRR You can just prepare training instances similar to .json files of coco dataset.

[Zhaosoi]

@siddiquemu Hi, can you get validation loss and accuracy curve during training now? Could you share your code. Many thanks!