Models: there is something wrong when detect with the exporting trained model.

[123chengbo]

1、 I tried to fine-tune a model ,using Oxford-III pet dataset（i have generated tfrecord files already）, with the base files: faster_rcnn_resnet101_coco_11_06_2017, then run the command below:

pwd: home/chengbo/tensorflow_models

python object_detection/train.py --logtostderr --train_dir=object_detection/train_result --pipeline_config_path=faster_rcnn_resnet101_pets.config

2、in faster_rcnn_resnet101_pets.config:
fine_tune_checkpoint:"object_detection/models/faster_rcnn_resnet101_coco_11_06_2017/model.ckpt"
train.....input_path:"object_detection/data_records/pets/pet_train.record"
label_map_path:"object_detection/data/pet_label_map.pbtxt"
eval......input_path:"object_detection/data_records/pets/pet_train.record"
label_map_path:"object_detection/data/pet_label_map.pbtxt"

3、i trained about 30hours， step 116416 ，loss = 0.013，then i got ：graph.pbtxt、model.ckpt-116416.data-00000-of-00001、model.ckpt-116416.index、model.ckpt-116416.meta；（and i‘ve watched the tensorboard graph， the loss get lower normally ）

4、then i would like to use this model to detect the pet dataset， firstly i use command:
python object_detection/export_inference_graph.py input_type image_tensor --pipeline_config_path faster_rcnn_resnet101_pets.config --checkpoint_path object_detection/train_result/model.ckpt --inference_graph_path output_inference_graph.pb

and i got a output_inference_graph.pb ,then use this in the object_detection_tutorial.py(modified from object_detection_tutorial.ipynb),　the detect result is really bad ， boxes are bad、and class is not correct（i use 20pictures， ）；
but when i used the frozen_inference_graph.pb from faster_rcnn_resnet101_coco_11_06_2017(which i used for finetune), the detect boxes is very correct、 and class is all mixed with others（eg：10 boxer_xx.jpg , result are all japanese_chin, 10 Egyptian_Mau_xx.jpg, result class are havanese）,and score is very high.
what is wrong with this？

tensorflow-1.2.0（from binary）
anaconda3-4.2.0
GTX 1080， 8G

[timisplump]

I have a very similar problem, although I spotted the error before even exporting to an inference graph (it may be the same cause?)
While training, the train loss goes down normally until it plateaus at some arbitrary fixed value (always around loss = 0.3496), and then when I evaluate the model, i get mAP's of 0 on every class. However, when I evaluate the out-of-the-box model, the mAP's are decent and work fine, so I don't think it's my dataset.

I'm using a batch size of 8 on the ssd_mobilenet_v2_coco model, and using essentially the same configs as the default otherwise. I'm on tensorflow 1.2.0 on a TITAN X

[cooliscool]

Someone please help me on converting the checkpoint files ( 3 of them ) into a frozen graph ( pb ) file.

A frozen graph file is required when running the object detection notebook in the API here notebook .

Or is it possible to run the detection directly from ckpt files ?

[tombstone]

@cooliscool please use object_detection/export_inference_graph.py to export a frozen graph.

[123chengbo]

@tombstone my question is when i use the export_inference_graph.py to export a frozen graph， the result is really terrible， can you give me some advice？
thank you

[cooliscool]

@tombstone , Thankyou!
That worked for me.
I could generate frozen inference graph pb files.
@123chengbo Please make sure you are providing the proper .config pipeline file.

[123chengbo]

@cooliscool .
i use the pipeline_config = which is i trained for my own model , is that wrong?

[cooliscool]

That's fine. I think I didn't get you properly.
Are you able to detect objects using the generated pb file ?
Did you try detection on any of the training dataset image, as well as on some new image outside training dataset ?
My trained model is really terrible on new images. But I have ran it only for ~50k iterations, and dataset was small.

[123chengbo]

yes ，i can detect objects with the generated pb file ，but the result is really bad（my company forbidden upload anything ），the IOU between result boxes with ground truth are about < 0.6， and labels are almost all right ；
when i use the pb file in faster_rcnn_resnet101_coco_11_06_2017/frozen_inference_graph.pb, the detected boxes are very correctly（IOU > 90%）、but label is wrong

[cy89]

This is a question better suited for StackOverflow, which we also monitor. Please ask it there and tag it with the tensorflow tag.