Models: Object Detection API - Feature Request: YOLO and YOLOv2

[VastoLorde95]

With reference to a previous thread, I am volunteering to implement the YOLO and YOLOv2 models for the Object Detection API.

[datlife]

I would like to contribute, too. Would you like to work together, @VastoLorde95 ? I have recently successfully ported YOLOv2 608x608 in C into Keras.

However, it is ideal to quantize the model for mobile applications, which Keras does not support yet.

Please let me know :)

[VastoLorde95]

I am actually already working on this with a couple of other folks and we are close to finishing it, but we would be happy to work with you on some other cool features in the future :)

[datlife]

@VastoLorde95 cool, I am looking forward to seeing your work :). Also, one cool feature is to integrate Hierarchical Soft-Max Tree into the model (discussed at YOLO9000). The idea is to not do soft-max on the whole feature vector, but rather on each subcategory.

[wendal]

any update for this ?? ^_^

[iraadit]

I would also be interested !
Thank you for your work

[mikeliao97]

Any update? Would be cool!

[VastoLorde95]

It will be ready soon :)

[niyazpk]

This is great news. Thanks for working on this. Will the YOLOV2 implementation support all the data augmentation techniques too?

[VastoLorde95]

We are done with the bulk of the implementation of YOLO, i.e., the meta architecture and the "feature extractor". But there are some things which we are not sure how to go about implementing.

Specfically we are having issues with loading pretrained weights, and the API's pipeline that automates the entire training process.

If anybody would like to help us and join us on this feature, reply below!

[rky0930]

@VastoLorde95 Hi, I'm glad to hear that there is much progress in the implementation.
I have a lot of interest in YOLO with Object Detection API and plan to use in my company work.
Let me join your work. I would like to help you as much as possible.

[VastoLorde95]

@rky0930 Awesome, let's chat over email. You can find my email id on my profile!

[datlife]

HI @VastoLorde95 , still need a hand? I can assist with the YOLOv2 architecture implementation. Once we figured out how to port the weight on YOLO, YOLOv2 should be similar (I suspect except for batch normalization).

Thanks.

[bendidi]

@VastoLorde95 also interested in assisting !

[trohit920]

Hello @VastoLorde95 any updates on API ? Many folks are waiting:)

[JulienSiems]

Same here! Do you still need some help?

[gustavz]

Any News?
Would also offer help.
Currently working on Yolo with the Jetson Tx2 and really would like to get it work with the object detection API :)=

[bendidi]

or you could point us to the unfinished fork/branch , and let us play with it :)

[KevinZhao]

I am on Jetson Tx2 with YOLO v2, hope it work on Tensorflow

[gustavz]

@KevinZhao Hey i am working on the same.
How far are you with your topic? Just running YoloV2 via Darknet on the Jetson?
How many Fps do you get?
Do you wanna connect and exchange experiences?

[KevinZhao]

@GustavZ I just made my Jetson TX2 on tensorflow object detection framework, however, the performance is too slow, and it even failed with resnet model, I am wondering the resnet model is too deep...
I would like to share my experience, please don't hesitate to send email.

[KevinZhao]

@GustavZ some updates, I get my Jetson TX2 running with Yolo v2, it is fast, and most important the memory consumption is quite low.
I am running it with YOLO v2 and tiny YOLO with COCO dataset, tiny gives me over 15 FPS, wow.

[bendidi]

I did run mobilnet ssd too on jetson Tx2 , and the speed was way too slow , around 1-2 fps (using 540x720 images )

[gustavz]

@bendidi, @KevinZhao hey guys i got similar, slightly faster results on YoloV2 and Tiny-Yolo.
I made a repo for my Object Detection Pipeline based on Googles API.
(https://github.com/GustavZ/realtime_object_detection) have a look at it, i can run Mobile SSD on about 5-6 fps and i will work on it to make it even faster after the christmas holidays. Would be cool if you try it out, you just need to clone the repo and run the python script.

Cheers,
Gustav

Edit: standard input is video0, if you wanna use Jetson's onboard Cam read this very usefull blog: https://jkjung-avt.github.io/yolov2/

Edit: With various modifications i am now able to run ssd_mobilenet with up to 15fps on the jetson tx2

[gustavz]

@VastoLorde95 any news about the progress?

[laupl0082]

Was anybody able to run any object detection with tensorFlow API on Jetson TX2 with any model other than ssd_v1?

[rky0930]

Hi. I'd like to share my YOLOv2 model which is implemented based on Object Detection API.
(This is not from @VastoLorde95. I'm waiting for his work as well. )
I thought YOLOv2 is an SSD model which has only one feature map.
There were not so many changes for YOLOv2, just add feature map extractor, anchor generators, and a loss function.

I also trained Darknet19 with Imagenet dataset for classification.
The result was 0.7 accuracy and 0.9 recall5.

The YOLOv2 model trained with PASCAL VOC and MS-COCO.
I pretty good result on MS-COCO. I could think the implementaion is work.
But, More experiences are needed to confirm officially use.

The source code is on my repos [link]
You can see details at [README.md]

If anybody would like to help me and have enough experience & computing power,
Please check out my implementation and train with various datasets.
I really welcome you find some bugs and miss-conception.

If you have a question, feel free to contact me via E-mail.
Thanks.

[gustavz]

hey @rky0930 nice to hear that.
Did you get further with your project?
Is it possible to add your files to the original object_detection repo without confilcts to the other models?

EDIT: Could you open a repo with just the files you added/changed for yolo usage? Its a pain to manually select your files from the whole tf/model repo :D

EDIT2: I did it for you, here is the repo, all credits to you
https://github.com/GustavZ/yolo_for_tf_od_api
I added you as Collaborators, would be nice if you update the files if you have new work on it!

EDIT3: i try to retrain your network but in your config for losses you put noobj_lambda=5.0 but in losses.proto you have just defined the optional variables object_scale noobject_scale
So when running training it gives following error:
google.protobuf.text_format.ParseError: 97:11 : Message type "object_detection.protos.ConfidenceWeightedSigmoidClassificationLoss" has no field named "noobj_lambda".
is this just a typo? but if yes, how were you able to train with this config in the first place?

EDIT4: after changing the typo, retraining from your provided checkpoint is not possible.
There are various errors, all of the type like:
WARNING:root:Variable [FeatureExtractor/darknet_19/base/Conv2D_9/weights] not available in checkpoint
which results in:

  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1251, in _build
    raise ValueError("No variables to save")
ValueError: No variables to save

[rky0930]

Hi @GustavZ

Did you get further with your project?
-> I trained my yolo_v2 model with VOC 2007+2012. the evaluation result is 0.5 mAP with VOC 2007 test.
-> I'm still try to figure out what did i wrong. maybe there are bugs or wrong configuration.
Is it possible to add your files to the original object_detection repo without confilcts to the other models?
-> Yes. I just changed anchor_generator builder & losses builder only for executing newly added anchor_generator and losses.

EDIT: Could you open a repo with just the files you added/changed for yolo usage? Its a pain to manually select your files from the whole tf/model repo :D

EDIT2: I did it for you, here is the repo, all credits to you
https://github.com/GustavZ/yolo_for_tf_od_api
I added you as Collaborators, would be nice if you update the files if you have new work on it!
-> Thanks! I will commit and push if there is change on my working repo :D

EDIT3: i try to retrain your network but in your config for losses you put noobj_lambda=5.0 but in losses.proto you have just defined the optional variables object_scale noobject_scale
So when running training it gives following error:
google.protobuf.text_format.ParseError: 97:11 : Message type "object_detection.protos.ConfidenceWeightedSigmoidClassificationLoss" has no field named "noobj_lambda".
is this just a typo? but if yes, how were you able to train with this config in the first place?
-> Sorry. I forgot to update the sample config file. I changed parameters of confidence loss function.
-> Try again with the new yolo_v2_darknet_19_voc.config. I commited to https://github.com/GustavZ/yolo_for_tf_od_api

EDIT4: after changing the typo, retraining from your provided checkpoint is not possible.
There are various errors, all of the type like:
WARNING:root:Variable [FeatureExtractor/darknet_19/base/Conv2D_9/weights] not available in checkpoint
which results in:

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1251, in _build
raise ValueError("No variables to save")
ValueError: No variables to save
-> I'm not sure about this error. try with new config file if it happend again. plz let me know.

[gustavz]

Hey do you know a quicker way to communicate than email?
I am training yolo right now on the egohands dataset.

Maybe WhatsApp or some online chat?

Am 31.01.2018 um 09:41 schrieb RYOO GYOUNG-YOON [e-mail]:

Hi @GustavZ

Did you get further with your project?
-> I trained my yolo_v2 model with VOC 2007+2012. the evaluation result is 0.5 mAP with VOC 2007 test.
-> I'm still try to figure out what did i wrong. maybe there are bugs or wrong configure.
Is it possible to add your files to the original object_detection repo without confilcts to the other models?
-> Yes. I just changed anchor_generator builder & losses builder only for executing newly added anchor_generator and losses.

EDIT: Could you open a repo with just the files you added/changed for yolo usage? Its a pain to manually select your files from the whole tf/model repo :D

EDIT2: I did it for you, here is the repo, all credits to you
https://github.com/GustavZ/yolo_for_tf_od_api
I added you as Collaborators, would be nice if you update the files if you have new work on it!
-> Thanks! I will commit and push if there is change on my working repo :D

EDIT3: i try to retrain your network but in your config for losses you put noobj_lambda=5.0 but in losses.proto you have just defined the optional variables object_scale noobject_scale
So when running training it gives following error:
google.protobuf.text_format.ParseError: 97:11 : Message type "object_detection.protos.ConfidenceWeightedSigmoidClassificationLoss" has no field named "noobj_lambda".
is this just a typo? but if yes, how were you able to train with this config in the first place?
-> Sorry. I forgot to update the sample config file. I changed parameters of confidence loss function.
-> Try again with the new yolo_v2_darknet_19_voc.config. I commited to https://github.com/GustavZ/yolo_for_tf_od_api

EDIT4: after changing the typo, retraining from your provided checkpoint is not possible.
There are various errors, all of the type like:
WARNING:root:Variable [FeatureExtractor/darknet_19/base/Conv2D_9/weights] not available in checkpoint
which results in:

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1251, in _build
raise ValueError("No variables to save")
ValueError: No variables to save
-> I'm not sure about this error. try with new config file if it happend again. plz let me know.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[rky0930]

@GustavZ How about Slack?
Please send me an email first :D
[email protected]

[gustavz]

@rky0930 i already sent you an email a view days ago.
Now i sent you a slack invitation for a new workspace

I would like to welcome anybody who is interested in the topic or likes to contribute to join the slack workspace:
https://join.slack.com/t/yolofortf/signup

[rzhai]

@GustavZ after click above slack link, it is asking @bshg.com. How do I apply that email?

Has anyone used object_detection/export_inference_graph.py to export .pb using @rky0930 "Frozen weight (trained on Pascal VOC) for YOLOv2 model. this weight still alot need to improve; (download)"

[digitalfalconAU]

@GustavZ Can you please send me an invitation to join the slack workspace?

[bendidi]

also interested in joining, and may need an invitation , thx

[gustavz]

@digitalfalconAU @rzhai guys i need your email, they are not visible in your github profile

[rzhai]

@GustavZ I made my email public now on my profile. [email protected]
Thanks.

[digitalfalconAU]

@GustavZ Thanks! [email protected]

[iraadit]

@GustavZ I'm interested as well. [email protected]

[ilkarman]

@GustavZ Hi, also interested! Email: ilia.[email protected]

[ezeddin]

@GustavZ Hi, also interested! Email: e.[email protected]

[DaneSpiritGOD]

@GustavZ hi, also interested! Email:[email protected]

[shivam6294]

Also interested. Email: [email protected]

[shivam6294]

@GustavZ Please send me an invitation. Thank you!

[DaneSpiritGOD]

@GustavZ Please send me an invitation.Thank you! sihan.[email protected]

[for-aiur]

Could you please send me an invitation as well? [email protected]

Thanks a lot.

[gustavz]

https://join.slack.com/t/yolofortf/shared_invite/enQtMzQ3NzQ4MTQ5NDI5LTNkMDkzYjA4ZjY1OWM4ZjIwN2ZjZGIwZTU2YjFkZDJmNGRiNTRiYjczMTQzMjQ4ODE1NGRkMGM4Njc4YmQ2Mzg

Invite yourself y’all :)

[crazysal]

This thread totally digressed. Did anyone get Yolo v2 working ?

Any hope of having it added to the official repo anytime soon ?? @VastoLorde95 @GustavZ @rky0930 ???

[gustavz]

@crazysal
this project contains the necessary files that need to be included to the API to train on YoloV2 (https://github.com/GustavZ/yolo_for_tf_od_api), that's a work of @rky0930. but i don't know if he is still working on this and which final results he achieved.
My own results were below expectations and because i pretty quickly changed to another project i did not investigate further on how to improve / tune it. But feel free to use it :)

[YijinLiu]

Hi, I wonder has anybody ever worked on YOLO v3, which seems to be better than SSD in both accuracy and speed.

[tanndx17]

@YijinLiu Have you seen anybody implementing YoloV3 in tensorflow object detection api? I am very looking forward for that, especially the AP and speed at mobile end

[YijinLiu]

@tanndx17 There are several promising projects, like
https://github.com/YunYang1994/tensorflow-yolov3
https://github.com/maiminh1996/YOLOv3-tensorflow
I don't think any of these are mature / stable for production usage yet.

I am actually looking for MobileNet 2 + YOLOv3.
There are several projects close to that:
https://github.com/Adamdad/keras-YOLOv3-mobilenet
https://github.com/sufeidechabei/gluon-mobilenet-yolov3 (also see https://gluon-cv.mxnet.io/model_zoo/detection.html)
These are not tensorflow based though.

There are many people trying to make YOLOv3 work with Tensorflow/MobileNet, many of these experiments are showing exciting results, in both precision and speed (sufeidechabei claims 3x speed compared to SSD).
It would be nice if tensorflow/models would have YOLOv3, which should save a lot of these duplicate efforts and produce better / stable results.

[YijinLiu]

Found another interesting project: https://github.com/zzh8829/yolov3-tf2 (I haven't tested it yet)

[DrChungAlbert]

Hi :
Does anyone know how to make the quantization-aware training YOLOv2/v3 work by using the Object Detection API ?

[jah3xc]

Any updates on when a YOLO will be part of the model zoo?

[xuldor]

Hi, @rky0930 @gustavz
Thank you for yolov2 with tensorflow od api !
I tried your code, and i got an error: google.protobuf.text_format.ParseError: 32:7 : Message type "object_detection.protos.AnchorGenerator" has no field named "yolo_anchor_generator".
can you help me ?

[AlexeyAB]

Is there any reason why Yolo v1/v2/v3 is not part of the model zoo?

[McBusinessTime]

I see multiple PRs for YOLO related changes. Any ETA on when YOLO models will be available in the OD API?