Edgetpu: Coral TPU transfer learning sample is not working

[neilyoung]

I already reported this problem earlier to no avail.

There are several issues with this tutorial here https://coral.ai/docs/edgetpu/retrain-detection/ and I'm completely baffled that nobody of you cares:

1) Regardless of how many traning steps I configure, the result is always: No object detected from the dog picture. I tried with 500 up to 2000. The calculation runs between 8 and 12 hours and then stops with an obscure warning, but in the end the detection result is NULL

2) The final command in the tutorial contains a wrong directory. It needs to be object_detection instead of detection

python3 detect_image.py \
--model ${HOME}/google-coral/tutorials/docker/object_detection/out/models/ssd_mobilenet_v1_catsdogs_quant_edgetpu.tflite \
--labels ${HOME}/google-coral/tutorials/docker/object_detection/out/models/labels.txt \
--input ${HOME}/google-coral/tutorials/docker/object_detection/out/models/dog.jpg \
--output dog_result.jpg

Feel free to ignore, but then I would suggest to close this issues section too.

Unbelievable...

[neilyoung]

This is all not worth the efforts. 5000 steps. "No object detected". What a mess....

[masc-it]

@neilyoung I won't get into your doc-not-perfect complaints, I just believe the team is busy with more important dev-related stuff ! (I challenge you to find a more complete starter-doc for a system like this, if it exists at all )

Anyways, you should provide more detailed information about your obscure warning, otherwise no one's gonna understand you.

PS: check this notebook of mine, it contains full code to train a Coral-TPU-ready object detector using tf 1.15, straight on Google Colab. It might help you! (no docker needed) JFYI (and for whoever's gonna read this), if you navigate in the repo you'll find also notebooks for classification and segmentation. They are not perfect, but may enlight you :)

[neilyoung]

Well, look. If they provide a "TUTORIAL" I think it totally legit to expect something working. This is just not working. And the obscure warning - I think, this is not tutorial related, more a tensorflow thing.

And I don't even "just complained". I also provided a correction. And this not just once. They ignore, so I don't think they are "too busy". This is Google, and it is as it always is: Half done things, updated quickly and buried out of the sudden. Why do I always again fall into this trap? I should have known better...

I'm not interested in classification or segmentation. I'm just interested in detection. And there is no model available, which just concentrates on persons, like personnet I just wanted to get rid of all the toothbrushes etc.

[Namburger]

@neilyoung i thought I shared this with you?
https://github.com/Namburger/edgetpu-ssdlite-mobiledet-retrain

I’m no longer with the coral team, but I can assure you that they’re working to get around to everyone.

[neilyoung]

@Namburger Yes, and I contributed a PR to this. The result was the same. "No object detected". I reported https://github.com/google-coral/edgetpu/issues/330

[Namburger]

@neilyoung have you tried other images?

[neilyoung]

@Namburger Do you mean training images or final test images? The answer is no for both. Would you be able to run your own Jupyter book and provide me a file I could load into the Coral? That at least I would see, that kind of this operation would work?

[Namburger]

You could just try a different image for testing, in my colab there are 5 other test images that was evaluated before compiling to the edgetpu and downloading the model

merit: in the evaluation stage, the results is actually shown, which is working. Also, it isn’t Jupiter notebook, colab is a free tool from google that allows you to run the training on Google’s free GPU so you wouldn’t have to worry about taking up your own resources

[neilyoung]

Yes, I remember. I tried them all in the end. No detection at all.

And the Colab thing is also not working very well for me. It runs for a while then you get locked out for unclear times. You cannot purchase a license ouside US. So what...

[Namburger]

As a matter of fact, just clone my repo and run this (I provided a sample output model with 8 test images):

python3 run_model.py models/ssdlite_mobiledet_dog_vs_cat_edgetpu.tflite test_images

[neilyoung]

Thanks. I appreciate your help, but I would like to create my own with some means at reasonable time. It is not about dogs and cats, in the end I need to adapt a resnet model to detect persons only in order to increase the inference rate. I tried the erroneous Google tutorial, the one, which provided me such nice comments here, as suggested with 500 steps. No detection. With 1000 steps. No detection. With 2000 steps. No detection. Currently running since > 28 hours with 25.000 steps. It didn't crash so far. That's a success. Will it work finally? Who knows...

I really like that Coral TPU, but this transfer learning is simply not working for me. Obviously not allowed to say that here.

[Namburger]

I wrote an entire article on it along with my colab, that should give you good insight on how to train it on other images. There’s also another colab on the bottom for mask detection plus I BBelieve I linked you to the official tf OD API doc? That’s really where them he source of our model came from

I suggest reopening the issue so the team still have it in the queue and they will get to fixing the official tutorial when they have a chance

[neilyoung]

Thanks again. And no. They are not interested and I'm not either. Chapter closed. Thanks

[neilyoung]

@Namburger Ran your code on my RPI4. What is it supposed to do?

It prints something but nothing else happens:

pi@detector:~/edgetpu-ssdlite-mobiledet-retrain $ python3 run_model.py models/ssdlite_mobiledet_dog_vs_cat_edgetpu.tflite test_images
Evaluating: test_images/image1.jpg
Evaluating: test_images/image2.jpg
Evaluating: test_images/image3.jpg
Evaluating: test_images/image4.jpg
Evaluating: test_images/image5.jpg
Evaluating: test_images/image6.jpg
Evaluating: test_images/image7.jpg
Evaluating: test_images/image8.jpg
Inference time: 0.0789523560000589

[Namburger]

@neilyoung if you look at the code, it'll show the image (but you need to connect the pi to a monitor and run the code on the pi instead of over a ssh session):
https://github.com/Namburger/edgetpu-ssdlite-mobiledet-retrain/blob/master/run_model.py#L77

If you want it to print the results also, simply add this on line 76, between the text = ... and the draw.... line line this:

      text = labels[int(classes[i])] + ' ' + str(scores[i]*100) + '%'
      print(text)
      draw.text((xmin+2, ymin-10), text, fill=(0, 0, 0), width=2)

The showm images should be similar to this:
https://github.com/Namburger/edgetpu-ssdlite-mobiledet-retrain#quick-run

[neilyoung]

I was running it via SSH first, with export DISPLAY=:0, that works with my inference codes. Then I used a monitor and keyboard. Same results. I suppose I must be missing a dependency.

With the test print the results are clearly different, so it works... Now I would like to have the same :) You said, you did 3000 steps? How many evals? (NUM_STEPS, NUM_EVALS)?

pi@detector:~/edgetpu-ssdlite-mobiledet-retrain $ python3 run_model.py models/ssdlite_mobiledet_dog_vs_cat_edgetpu.tflite test_images
Evaluating: test_images/image1.jpg
american bulldog 91.796875%
Evaluating: test_images/image2.jpg
abyssian cat 97.65625%
Evaluating: test_images/image3.jpg
abyssian cat 99.21875%
Evaluating: test_images/image4.jpg
american bulldog 95.703125%
Evaluating: test_images/image5.jpg
american bulldog 98.828125%
Evaluating: test_images/image6.jpg
abyssian cat 99.609375%
Evaluating: test_images/image7.jpg
american bulldog 91.015625%
Evaluating: test_images/image8.jpg
abyssian cat 93.75%
Inference time:  0.07726445500020418
pi@detector:~/edgetpu-ssdlite-mobiledet-retrain $ python3 run_model.py models/ssdlite_mobiledet_dog_vs_cat.tflite test_images
Evaluating: test_images/image1.jpg
american bulldog 91.015625%
Evaluating: test_images/image2.jpg
abyssian cat 97.65625%
Evaluating: test_images/image3.jpg
abyssian cat 99.21875%
Evaluating: test_images/image4.jpg
american bulldog 96.09375%
Evaluating: test_images/image5.jpg
american bulldog 98.828125%
Evaluating: test_images/image6.jpg
abyssian cat 99.609375%
Evaluating: test_images/image7.jpg
american bulldog 91.015625%
Evaluating: test_images/image8.jpg
abyssian cat 92.578125%
Inference time:  2.7272272239995345
pi@detector:~/edgetpu-ssdlite-mobiledet-retrain $ 

[neilyoung]

This does not show any image from the test_images directory, not sure why...

from PIL import Image
im = Image.open('image.jpg')
im.show()

[neilyoung]

sudo apt-get install imagemagick

was missing...

At least it works now, but you should mention it

[Namburger]

@neilyoung I don't remember the exact params but in general, I suggest that you play around with it..
The exact same set of images with the same params may not give the same exact result, welcome to ML :)

[neilyoung]

The exact same set of images with the same params may not give the same exact result, welcome to ML :)

Well, the situation is: Here is NO set of params, which gives ANY result (other than "No detection"). If this is ML than thank you very much :)

[neilyoung]

@Namburger OK, thanks to your impulse I tried your Colab again today. By that occasion I found an already "colabed" tflite edgetpu file from Feb 27, showing me, that I already once came through. I tried this model on my PI and found, that it not detected 2 images out of the bunch of test images. If I recall correctly, this was the moment, at which I gave up with Google. And in the past I never made it to return to it, I was always locked out, since I exceeded my free quota.

Returning to this was worth is, because I recovered several things:

1) The original Google tutorial is crap, is not working that way. As reported I tried to run it with 25.000 steps. It ran > 28 hours, but then broke with "Killed". I know, memory problem... Anyway, this is not practical.

2) Your Colab is only producing reliable results if NOT run on a T4, since the number of steps is too small. Today I was in luck and got a PCI, which ran 25.000 steps in about 7 hours

3) The resulting tflite edgedtpu file worked finally. Not too bad all in all: Compared to the non-accelerated model (3.2 seconds) it rushed through in 0.09 seconds... And it detected all the dogs and cats.

Anyway, if you are still maintaining this Colab, there are two things:

1) The evaluation step after finishing the training does not work. It circles and circles but nothing is shown. Chromium at Ubunutu 18.04

2) Step 15 does obviously not work on the Google server, should be excluded there

ERROR: tflite_runtime-2.5.0-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.

2) Step 18, the download, for some reasons does also not work. Doesn't come to an end. Fortunately I was able to download the tar via the menu.

Thanks nevertheless. At least a more than half-way working sample :)