Imageai: How to do fine-tuning with ImageAI?

[zlbleues]

Hi,

Thank you for the great work!

I would like to modify the model resnet50_coco_best_v2.1.0.h5 to modify the weights by adding a sample of data. This sample is not very large.
I would like to refine the object detection on a photo or video, especially looking for cars.

I read the readme and the examples but I did not find a solution to my problem. It's been a while now that I search on the internet but without success.

Could someone inform me?

thank you in advance

[OlafenwaMoses]

Hello,

If you are trying to recognize one or a few distinct object(s), you can use the Custom Object Detection and Custom Objects Video Detection provided in ImageAI. See the links below:
For Images : https://github.com/OlafenwaMoses/ImageAI/tree/master/imageai/Detection#customdetection

For Video : https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/VIDEO.md#customvideodetection

You will find in the above links the different objects the current RetinaNet model file provided can detect.

In future versions of ImageAI, we will allow training new models for detecting your own list of objects.

[zlbleues]

Hello,
Thank you for your answer.
I already use the custom object list.

I think I misunderstood. I want to modify the model: resnet50_coco_best_v2.0.1.h5. This is to add new images of cars to improve the detection. How is this possible?

thank you in advance

[OlafenwaMoses]

In future versions of ImageAI, we will support training object detection models on new and custom images.

[zlbleues]

When will the new version come out?

[OlafenwaMoses]

Very soon. In maximum of a month's time.

[ankitaggarwal-trantor]

Hi @OlafenwaMoses

Any luck that by when we would be able to train resnet with custom objects for object detection in videos.

Regards,
Ankit Aggarwal

[OlafenwaMoses]

My team is working on it. Due to other projects that we have at hand, the feature will most likely be available by December this year. However, there will 2 or more releases before then that will come with improvements to existing features and addition of new features and models.

[drorhilman]

can you provides a script that show how did you train these models in the first place?

[yuskey]

What's the status on this feature? I'd like to train on images with classes outside the 80 included.

[polytech1]

What's the status on this feature? I'd like to train on images with classes outside the 80 included.

Me too

[OlafenwaMoses]

We are looking at adding this feature latest by January 2019. We will keep you all updated.

[Poonamjo]

Is there any progress on this?

[pytesnim]

Hello,
Is it possible to use custom trained objects fordetection in videos? Thanks

[Zengdongxu]

Hello @OlafenwaMoses

I really appreciate your work Imageai, it's short and effective, and I gonna use it in my graduation thesis. But I need to train my own image models outside the 80 classes to perform video detection, is it possible for me to get new release before March? Otherwise I have to chose other libraries?

All the best in your work!

[alexmotoc]

Hello,

I would like to try using custom trained objects for my university project (hands from the VIVA challenge in particular). Any updates on the timeline of this release?

Thanks and keep up the great work guys, this is very intuitive to use!

[OlafenwaMoses]

Hi everyone, support for training object detection models on custom dataset/objects will be added later than I initially mentioned. Due to my schedule and work on other projects, I have shifted the window for this to early April.

However, I will update you all soon on a new AI project which will also be open-sourced. The new project will provide more features in recognition, detection and custom models which programmers and developers working with any programming language can use.

Will drop new updates soon.

[veesamkrao999]

Hello,

If you are trying to recognize one or a few distinct object(s), you can use the Custom Object Detection and Custom Objects Video Detection provided in ImageAI. See the links below:
For Images : https://github.com/OlafenwaMoses/ImageAI/tree/master/imageai/Detection#customdetection

For Video : https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/VIDEO.md#customvideodetection

You will find in the above links the different objects the current RetinaNet model file provided can detect.

In future versions of ImageAI, we will allow training new models for detecting your own list of objects.

Can we add "New objects" to "object detection model" that it contains 80 objects. ? If yes then how to add new objects?

[rageshgopan]

Hello @OlafenwaMoses .. eagerly waiting for training object detection models on custom dataset/objects. Hope you will have a update for us soon .

I am trying to count the objects detected based on custom dataset. Hope this feature will support it

Thanks in advance !

[Prati369]

Hi @OlafenwaMoses .

Are there any updates on the new AI project?

Can the model be trained for fish eye images?

[RomuloUnespian]

Hi @OlafenwaMoses

Looking forward for the updates on the AI project!!!

[xinming31]

Hi @OlafenwaMoses,
Long time past, we are still waiting.

[gautamkadian]

Hi @OlafenwaMoses,

Really appreciate your work on IMAGEAI.

I read out the above message trail, Is there any way to train the same model with car type and etc.
or
Is there any way that i can convert the h5 model into tensorflow model's.

Really appreciate for your reply

Thanks

[JStuve]

Excited for this feature!

[AlenaSuchilkina]

Good day!
Earlier you wrote that this feature will be released in early April 2019. When will the update be released?
I use your library in my diploma. Many thanks for your work!

[sathiya1926]

Hello Moses,

Eagerly waiting for the new project, this will really help us.

[montycobra]

Hey imageAI team great job !! While we all wait for this new feature. Can we list down some alternatives here. Don't wish this to be dead end I can see lot of us eagerly awaiting delivery of new feature and it could be helpful to try out on some alternatives softwares. We can potentially enrichen imageAI based on the experience of the alternatives. Win win

[bmagnetta]

Fine tuning would be a great ImageAI feature! In the meantime I found a work around for a specific case; the YOLOv3 pre-trained model used in ImageAI. There exists code to take YOLOv3 and fine tune it with new data (keras-yolo3). I was able to get an ImageAI object detection example working specifically for cellphone objects. Thought I'd share in case some of you need to implement a feature like this sooner rather than later: cellphonefinder

[OlafenwaMoses]

See this article for latest update on this.

https://towardsdatascience.com/the-story-and-future-of-imageai-one-year-anniversary-e63c80f527c8

[JStuve]

Following @OlafenwaMoses

TLDR: June 15, 2019

We get the following related to Custom Modeling:

• Save custom trained models in full, allowing custom recognition with any Keras model without specifying model type
• Export custom trained model to Tensoflow .pb format

[nineclicks]

Forgive me, I am trying to understand. Will this update allow us to train custom objects and do object detection with bounding boxes on custom objects?

[OlafenwaMoses]

Hello everyone, we just published a new update (v 2.0.3) with the features as mentioned by @JStuve and more. You can find details on the new features in the repository's description. https://github.com/OlafenwaMoses/ImageAI/blob/master/README.md

Sadly, training of custom models for object detection is yet to be included. We understand this is a highly demanded feature and we are working hard to provide it in the most simple yet customizable implementation possible. Also, delay on this feature has been largely due to the number of projects at hand.

We have moved the support for training custom object detection models to July 30, 2019. All things being equal, we will deliver this feature by this date unfailingly. Thank you all for your patience.

[Prati369]

Hi,
Does it support 3D annotation of objects?

[patrickduhaime]

We have moved the support for training custom object detection models to July 30, 2019. All things being equal, we will deliver this feature by this date unfailingly. Thank you all for your patience.

Any update on this ?

Thank you

[Amir-Omidfar]

Hi @OlafenwaMoses ! Thanks for your great work!
Just wanted to know if there's any update for custom object detection feature on your platform?

Thanks

[Gytis123]

'We have moved the support for training custom object detection models to July 30, 2019. All things being equal, we will deliver this feature by this date unfailingly. Thank you all for your patience.'

Hello

Any updates?

[OlafenwaMoses]

Hello everyone, I really appreciate the patience you have all exercised since the beginning of this thread.

I am glad to announce that training and performing image detection, video detection and video analysis with YOLOv3 on your custom image datasets is now available. KINDLY READ THE DETAILS BELOW.

N.B: The new features have been released but not documented yet in README and official API documentation. This will will done before the end of the week.
However, we have provided sufficient example codes, example datasets and sample trained models for you to start using this feature. Everything you need to start using the new features are detailed below.

1) INSTALLATION:
ImageAI (v2.0.1) is now available on PyPi. To install the latest release, install via the command below.

pip3 install imageai --upgrade

2) ANNOTATING YOUR CUSTOM DATASET
For custom detection model training, ImageAI uses the Pascal VOC format for image annotation. You can easily annotate your dataset using the tool LabelIMG which is available on Windows, Linux and MacOS

https://github.com/tzutalin/labelImg

Also, I will advice that your image dataset contains up to about 200 image samples, from which you can allocate 150 for training and 50 for validation. Your dataset can contain one or many objects.

3)ORGANIZING YOUR CUSTOM DATASET
Once you have annotated your dataset in the Pascal VOC format, ensure your dataset directory is organized as in the sample below.

`

train >> images >> img_1.jpg
train >> images >> img_2.jpg
train >> images >> img_3.jpg
train >> annotations >> img_1.xml
train >> annotations >> img_2.xml
train >> annotations >> img_3.xml

validation >> images >> img_151.jpg
validation >> images >> img_152.jpg
validation >> images >> img_153.jpg
validation >> annotations >> img_151.xml
validation >> annotations >> img_152.xml
validation >> annotations >> img_153.xml`

4) TRAINING ON YOUR CUSTOM DATASET

You can train your custom detection model completely from scratch or use transfer learning (recommended for better accuracy) from a pre-trained YOLOv3 model. Below is the link to the pre-trained model and also sample datasets you can use.

https://github.com/OlafenwaMoses/ImageAI/releases/tag/essential-v4

The example code to train your object detection model is as linked below after the notes.

N.B:

• The default batch_size is 4. If you are training with Google Colab, this will be fine. However, I will advice you use a more powerful GPU than the K80 offered by Colab as the higher your batch_size (8, 16), the better the accuracy of your detection model.
• If you experience '_TfDeviceCaptureOp' object has no attribute '_set_device_from_string' in Google Colab, it is due to a bug in Tensorflow. You can solve this by installing Tensorflow GPU 1.13.1 pip3 install tensorflow-gpu==1.13.1.

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_train.py

Once your training starts, your models and the _detection_config.json_ file you need for performing custom detection will be saved in your dataset/models and dataset/json folder respectively.

5) EVALUATING YOUR MODEL's mAP

After traing your model, you can evaluate the mAP of your models by specifying your desired IoU and Non-maximum suppression values. The sample codes are linked below.

Single Model Evaluation = https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_single_model_evaluation.py

Multi-Model Evaluation = https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_multi_model_evaluation.py

6)IMAGE AND VIDEO OBJECT DETECTION WITH YOUR CUSTOM MODEL

We have provided a number of sample codes for using your custom detection model in detecting objects in images and videos. In addition, we have provided a sample model trained to detect the Hololens headset. See the links below for all the resources.

https://github.com/OlafenwaMoses/ImageAI/releases/tag/essential-v4

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection.py

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_array_input_output.py

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_extract_objects.py

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_from_array_extract_objects_array.py

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_from_file_extract_objects_array.py

https://github.com/OlafenwaMoses/ImageAI/blob/master/examples/custom_detection_video.py

[OlafenwaMoses]

GitHub repo documentation updated. Official documentation via https://imageai.readthedocs.io coming and tutorial coming soon.

[plenari]

thanks verymuch

[OlafenwaMoses]

Full tutorial on this.

https://medium.com/deepquestai/train-object-detection-ai-with-6-lines-of-code-6d087063f6ff

https://medium.com/deepquestai/object-detection-training-preparing-your-custom-dataset-6248679f0d1d

[nineclicks]

Is it possible to keep the classes in the pretrained model? Say I create a model for A, B and C. Then later use that model to train D, how would I create an detection_config.json with A,B,C and D? I am not clear on what the "anchors" are.

[OlafenwaMoses]

It will surely be a good idea to keep the classes in the pre-trained model. However in transfer learning, the newly trained model is adjusted to the new training data. The only way to keep the previous classes is to include the images for these classes in the training data.

For the detection_config.json file, contents of the JSON file as as detailed below:

• Names of your classes to enable the trained model obtain the corresponding labels for its index representation.
• The anchor box values. This values are used to define the best fit coordinates for the shape and size of your all objects samples (train data). This values are used by the YOLOv3 architecture to detect objects across 3 scales. You can learn more via the link below.

https://github.com/pjreddie/darknet/issues/568

[OlafenwaMoses]

Hello everyone, please see an urgent bug fix update here #278

[Eddie-cyberking]

I got an error while loading custom trained model.
Here is my code
detector = CustomObjectDetection()
detector.setModelTypeAsYOLOv3()
detector.setModelPath(os.path.join(baseFolder, 'models/detection_model-ex-05--loss-20.23.h5'))
detector.setJsonPath(os.path.join(baseFolder, 'json/detection_config.json'))
detector.loadModel()

File "tensorflow-gpulibsite-packagestensorflowpythonframeworkops.py", line 1662, in _create_c_op
raise ValueError(str(e))

ValueError: Dimension 0 in both shapes must be equal, but are 1 and 18. Shapes are [1,1,1024,855] and [18,1024,1,1]. for 'Assign_360' (op: 'Assign') with input shapes: [1,1,1024,855], [18,1024,1,1].

[OlafenwaMoses]

@Eddie-cyberking please refer to #271

[Eddie-cyberking]

Hi @OlafenwaMoses The problem of #271 due to using the wrong class for custom video detection. But in my case, I am using CustomObjectDetection for image detection. Am I using the wrong class?

[OlafenwaMoses]

@Eddie-cyberking have you tried using other models generated during the training? Let me know if you experience the same problem with those too.

[Eddie-cyberking]

@OlafenwaMoses It is ok now. The problem is caused by using a mismatched json with the trained model. Thanks for your help.

[OlafenwaMoses]

@Eddie-cyberking You are welcome.

[auphofBSF]

hello @OlafenwaMoses I have your hololens training example for Object Detection working very well thank you.

Now I am trying to setup a custom Object Detection training but with 3 objects Object1, Object2, Object3. I have tried various combinations of structuring the training data directories without success.

could you please supplement your instructions at #issuecomment-516432082 on how to structure with multiple objects

3)ORGANIZING YOUR CUSTOM DATASET

`
Once you have annotated your dataset in the Pascal VOC format, ensure your dataset directory is organized as in the sample below.

train >> images >> img_1.jpg
train >> images >> img_2.jpg
train >> images >> img_3.jpg
train >> annotations >> img_1.xml
train >> annotations >> img_2.xml
train >> annotations >> img_3.xml

validation >> images >> img_151.jpg
validation >> images >> img_152.jpg
validation >> images >> img_153.jpg
validation >> annotations >> img_151.xml
validation >> annotations >> img_152.xml
validation >> annotations >> img_153.xml``

[OlafenwaMoses]

All you need to do is keep all images of all objects in the same folder as well as their corresponding annotation XMLs in the same folder. Then set your object_names_array in the .setTrainConfig() function to the names of the unique objects as seen in the example below.

trainer.setTrainConfig(object_names_array=["hololens", "google-glass", "oculus"], ........)

[OlafenwaMoses]

@auphofBSF I have updated the code and README documentation to clarify this. See the link below.

https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/Custom/CUSTOMDETECTIONTRAINING.md#trainingdataset

[auphofBSF]

@auphofBSF I have updated the code and README documentation to clarify this. See the link below.

https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/Custom/CUSTOMDETECTIONTRAINING.md#trainingdataset

Thank you very much @OlafenwaMoses , you have got me onto the right track and all appears to be working well.
I have further to your doc update submitted PR #295 to further clarify directory structure for multiple object learning

[OlafenwaMoses]

@auphofBSF I have updated the code and README documentation to clarify this. See the link below.
https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/Custom/CUSTOMDETECTIONTRAINING.md#trainingdataset

Thank you very much @OlafenwaMoses , you have got me onto the right track and all appears to be working well.
I have further to your doc update submitted PR #295 to further clarify directory structure for multiple object learning

You are most welcome. I have made some comment on the PR.

[Thanakorn-VK]

I got an error please help!

Generating anchor boxes for training images and annotation...
Average IOU for 9 anchors: 0.95
Anchor Boxes generated.
Detection configuration saved in nice/json/detection_config.json
Some labels have no annotations! Please revise the list of labels in your configuration.
Training on: None
Training with Batch Size: 4

Number of Experiments: 100

TypeError Traceback (most recent call last)
in ()
5 trainer.setDataDirectory(data_directory="nice")
6 trainer.setTrainConfig(object_names_array=["nappacabbage"], batch_size=4, num_experiments=100, train_from_pretrained_model="pretrained-yolov3.h5")
----> 7 trainer.trainModel()

1 frames
/usr/local/lib/python3.6/dist-packages/imageai/Detection/Custom/generator.py in __init__(self, instances, anchors, labels, downsample, max_box_per_image, batch_size, min_net_size, max_net_size, shuffle, jitter, norm)
34 self.net_w = 416
35
---> 36 if shuffle: np.random.shuffle(self.instances)
37
38 def __len__(self):

mtrand.pyx in numpy.random.mtrand.RandomState.shuffle()

TypeError: object of type 'NoneType' has no len()

[anujapinge]

Hello, I am trying to perform transfer learning on the yolov3. Why it is recommended to have 200 images of each object to be trained? And also can I include as many number of classes as I want?