I am trying to train a model for object detection of a class that does not exist in standard datasets, so I created my own, labeled it and converted the whole thing to TF Record.
However, my different models (based on config files from the sample models in /samples/configs) return many false positives, and I believe it would be best to enrich my dataset with standard ones (such as COCO).
To do that, it'd be ideal if there was a function such as tf.add_data(old_tf_record, new_tf_record) in order to combine both and result a TF Record with additional classes.
This is related to #1809, where @humayun wants to add new classes to COCO.
LaurentBerder
All 9 comments
Instead of combining TF Records together, I'm doing it before converting both my dataset and the COCO dataset from JSON to TF Records.
I made a python program to do so, see below.
import json
import os
import argparse
from PIL import Image
def add_info_to_coco_json(args):
with open(args.COCO_annotation_file, 'r') as f:
coco = json.load(f)
# Check the highest id numbers for images and annotations, we'll work incrementally from there
im_ids = max([im['id'] for im in coco['images']])
an_ids = max([im['id'] for im in coco['annotations']])
# Add info to 'categories'
cats = {'new_category_1': 91, 'existing_category_1': 8, 'new_category_2': 92}
coco['categories'].append({'id': 91, 'name': 'new_category_1', 'supercategory': 'outdoor'})
# coco['categories'].append({'id': 8, 'name': 'existing_category_1', 'supercategory': 'vehicle'})
coco['categories'].append({'id': 92, 'name': 'new_category_2', 'supercategory': 'vehicle'})
)
# Loop over custom files
for file in os.listdir(args.custom_annotation_folder):
if '__labels.json' in file:
with open(args.custom_annotation_folder + file, 'r') as f:
img = json.load(f)
im_ids += 1
filename = img['image_filename']
print(filename)
with Image.open(args.custom_annotation_folder + filename) as im_file:
width, height = im_file.size # Image height and width
# Add info to 'images'
new_image = {'id': im_ids, 'filename': filename, 'width': width, 'height': height}
coco['images'].append(new_image)
# Add info to 'annotations'
for label in img['labels']:
an_ids += 1
xmin = (label['centre']['x'] - (label['size']['x'] / 2)) / width
ymin = (label['centre']['y'] - (label['size']['y'] / 2)) / height
new_annotation = {'id': an_ids, 'category_id': cats.get(label['label_class']),
'bbox': [xmin, ymin, width, height], 'image_id': im_ids}
coco['annotations'].append(new_annotation)
return(coco)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--COCO_annotation_file', help='The COCO annotation file downloaded from COCO website')
parser.add_argument('--custom_annotation_folder', help='The folder in which json files are saved')
parser.add_argument('--output_json', help='The new combined annotation file to save')
args = parser.parse_args()
new_coco = add_info_to_coco_json(args)
with open(args.output_json, 'w') as output:
json.dump(new_coco, output)
if __name__ == '__main__':
main()
However, it is of course highly customized to the format I had labeled my images in (based on this labeling tool) which is not so common...
LaurentBerder
on 8 Mar 2018
Why not just use both .record files and shuffle them? You can add a list of tf records like this:
tf_record_input_reader {
input_path: ['path_to_file_1', 'path_to_file_2']
}
nguyeho7
on 9 Mar 2018
👍3
@nguyeho7 that simple?!
Thanks a lot: that's a lot of effort taken away from me!
However, when I tried that, it returned this Traceback:
Traceback (most recent call last):
File "object_detection/train.py", line 167, in <module>
tf.app.run()
File "/opt/rh/rh-python35/root/usr/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 124, in run
_sys.exit(main(argv))
File "object_detection/train.py", line 92, in main
FLAGS.pipeline_config_path)
File "/data/science/gpu/Reconnaissance_Container/models/research/object_detection/utils/config_util.py", line 88, in get_configs_from_pipeline_file
text_format.Merge(proto_str, pipeline_config)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 533, in Merge
descriptor_pool=descriptor_pool)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 587, in MergeLines
return parser.MergeLines(lines, message)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 620, in MergeLines
self._ParseOrMerge(lines, message)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 635, in _ParseOrMerge
self._MergeField(tokenizer, message)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 735, in _MergeField
merger(tokenizer, message, field)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 823, in _MergeMessageField
self._MergeField(tokenizer, sub_message)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 735, in _MergeField
merger(tokenizer, message, field)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 823, in _MergeMessageField
self._MergeField(tokenizer, sub_message)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 735, in _MergeField
merger(tokenizer, message, field)
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 874, in _MergeScalarField
value = tokenizer.ConsumeString()
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 1237, in ConsumeString
the_bytes = self.ConsumeByteString()
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 1252, in ConsumeByteString
the_list = [self._ConsumeSingleByteString()]
File "/opt/rh/rh-python35/root/usr/lib64/python3.5/site-packages/google/protobuf/text_format.py", line 1271, in _ConsumeSingleByteString
raise self.ParseError('Expected string but found: %r' % (text,))
google.protobuf.text_format.ParseError: 173:17 : Expected string but found: '{'
You should probably move this to stackoverflow and close the issue, but did you edit the pipeline config file correctly? Post a snipped there.
nguyeho7
on 9 Mar 2018
Here is the content of the pipeline config:
# SSD with Inception v2 configuration for MSCOCO Dataset.
# Users should configure the fine_tune_checkpoint field in the train config as
# well as the label_map_path and input_path fields in the train_input_reader and
# eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
# should be configured.
model {
ssd {
num_classes: 92
box_coder {
faster_rcnn_box_coder {
y_scale: 10.0
x_scale: 10.0
height_scale: 5.0
width_scale: 5.0
}
}
matcher {
argmax_matcher {
matched_threshold: 0.5
unmatched_threshold: 0.5
ignore_thresholds: false
negatives_lower_than_unmatched: true
force_match_for_each_row: true
}
}
similarity_calculator {
iou_similarity {
}
}
anchor_generator {
ssd_anchor_generator {
num_layers: 6
min_scale: 0.2
max_scale: 0.95
aspect_ratios: 1.0
aspect_ratios: 2.0
aspect_ratios: 0.5
aspect_ratios: 3.0
aspect_ratios: 0.3333
reduce_boxes_in_lowest_layer: true
}
}
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
box_predictor {
convolutional_box_predictor {
min_depth: 0
max_depth: 0
num_layers_before_predictor: 0
use_dropout: true
dropout_keep_probability: 0.8
kernel_size: 3
box_code_size: 4
apply_sigmoid_to_scores: false
conv_hyperparams {
activation: RELU_6,
regularizer {
l2_regularizer {
weight: 0.00004
}
}
initializer {
truncated_normal_initializer {
stddev: 0.03
mean: 0.001
}
}
}
}
}
feature_extractor {
type: 'ssd_inception_v2'
min_depth: 16
depth_multiplier: 1.0
conv_hyperparams {
activation: RELU_6,
regularizer {
l2_regularizer {
weight: 0.00004
}
}
initializer {
truncated_normal_initializer {
stddev: 0.03
mean: 0.001
}
}
batch_norm {
train: true,
scale: true,
center: true,
decay: 0.9997,
epsilon: 0.001,
}
}
}
loss {
classification_loss {
weighted_sigmoid {
}
}
localization_loss {
weighted_smooth_l1 {
}
}
hard_example_miner {
num_hard_examples: 3000
iou_threshold: 0.99
loss_type: CLASSIFICATION
max_negatives_per_positive: 3
min_negatives_per_image: 0
}
classification_weight: 1.0
localization_weight: 1.0
}
normalize_loss_by_num_matches: true
post_processing {
batch_non_max_suppression {
score_threshold: 1e-8
iou_threshold: 0.6
max_detections_per_class: 100
max_total_detections: 100
}
score_converter: SIGMOID
}
}
}
train_config: {
batch_size: 24
optimizer {
rms_prop_optimizer: {
learning_rate: {
exponential_decay_learning_rate {
initial_learning_rate: 0.004
decay_steps: 800720
decay_factor: 0.95
}
}
momentum_optimizer_value: 0.9
decay: 0.9
epsilon: 1.0
}
}
batch_queue_capacity: 2
prefetch_queue_capacity: 2
fine_tune_checkpoint: "/data/science/gpu/Reconnaissance_Container/Tensorflow/frozen_model/ssd_inception_v2_coco_2017_11_17/model.ckpt"
from_detection_checkpoint: true
# Note: The below line limits the training process to 200K steps, which we
# empirically found to be sufficient enough to train the pets dataset. This
# effectively bypasses the learning rate schedule (the learning rate will
# never decay). Remove the below line to train indefinitely.
num_steps: 100000
data_augmentation_options {
random_horizontal_flip {
}
}
data_augmentation_options {
ssd_random_crop {
}
}
}
train_input_reader: {
tf_record_input_reader {
input_path: {"/data/science/gpu/Reconnaissance_Container/Tensorflow/Train/TF_Record_containers",
"/tools/COCO/tf_record/coco_train.record"]
}
label_map_path: "/data/science/gpu/Reconnaissance_Container/Tensorflow/Train/coco_container_label_map.pbtxt"
}
eval_config: {
num_examples: 8000
# Note: The below line limits the evaluation process to 10 evaluations.
# Remove the below line to evaluate indefinitely.
max_evals: 1000
}
eval_input_reader: {
tf_record_input_reader {
input_path: ["/data/science/gpu/Reconnaissance_Container/Tensorflow/Eval/TF_Record_containers",
"/tools/COCO/tf_record/coco_val.record"]
}
label_map_path: "/data/science/gpu/Reconnaissance_Container/Tensorflow/Train/coco_container_label_map.pbtxt"
shuffle: true
num_readers: 1
}
train_input_reader: {
tf_record_input_reader {
input_path: {"/data/science/gpu/Reconnaissance_Container/Tensorflow/Train/TF_Record_containers", "/tools/COCO/tf_record/coco_train.record"]
}
{ should be [
input_path:["/data/science/gpu/Reconnaissance_Container/Tensorflow/Train/TF_Record_containers", "/tools/COCO/tf_record/coco_train.record"]
@LaurentBerder I just wondering if you got any luck with your fine tuning using the suggested approach or please advice as I'm running the same problem
Thanks
cyberphantom
on 9 Apr 2018
👍1
@cyberphantom No, I didn't. I ended up not using Tensorflow because it is too cumbersome, and had a working project in ChainerCV. You can have a look at my repo, I made a script to use COCO in the file format needed for that project and combine with my custom-labeled data.
LaurentBerder
on 10 Apr 2018
👍4
Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.
tensorflowbutler
on 30 Jan 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
XavDCtpz
路
3Comments
chenyuZha
路
3Comments
jacknlliu
路
3Comments
nmfisher
路
3Comments
kamal4493
路
3Comments
Most helpful comment
@cyberphantom No, I didn't. I ended up not using Tensorflow because it is too cumbersome, and had a working project in ChainerCV. You can have a look at my repo, I made a script to use COCO in the file format needed for that project and combine with my custom-labeled data.