Detectron: Providing a Proposal File

[chrisby]

Hi there,

I was able to train a mask RCNN on my data set but run into problems when I run

python2 tool/test_net.py --cfg config/12_2017_baselines/MY_VERSION_OF_mask_rcnn_R-50-C4_sx.yaml

The error I get is

cfg.TEST.PROPOSAL_FILE = cfg.TEST.PROPOSAL_FILES[i] IndexError: tuple index out of range

I assume this arises because I did not specify a proposal file. My question is: in which format can I provide a proposal file for a private dataset?

Thanks,
Christian

[ir413]

Hi @chrisby, we expect the proposals for a dataset to be provided in a pkl file that contains a serialized proposals dictionary with the following entries:

proposals.keys() - ['cfg', 'ids', 'scores', 'boxes']

where

proposals['cfg'] - config used to compute the proposals
proposals['ids][i] - id for the i-th image
proposals['scores'][i] - ndarray (N_i,) containing scores for the proposals in the i-th image
proposals['boxes'][i]  - ndarray (N_i, 4) containing coordinates (x1, y1, x2, y2) for the proposals in the i-th image

It may be instructive to download one of the proposal files from the Model Zoo and inspect it interactively. You may also find it useful to look at _add_proposals_from_file to see how proposals get loaded from file and rpn_generator to see how RPN proposals get computed and stored to a pkl file.

If you require proposals for COCO classes, you can use one of the RPN models from the Model Zoo to compute (and store in the expected format) the proposals for your dataset.

We plan to add the information about the proposals file format to the documentation in the future.
In the meantime, let us know if the above is unclear.

[chrisby]

Thanks, that's already very helpful! I think I will use one of the baseline RPNs to compute the proposal file an subsequently link it to my dataset.

[CoinCheung]

Excuseme, but what is the use of the file 'cfg' ? If I use the proposals that I extract from the training set by hand instead of some RPN, what would the structure of the proposal file be like, do I still need other filed other than the 'boxes' ? Would it be possible that I simply use naive fast rcnn combined with my self-made proposals with detectron ?

[CoinCheung]

Excuseme, but what is the use of the file 'cfg' ? If I use the proposals that I extract from the training set by hand instead of some RPN, what would the structure of the proposal file be like, do I still need other filed other than the 'boxes' ? Would it be possible that I simply use naive fast rcnn combined with my self-made proposals with detectron ?

[jason718]

proposals['cfg'] seems useless if you want to use your own proposals. In Detectron/tools/convert_selective_search.py they are not using 'cfg' keys.

Is this correct @ir413 ?

[ir413]

@CoinCheung, @jason718: proposals['cfg'] is not required so using a proposal file without it (e.g. as produced by convert_selective_search) should still work. I think that having the config file embedded in the proposals file is still useful for reproducibility as it records the hyper-parameters that were used to compute the proposals.

[tengerye]

Hi, do I need to use external selective search codes? I could not find any in this project.

[tengerye]

@ir413 Hi, I want to inference my own images using selective search. I have the proposals in .mat format. But when I try to convert it to .pkl file using convert_selective_search, I got an error.

My command is:
python Detectron/tools/convert_selective_search.py inference ./inference.mat ./inference.pkl.

The error is:

Traceback (most recent call last):
  File "Detectron/tools/convert_selective_search.py", line 40, in <module>
    ds = JsonDataset(dataset_name)
  File "/home/tenger/Detectron/detectron/datasets/json_dataset.py", line 56, in __init__
    'Unknown dataset name: {}'.format(name)
AssertionError: Unknown dataset name: inference

Does it mean I can't inference with my own image?

[tengerye]

@chrisby @CoinCheung Hi, have you solved your problem? May I know how did you feed the proposals to the infer_simple.py please?

[ir413]

@ir413 Hi, I want to inference my own images using selective search. I have the proposals in .mat format. But when I try to convert it to .pkl file using convert_selective_search, I got an error.

My command is:
python Detectron/tools/convert_selective_search.py inference ./inference.mat ./inference.pkl.

The error is:

Traceback (most recent call last):
  File "Detectron/tools/convert_selective_search.py", line 40, in <module>
    ds = JsonDataset(dataset_name)
  File "/home/tenger/Detectron/detectron/datasets/json_dataset.py", line 56, in __init__
    'Unknown dataset name: {}'.format(name)
AssertionError: Unknown dataset name: inference

Does it mean I can't inference with my own image?

@tengerye: The tool expects proposals for one of the available datasets and the first argument to the tool should be a valid json dataset name (e.g. see here for all available options).