Caffe: Replace `input` fields with a proper layer

[shelhamer]

The input fields cause a few headaches like

(1) They are not a proper layer so they cannot be included in all-in-one nets by stage or other NetFilter rules
(2) They are nothing but bare blobs so input preprocessing cannot be configured or done automatically
(3) As a result of (2), the Python and MATLAB interfaces duplicate a lot of code and lead to brittle transitions between train / val / data layers and the input fields for deployment.

so there's a plan to do away with them. I'll post an issue with the fully story when I can, but the core idea is to replace these fields with MemoryDataLayer for deployment, make the tools understand stages so we can mark the layer as the "deploy" input, and then re-define all the reference deploy models and finally deprecate the pycaffe and matcaffe preprocessing code.

The deploy.prototxt could go away completely -- just like train and val were merged into train_val.prototxt there will be one net definition to rule them all that includes the layers and rules for making the train / val / deploy / whatever variations of the net. Alternatively one can stamp out each variation through Python net specification.

(1) is the top priority while (2) and (3) might be nice but leaving preprocessing to the interface languages is alright.

[lukeyeager]

Can't wait for this. Here's my 2 cents:

While you're at it, why not move to the machine learning naming convention of train-val-test (see ImageNet)? It's confusing that the train_val network uses phases TRAIN and TEST for my training and validation datasets, and where did "deploy" come from? Why not use this:

enum Phase {
    TRAIN = 0;
    VAL = 1; // Was TEST
    TEST = 2; // Instead of deploy
}

And then rename all the test_net stuff to val_net, etc. I get why it would make sense to keep the naming convention for supporting legacy networks and models, but why was this naming convention chosen in the first place?

[StevenLOL]

Yes,we need this.

[shelhamer]

@lukeyeager the TRAIN and TEST phases are really about learning and inference respectively and not a particular collection of data points. There are layers that operate differently during learning and inference -- like dropout -- so Caffe distinguishes between these phases of operation.

There are all kinds of data splits that can be useful in practice, but we leave that up to the selection of inputs given to a Caffe model (as an lmdb, in memory, or what have you).

[antran89]

I totally support the idea:

The deploy.prototxt will go away completely.

I feel that deploying a trained model on a lmdb test data right now is a bit unnatural. I have to treat them like validation data and call caffe test with iterations equal to #test items / batch_size.

[shelhamer]

The switch to InputLayer instead of input fields is done in #3211. However adding preprocessing and de-duping the pycaffe + matcaffe interfaces is left as a follow-up.

[lukeyeager]

Is this done now?

1. Solved by https://github.com/BVLC/caffe/pull/3211
2. Not really required, IMHO
3. Solved by https://github.com/BVLC/caffe/issues/3863 (but not for MATLAB)

[shelhamer]

Closing to declare victory on this. Making it a layer type was most of the battle.