Incubator-mxnet: Loading pretrained model raise error

[rorschachhb]

Hi,

I have a CNN model converted from caffe using convert_model.py. When I try to load it(fllowing steps in this tutorial), MXNet raises the following error:

RuntimeError: layer21_label is not presented

The last layer of my CNN is softmax, and part of my symbol.json file looks like this:

    {                                                                           
      "op": "FullyConnected",                                                   
      "param": {                                                                
        "no_bias": "False",                                                     
        "num_hidden": "250"                                                     
      },                                                                        
      "name": "layer20",                                                        
      "inputs": [[32, 0], [33, 0], [34, 0]],                                    
      "backward_source_id": -1                                                  
    },                                                                          
    {                                                                           
      "op": "null",                                                             
      "param": {},                                                              
      "name": "layer21_label",                                                  
      "inputs": [],                                                             
      "backward_source_id": -1                                                  
    },                                                                          
    {                                                                           
      "op": "SoftmaxOutput",                                                    
      "param": {                                                                
        "grad_scale": "1",                                                      
        "ignore_label": "-1",                                                   
        "multi_output": "False",                                                
        "normalization": "null",                                                
        "use_ignore": "False"                                                   
      },                                                                        
      "name": "layer21",                                                        
      "inputs": [[35, 0], [36, 0]],                                             
      "backward_source_id": -1                                                  
    }

I compared my symbol.json file to resnet and inception, couldn't find the bug. Does anyone know how to fix this? Thanks!

[piiswrong]

Could you paste the full error? What's the stack trace?

[rorschachhb]

The full error is as follows:

>>> sym, arg_params, aux_params = mx.model.load_checkpoint('sketchanet_fromcaffe', 1)
>>> mod = mx.mod.Module(symbol=sym, context=mx.gpu())
>>> mod.bind(for_training=False, data_shapes=[('data', (1,1,225,225))])
>>> mod.set_params(arg_params, aux_params)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/base_module.py", line 478, in set_params
    allow_missing=allow_missing, force_init=force_init)
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/module.py", line 190, in init_params
    _impl(name, arr, arg_params)
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/module.py", line 183, in _impl
    raise RuntimeError("%s is not presented" % name)
RuntimeError: layer21_label is not presented

[rorschachhb]

I tried to convert vgg16 caffemodel into an mxnet model, and load it in mxnet, the same error is raised:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/base_module.py", line 478, in set_params
    allow_missing=allow_missing, force_init=force_init)
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/module.py", line 190, in init_params
    _impl(name, arr, arg_params)
  File "/home/bo/venv/lib/python2.7/site-packages/mxnet-0.7.0-py2.7.egg/mxnet/module/module.py", line 183, in _impl
    raise RuntimeError("%s is not presented" % name)
RuntimeError: prob_label is not presented

Seems like there are some incompatibilities between MXNet modules? @piiswrong

[rorschachhb]

Update:

I can load the CNN model using mx.model.FeedForward.load(), but the class FeedForward seems to be designed for training only, and doesn't have a forward interface like mx.mod.Module.

What's the relationship between mx.model and mx.module? And how can I convert mx.model.FeedForward object into mx.module.Module? @piiswrong

[rorschachhb]

I finally bypassed the problem by adding a line

arg_params['layer21_label'] = mx.nd.array([0])

before

mod.set_params(arg_params, aux_params)

It turns out that there's a key named 'layer21_label' in mod._exec_group.execs[0].arg_dict, but not in arg_params. Though I'm not sure why we need label in a forward pass.

Hope it get fixed soon enough.

[burness]

It works for me, Thanks

[ysh329]

It also works for me. Thanks a lot!

[anshumankmr]

@rorschachhb thanks a lot mate