Nmt: After training, How I do I generate inferences with specific beam width?

[ssokhey]

I trained with beam_width=0 (which is Greedy Decoding). When I try to generate inferences using:

python -m nmt.nmt \
    --out_dir=/tmp/nmt_model \
    --inference_input_file=/tmp/my_infer_file.vi \
    --inference_output_file=/tmp/nmt_model/output_infer

It obviously loads the saved hyperparameters and decodes using beam_width=0. However, I would like to generate inferences from my trained model with (say) beam_width=10. I tried with setting --override_loaded_hparams=True, but it ends up in the following error:

NotFoundError (see above for traceback): Key dynamic_seq2seq/decoder/multi_rnn_cell/cell_0/basic_lstm_cell/bias not found in checkpoint
     [[Node: save/RestoreV2_1 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_1/tensor_names, save/RestoreV2_1/shape_and_slices)]]
     [[Node: save/RestoreV2_10/_19 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_56_save/RestoreV2_10", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

I would like to know a way out.

[oahziur]

@ssokhey

--override_loaded_hparams=True also overrides other configurations such as num_units, num_layers etc. You will need to use your original training command plus the inference arguments in order to override loaded hparams but also keep your trained model's configuration.

For example:

python -m nmt.nmt \
    ...\ # (all your training flags)
    --inference_input_file=/tmp/my_infer_file.vi \
    --inference_output_file=/tmp/nmt_model/output_infer \
    --beam_width=10 \
    --override_loaded_hparams=True

[ssokhey]

@oahziur Thanks for the prompt reply. It is working. I have just generated instances with your suggestions.

I just wanted to ask if there is any other cleaner way to do the same thing. As it will require copy-pasting the whole thing again and again.

[oahziur]

@ssokhey

You could try to use one of the standard_hparams, or define something similar.

You can find an example usage here.

[ssokhey]

@oahziur
These I have already seen. Anyways thank you very much for your time. Really appreciate that.