Coremltools: Huggingface's DistilBERT not convertible with 4.0b2

[nneubauer]

🐞Describe the bug

I just tried converting Huggingface's DistilBERT using the direct conversion API but it fails with "ValueError: node 110 (ones) got 5 input(s), expected 6".

Trace

WARNING:root:Tuple detected at graph output. This will be flattened in the converted model.
Converting Frontend ==> MIL Ops:   0%|          | 0/632 [00:00<?, ? ops/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-654ad5181c19> in <module>()
     18 converted_model = ct.convert(
     19     traced_model,
---> 20     inputs=[ct.TensorType(shape=example_input.shape)]
     21 )

8 frames
/usr/local/lib/python3.6/dist-packages/coremltools/converters/mil/frontend/torch/ops.py in _get_inputs(context, node, expected)
    124         raise ValueError(
    125             "node {} ({}) got {} input(s), expected {}".format(
--> 126                 node.name, node.kind, len(inputs), expected
    127             )
    128         )

ValueError: node 110 (ones) got 5 input(s), expected 6

To Reproduce

Minimal example which can be used within a Google colab.

!pip install transformers scikit-learn==0.19.2 torch==1.5.1 coremltools==4.0b2

import torch

from transformers.modeling_distilbert import DistilBertModel
from transformers.tokenization_distilbert import DistilBertTokenizer

import coremltools as ct

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')
model = DistilBertModel.from_pretrained('distilbert-base-multilingual-cased', torchscript=True)
model.eval()

example_input = torch.tensor(tokenizer.encode("So long and thanks for all the fish!")).unsqueeze(0)

traced_model = torch.jit.trace(model, example_input)

converted_model = ct.convert(
    traced_model,
    inputs=[ct.TensorType(shape=example_input.shape)]
)

System environment (please complete the following information):

• coremltools version 4.0b2
• Within Google Colab

[leovinus2001]

Couple of things going on here
1) With the few lines of extra logging in issue #818 we get this

WARNING:root:Tuple detected at graph output. This will be flattened in the converted model.
Converting Frontend ==> MIL Ops: 0%| | 0/632 [00:00 Converting op name = 102 : kind = size : shape = (1, 11) inputs = ['input_ids', '101'] attr = {'value': None}
Converting op name = 103 : kind = constant : shape = n/a inputs = [] attr = {'value': 1}
Converting op name = 104 : kind = size : shape = (1, 11) inputs = ['input_ids', '103'] attr = {'value': None}
Converting op name = 105 : kind = listconstruct : shape = () inputs = ['102', '104'] attr = {'value': None}
Converting op name = 106 : kind = constant : shape = n/a inputs = [] attr = {'value': 6}
Converting op name = 107 : kind = constant : shape = n/a inputs = [] attr = {'value': 0}
Converting op name = 108 : kind = constant : shape = n/a inputs = [] attr = {'value': 'cpu'}
Converting op name = 109 : kind = constant : shape = n/a inputs = [] attr = {'value': False}
Converting op name = 110 : kind = ones : shape = (2,) inputs = ['105', '106', '107', '108', '109'] attr = {'value': None}

In other words, the converter fails in the function "ones" in
coremltools/converters/mil/frontend/torch/ops.py
because it sees 5 arguments instead of 6 for "ones".

The code for "def ones(context, node):" describes which arguments are necessary and the missing sixth is marked as unused. Hence, an extra code path that uses
inputs = _get_inputs(context, node, expected=5)
instead of 6 gets you past that point.

2) Then we get to

Converting op name = 111 : kind = constant : shape = n/a inputs = [] attr = {'value': 1}
Converting op name = 112 : kind = size : shape = (1, 11) inputs = ['input_ids', '111'] attr = {'value': None}
Converting op name = 113 : kind = constant : shape = n/a inputs = [] attr = {'value': 4}
Converting op name = 114 : kind = constant : shape = n/a inputs = [] attr = {'value': 0}
Converting op name = 115 : kind = constant : shape = n/a inputs = [] attr = {'value': 'cpu'}
Converting op name = 116 : kind = constant : shape = n/a inputs = [] attr = {'value': False}
Converting op name = position_ids : kind = arange : shape = () inputs = ['112', '113', '114', '115', '116'] attr = {'value': None}
Converting op name = 118 : kind = constant : shape = n/a inputs = [] attr = {'value': 0}
Converting op name = 119 : kind = unsqueeze : shape = (11,) inputs = ['position_ids', '118'] attr = {'value': None}
Converting op name = input.1 : kind = expand_as : shape = (1, 11) inputs = ['119', 'input_ids'] attr = {'value': None}
Converting op name = 121 : kind = constant : shape = n/a inputs = [] attr = {'value': 0}
Converting op name = 122 : kind = constant : shape = n/a inputs = [] attr = {'value': False}
Converting op name = 123 : kind = constant : shape = n/a inputs = [] attr = {'value': False}
Converting op name = word_embeddings : kind = embedding : shape = (119547, 768) inputs = ['1', 'input_ids', '121', '122', '123'] attr = {'value': None}
WARNING:root:CoreML embedding (gather) layer does not support any inputs besides the weights and indices. Those given will be ignored.

File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 835, in embedding
gather = mb.gather(x=_input, indices=indices, name=node.name)
File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
return cls._add_op(op_cls, kwargs)
File "/Users/hans-web/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/builder.py", line 188, in _add_op
new_op = op_cls(kwargs)
File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/scatter_gather.py", line 59, in __init__
super(gather, self).__init__(kwargs)
File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 148, in __init__
self._validate_and_set_inputs(kwargs)
File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 360, in _validate_and_set_inputs
parsed_inputs = self.input_spec.parse_inputs(kwargs)
File "Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/input_type.py", line 67, in parse_inputs
raise TypeError(msg)
TypeError: Input indices has type .tensor'> not compatible with expected type IntOrIntTensorInputType

i.e. an issue in converting word_embeddings but I do not know what that issue is

[daiwei89]

I've created a ticket internally and we'll address this in the coming releases

[jbmaxwell]

I'm seeing a similar error—node attention_mask.1 (ones) got 5 input(s), expected 6—when trying to convert HuggingFace's RoBERTa. Any updates on this conversion?
I'm running torch 1.6.0, transformers 3.2.0, and coremltools 4.0b3

[1duo]

Thanks for reporting this issue, this has been fixed in the coremltools==4.0b4 release. Feel free to re-open or create another issue if you're still experiencing issues. Thanks!