Transformers: Converting to ONNX doesn't apply to all models

[alvations]

Is it possible to list or highlight which models are ONNX convertible?

On a fresh python environment:

python -m pip install -U transformers
python -m  pip install mosestokenizer

The transformer version on the machine is 2.11.0

When trying to convert the https://huggingface.co/transformers/model_doc/marian.html to ONNX, it throws the following error:

$ python convert_graph_to_onnx.py --framework pt --model Helsinki-NLP/opus-mt-en-ROMANCE opus-mt-en-romance.onnx

Neither PyTorch nor TensorFlow >= 2.0 have been found. Models won't be available and only tokenizers, configurationand file/data utilities can be used.
ONNX opset version set to: 11

Loading pipeline (model: Helsinki-NLP/opus-mt-en-ROMANCE, tokenizer: Helsinki-NLP/opus-mt-en-ROMANCE)
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/tokenization_utils.py:828: FutureWarning: Parameter max_len is deprecated and will be removed in a future release. Use model_max_length instead.
  warnings.warn(
stdbuf was not found; communication with perl may hang due to stdio buffering.
Error while converting the model: 'NoneType' object has no attribute 'from_pretrained'

Then after installing pytorch

$ python -m pip install -U pytorch

$ python convert_graph_to_onnx.py --framework pt --model Helsinki-NLP/opus-mt-en-ROMANCE opus-mt-en-romance.onnx

ONNX opset version set to: 11
Loading pipeline (model: Helsinki-NLP/opus-mt-en-ROMANCE, tokenizer: Helsinki-NLP/opus-mt-en-ROMANCE)
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/tokenization_utils.py:828: FutureWarning: Parameter max_len is deprecated and will be removed in a future release. Use model_max_length instead.
  warnings.warn(
stdbuf was not found; communication with perl may hang due to stdio buffering.

Downloading: 100%|██████████████████████████████| 312M/312M [01:59<00:00, 2.61MB/s]

Error while converting the model: Folder /Users/username/git-stuff/transformers/src/transformers is not empty, aborting conversion

Then after creating a new directory:

$ python ../convert_graph_to_onnx.py --framework pt --model Helsinki-NLP/opus-mt-en-ROMANCE opus-mt-en-romance.onnx

ONNX opset version set to: 11
Loading pipeline (model: Helsinki-NLP/opus-mt-en-ROMANCE, tokenizer: Helsinki-NLP/opus-mt-en-ROMANCE)

/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/tokenization_utils.py:828: FutureWarning: Parameter max_len is deprecated and will be removed in a future release. Use model_max_length instead.
  warnings.warn(
stdbuf was not found; communication with perl may hang due to stdio buffering.

Using framework PyTorch: 1.5.0
Found input input_ids with shape: {0: 'batch', 1: 'sequence'}
Found input attention_mask with shape: {0: 'batch', 1: 'sequence'}
Found output output_0 with shape: {0: 'batch', 1: 'sequence'}
Found output output_1 with shape: {0: 'batch', 1: 'sequence'}
Ensuring inputs are in correct order
decoder_input_ids is not present in the generated input list.
Generated inputs order: ['input_ids', 'attention_mask']
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:173: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not padding_mask.any():
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:590: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert embed_dim == self.embed_dim
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:591: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert list(query.size()) == [tgt_len, bsz, embed_dim]
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:633: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert attn_weights.size() == (bsz * self.num_heads, tgt_len, src_len)
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:642: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert key_padding_mask is None or key_padding_mask.size()[:2] == (bsz, src_len,)
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/transformers/modeling_bart.py:654: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert attn_output.size() == (bsz * self.num_heads, tgt_len, self.head_dim)
/Users/username/.pyenv/versions/3.8.0/lib/python3.8/site-packages/torch/onnx/utils.py:736: UserWarning: ONNX export failed on ATen operator triu because torch.onnx.symbolic_opset11.triu does not exist
  warnings.warn("ONNX export failed on ATen operator {} because "
Error while converting the model: Exporting the operator triu to ONNX opset version 11 is not supported. Please open a bug to request ONNX export support for the missing operator.

It is understandable that ONNX might not be supporting all models that are available through Huggingface's transformers.

But is there a way to know which model(s) are ONNX convertible and which aren't?

[tianleiwu]

Here is a related PyTorch issue: https://github.com/pytorch/pytorch/issues/32968.

Here is a list of pretraineds model that can be exported to ONNX:
bert-base-cased, distilbert-base-uncased, roberta-base, gpt2, distilgpt2, openai-gpt, albert-base-v2, xlnet-base-cased

So far, the following models have problem in exporting to ONNX:
bart-large, transfo-xl-wt103, t5-base, xlm-mlm-en-2048

[HenryDashwood]

Anyone had any luck resolving this issue? The Pytorch issue linked above has a couple of potential workarounds. I notice it affects Pegasus as well

[HenryDashwood]

This pull request was closed by its author but doesn't raise the issue when I run convert on a Bart model. https://github.com/huggingface/transformers/pull/6334