Onnxruntime: Optimizing BART: encoder/decoder attention

[EricMichaelSmith]

Is your feature request related to a problem? Please describe.
I'd like to optimize BART with ONNX Runtime, but it looks like the only Attention operator currently supported is self-attention, and BART requires encoder/decoder cross-attention.

System information

• ONNX Runtime version (you are using): 1.4.0

Describe the solution you'd like
A fused operator implementing encoder/decoder cross-attention

Describe alternatives you've considered

• In the meantime I'm planning to check out APEX's Fast Multihead Attention.

[tianleiwu]

It is in the backlog, there is no ETA currently. @wangyems.

[wangyems]

It is in the backlog, there is no ETA currently. @wangyems.

We could target this in ORT 1.6 release @tianleiwu