Onnxruntime: HardMax/LpNormalization fail with negative axis

[fdwr]

Describe the bug
HardMax fails with negative axis. It should call HandleNegativeAxis to resolve the value into an absolute positive axis.

HRESULT=0x80004005 message=Exception during initialization: S:\WindowsAI\engine\lotus\onnxruntime\core/providers/cpu/math/hardmax.h:24 onnxruntime::Hardmax<float>::Hardmax axis_ >= 0 was false. Invalid axis provided.

https://github.com/microsoft/onnxruntime/blob/d9fa632863698c083a06004c5de8ea216ba35c7b/onnxruntime/core/providers/cpu/math/hardmax.h#L24

From ONNX spec: "Accepted range in [-r, r-1] where r = rank(input)."

https://github.com/onnx/onnx/blob/master/docs/Operators.md#Hardmax

Urgency
Preferably before Vibranium release.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 2019-09-09
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version: 0.5
• Python version: NA
• Visual Studio version (if applicable): 2017 15.8.5
• GCC/Compiler version (if compiling from source): NA
• CUDA/cuDNN version: NA
• GPU model and memory: NA

To Reproduce
Pass negative axis value. e.g.

// 2D array, axis 0 via negative axis.
{
  "op_type": "Hardmax",
  "axis": -2,
  "input":  [[0, 1, 2, -2, 2],
             [0, 1, 2, -1, 3],
             [0, 0, 3,  0, 0]],
  "output": [[0, 0, 0, 0, 0],
             [0, 0, 0, 0, 1],
             [0, 0, 0, 0, 0]],
  "T": "float32"
},

Expected behavior
HardMax should call HandleNegativeAxis to resolve the value into an absolute positive axis. Note GPU operator passes this.

👍 ArgMax
👍 ArgMin
👍 Concat
👍 Flatten
👍 Gather
✖ Hardmax
👍 LogSoftmax
✖ LpNormalization
👍 OneHot
👍 ReduceL1
👍 ReduceL2
👍 ReduceLogSum
👍 ReduceLogSumExp
👍 ReduceMax
👍 ReduceMean
👍 ReduceMin
👍 ReduceProd
👍 ReduceSum
👍 ReduceSumSquare
👍 Scatter
👍 Slice
👍 Softmax
👍 Split
👍 TopK

Additional context
Found while testing WinML ONNX conformance GPU vs CPU.

[hariharans29]

Hi @fdwr,

Actually the opset-10 HardMax spec is unclear - https://github.com/onnx/onnx/blob/rel-1.5.0/docs/Operators.md#Hardmax. The line you are referring to is from current master which means opset-11 HardMax needs to support it.

Having said that, it does seem reasonable to support negative axis for HardMax-10 as well.

[fdwr]

@hariharans29 : True, that clarification was added recently. Though it's always been assumed, even when unstated, that operators with an axis attribute should resolve negative values (and for consistency, Softmax works in ORT, so HardMax would be expected to for consistency).

I tested the following for negative axes and found HardMax was the only one missing the check: ArgMin, ArgMax, Concat, Gather, Hardmax, LogSoftmax, ReduceL1, ReduceL2, ReduceLogSum, ReduceLogSumExp, ReduceMax, ReduceMean, ReduceMin, ReduceProd, ReduceSum, ReduceSumSquare, Scatter, Slice, Softmax, Split, TopK

[hariharans29]

@fdwr - makes sense to add it then :). Will do it.

[fdwr]

@hariharans29 : Found one more (after double checking and sweeping through all of Operators.md for anything with an axis attribute) after adding LpNormalization, OneHot, Flatten.

LpNormalization accepts -1 but rejects -2.

https://github.com/microsoft/onnxruntime/blob/2b8677b2102b47a75c85422d4e2db1385b5e49be/onnxruntime/core/providers/cpu/nn/lp_norm.cc#L58

[hariharans29]

Fixed via #1835