Coremltools: PyTorch to CoreML converted BatchNorm1D model fails in inference/prediction with RuntimeError: { NSLocalizedDescription = "Error computing NN outputs.";

[leovinus2001]

Relevance:-----------------------------------------------------------

While the conversion and inference/prediction on BatchNorm2D from the CoreML model is fine, when we use the BatchNorm1D instead the prediction fails with

RuntimeError: {
NSLocalizedDescription = "Error computing NN outputs.";
}

Unless I made some other mistake (always possible :), this makes it currently impossible to do inference on a BatchNorm1D in a converted PyTorch to CoreML model. At least in my setups.

https://pytorch.org/docs/master/generated/torch.nn.BatchNorm1d.html
https://pytorch.org/docs/master/generated/torch.nn.BatchNorm2d.html

Reproducible:-----------------------------------------------------------

Yes

Testcase:-----------------------------------------------------------

Attached.
testScripting.bn1d.execution.error.txt

Run as

python3 testScripting.bn1d.execution.error.py

We see the error with
self.use1DBatchNorm = True
but you could compare to the prediction and inference with (a functionally equivalent) BatchNorm2D via
self.use1DBatchNorm = False

Error message:-----------------------------------------------------------

Traceback (most recent call last):
File "testScripting.bn1d.executiona.error.py", line 61, in
output = mlmodel.predict({'input1': dummy_input.detach().numpy() }, useCPUOnly=True)
File "Library/Python/3.7/lib/python/site-packages/coremltools/models/model.py", line 329, in predict
return self.__proxy__.predict(data, useCPUOnly)
RuntimeError: {
NSLocalizedDescription = "Error computing NN outputs.";
}

Full log:-----------------------------------------------------------

python3 testScripting.bn1d.executiona.error.py

Torch version : 1.5.1
CoreML tools version : 4.0b1

TestModel(
(bn1): BatchNorm1d(28, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(bn2): BatchNorm2d(28, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(fc1): Linear(in_features=28, out_features=10, bias=True)

)

Graph from model

graph(%self.1 : __torch__.TestModel,
%x_ : Float(30, 1, 28)):
%41 : __torch__.torch.nn.modules.linear.Linear = prim::GetAttrname="fc1"
%38 : __torch__.torch.nn.modules.batchnorm.BatchNorm1d = prim::GetAttrname="bn1"
%21 : int = prim::Constantvalue=1 # testScripting.bn1d.executiona.error.py:30:0
%input.1 : Float(30, 28) = aten::squeeze(%x_, %21) # testScripting.bn1d.executiona.error.py:30:0
%44 : Tensor = prim::CallMethodname="forward"
%45 : Tensor = prim::CallMethodname="forward"
return (%45)

---

---

Code from model

def forward(self,
x_: Tensor) -> Tensor:
_0 = self.fc1
_1 = self.bn1
input = torch.squeeze(x_, 1)
return (_0).forward((_1).forward(input, ), )

---

Converting Frontend ==> MIL Ops: 0%| | 0/11 [00:00 Converting op 7 : constant : shape = n/a
Converting op input.1 : squeeze : shape = (30, 1, 28)
Converting op 9 : constant : shape = n/a
Converting op 10 : constant : shape = n/a
Converting op 11 : constant : shape = n/a
Converting op 12 : constant : shape = n/a
Converting op input : batch_norm : shape = (30, 28)
Converting op 14 : t : shape = (10, 28)
Converting op 15 : constant : shape = n/a
Converting op 16 : constant : shape = n/a
Converting op 17 : addmm : shape = (10,)
Converting Frontend ==> MIL Ops: 91%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 10/11 [00:00<00:00, 3059.08 ops/s]
Running MIL optimization passes: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 2180.51 passes/s]
Translating MIL ==> MLModel Ops: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 31428.71 ops/s]
name: "input1"
type {
multiArrayType {
shape: 30
shape: 1
shape: 28
dataType: FLOAT32
}
}
}
output {
name: "17"
type {
multiArrayType {
dataType: FLOAT32
}
}
}
metadata {
userDefined {
key: "com.github.apple.coremltools.source"
value: "torch==1.5.1"
}
userDefined {
key: "com.github.apple.coremltools.version"
value: "4.0b1"
}
}
>
Traceback (most recent call last):
File "testScripting.bn1d.executiona.error.py", line 61, in
output = mlmodel.predict({'input1': dummy_input.detach().numpy() }, useCPUOnly=True)
File "Library/Python/3.7/lib/python/site-packages/coremltools/models/model.py", line 329, in predict
return self.__proxy__.predict(data, useCPUOnly)
RuntimeError: {
NSLocalizedDescription = "Error computing NN outputs.";
}

[leovinus2001]

Same error with 4.0b3 and PyTorch 1.6.0

[pandov]

Workaround:

class BatchNorm1d(torch.nn.BatchNorm2d):
    def forward(self, x):
        x = x.reshape(x.shape[0], -1, 1, 1)
        x = super().forward(x)
        x = x.flatten(1)
        return x

[igiloh]

Seems like the problem might be 2D signal which is entering BatchNorm1D.

For me, at least, this workaround seems to work:

def forward(x):
        ...
        x = x.unsqueeze(dim=-1)
        x = self.batch_norm(x)
        x = x.squeeze(dim=-1)