Cntk: Batch normalization training on CPU is not yet implemented (2.0 b7 GPU 1bit binary)

[ghost]

Training CNN with BatchNormalization() results in:

EXCEPTION occurred: Batch normalization training on CPU is not yet implemented

Same script fails on the fresh 2.0 beta 8 GPU 1bit binary (with the same message as in b7 re Using CNTK batch normalization engine)

Any rough idea on ETA?

[KeDengMS]

Training support for batch normalization on CPU is on our TODO list. In the meanwhile, please train with GPU if possible.

[ghost]

@KeDengMS There is a lot of OS code around for this. One in particular comes to mind: MKL-DNN under Apache license. https://github.com/01org/mkl-dnn

[KeDengMS]

Thanks for the reference implementation. We'll keep you posted on the progress of this issue.

[bencherian]

Any update on this? (Is it expected by 2.0 release?) It would be nice to verify code functionality on a CPU only system prior to taking up time on a GPU system.

[KeDengMS]

We are working with Intel to incorporate MKL DNN for CPU support on BN/conv/pooling after 2.0 release. In the meanwhile, we would be much appreciated on contribution from the community on this issue.

[bencherian]

Thanks for the update, that's probably a bit too large of a project to undertake myself, but I'm looking forward to it.

[philippgille]

This issue still exists, right?

I tried:

1. docker run -it --rm -w /cntk/Examples/Image/DataSets/CIFAR-10 microsoft/cntk:2.0.rc3-cpu-python3.5
2. python install_cifar10.py
3. cd /cntk/Examples/Image/Classification/ResNet/Python
4. python TrainResNet_CIFAR10.py -n resnet20

And I get:
RuntimeError: Batch normalization training on CPU is not yet implemented.

All the resources about CNTK (GitHub wiki, website, Youtube videos, foreign articles) state that CNTK can be used with CPU. But this basic training doesn't work on a CPU.

Am I doing anything wrong or does model training simply not work with a CPU in CNTK? Why doesn't the README (Examples\Image\Classification\ResNet\Python\README.md) mention that?

[moprescu]

+1 I have the same issue

[ThiagoSousa]

+1 I have the same issue

[JimSEOW]

+1 today

[JimSEOW]

Can anyone share a link to ResNet_20.model ??

[KeDengMS]

@JimSEOW Uploaded ResNet20_CIFAR10 model

[JimSEOW]

@KeDengMS Big Thanks!

[KeDengMS]

Fixed