Glow: [ONNXModelLoader] Support for LogSoftmax operator is missing

[antimora]

Tried compiling a ONNX model containing LogSoftmax operator and the compiler is failing with this error:

Error code: MODEL_LOADER_UNSUPPORTED_OPERATOR
Error message: Failed to load operator LogSoftmax .
Error return stack:
/workspaces/pytorch-glow/glow/lib/Importer/ONNXModelLoader.cpp:3745
/workspaces/pytorch-glow/glow/lib/Importer/ONNXModelLoader.cpp:3820
/workspaces/pytorch-glow/glow/lib/Importer/ONNXModelLoader.cpp:3949
/workspaces/pytorch-glow/glow/lib/Importer/ONNXModelLoader.cpp:3963
*** Check failure stack trace: ***

The glow version is at:
'''
commit e19f525b6e1bc91e04b79b55a80d7ed786faad7f
Date: Thu Apr 2 18:21:35 2020 -0700
'''

Running the following command line:

./build/bin/model-compiler -model=./mymodel.onnx -emit-bundle=./output -backend=CPU

[jfix71]

This should be really simply to add basic support for through lowering. We just need to add an importer case to ONNXModelLoader. Would you like to work on this @antimora?

[antimora]

@jfix71 I really wish but I am new to C++, so it would take me longer to get started. Otherwise I am happy to contribute.

[mciprian13]

@antimora Don't worry about it. Glow was (also) my first ever C++ real coding project. It took a while to get acquainted with Glow and C++ but in the end it's worth it because I find this technology very interesting.

[muhdmud]

Hi all, any updates on this? would like to help out

[mciprian13]

Hi @muhdmud. In my knowledge this is not done. Any contribution to this is warmly welcomed 😄

[muhdmud]

thanks for the opportunity!

if you could point and guide me to right direction on where to start, looks like I have to look in ONNXModelLoader and understand how the different operators are being loaded? then in loadoperator will check for type and return some loadLogSoftMax function?

correct me if im wrong in my understanding, am new to such big c++ projects

[mciprian13]

@muhdmud You can find some general guidelines here.
The main steps would be in this case:

1. You should define a new node LogSoftmax and define a lowering logic here to decompose it into Log + Softmax which are already supported by Glow.
2. Add logic in ONNXModelLoader to load a LogSoftmax node based on the proto description.
3. Add some unit tests in OnnxImporterTest

[muhdmud]

thank you! i will have a look and need do some additional reading on the lowering logic part.
also, do i implement the node first in graph and submit a pr? like work on it incrementally, or i should make sure everything is all done before submitting?
thanks again for the pointers! 😄

[mciprian13]

@muhdmud You should submit a single PR with everything. Thanks!

[muhdmud]

I am trying to build the project in debug, and it looks like its taking bigger than 5gb space(which is what i have currently), so i stopped it.
what is the expected size i should have or am i building it wrongly?

[muhdmud]

its ok managed to delete and clear somestuff, now facing some ram issue with wsl2
https://github.com/microsoft/WSL/issues/4166

[muhdmud]

have these warnings after ninja all, are they alright for now?

muhd@mud:/opensource/build_Debug$ ninja all
[218/653] Building CXX object thirdparty/folly/CMakeFiles/folly_base.dir/folly/synchronization/detail/Hardware.cpp.o
/home/muhd/opensource/glow/thirdparty/folly/folly/synchronization/detail/Hardware.cpp:122:72: warning: unused parameter 'status' [-Wunused-parameter]
[[noreturn]] FOLLY_DISABLE_SANITIZERS static void rtmAbortFunc(uint8_t status) {
^
1 warning generated.
[258/653] NodeGen: Generating nodes.
Writing node descriptors to:
/home/muhd/opensource/build_Debug/glow/AutoGenNodes.h
/home/muhd/opensource/build_Debug/glow/AutoGenNodes.cpp
/home/muhd/opensource/build_Debug/glow/AutoGenNodes.def
/home/muhd/opensource/build_Debug/glow/AutoGenNodesImport.h
/home/muhd/opensource/build_Debug/glow/AutoGenNodesExport.h
[262/653] InstrGen: Generating instructions.
Writing instr descriptors to:
/home/muhd/opensource/build_Debug/glow/AutoGenInstr.h
/home/muhd/opensource/build_Debug/glow/AutoGenInstr.cpp
/home/muhd/opensource/build_Debug/glow/AutoGenInstr.def
/home/muhd/opensource/build_Debug/glow/AutoGenIRBuilder.h
/home/muhd/opensource/build_Debug/glow/AutoGenIRBuilder.cpp
/home/muhd/opensource/build_Debug/glow/AutoGenIRGen.h
[320/653] Building CXX object lib/LLVMIRCodeGen/CMakeFiles/LLVMIRCodeGen.dir/GlowJIT.cpp.o
/home/muhd/opensource/glow/lib/LLVMIRCodeGen/GlowJIT.cpp:139:7: warning: 'LegacyLocalCXXRuntimeOverrides' is deprecated: ORCv1 utilities (utilities with the 'Legacy' prefix) are deprecated. Please use the ORCv2 LocalCXXRuntimeOverrides utility instead [-Wdeprecated-declarations]
cxxSymbolOverride_(
^
/usr/lib/llvm-9/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h:224:33: note: 'LegacyLocalCXXRuntimeOverrides' has been explicitly marked deprecated here
LegacyLocalCXXRuntimeOverrides::LegacyLocalCXXRuntimeOverrides(
^
/home/muhd/opensource/glow/lib/LLVMIRCodeGen/GlowJIT.cpp:183:7: warning: 'LegacyRTDyldObjectLinkingLayer' is deprecated: ORCv1 layers (layers with the 'Legacy' prefix) are deprecated. Please use ORCv2 (see docs/ORCv2.rst) [-Wdeprecated-declarations]
objectLayer_(ES_,
^
/usr/lib/llvm-9/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:356:3: note: 'LegacyRTDyldObjectLinkingLayer' has been explicitly marked deprecated here
LLVM_ATTRIBUTE_DEPRECATED(
^
/usr/lib/llvm-9/include/llvm/Support/Compiler.h:277:23: note: expanded from macro 'LLVM_ATTRIBUTE_DEPRECATED'
decl __attribute__((deprecated(message)))
^
/home/muhd/opensource/glow/lib/LLVMIRCodeGen/GlowJIT.cpp:191:7: warning: 'LegacyIRCompileLayer' is deprecated: ORCv1 layers (layers with the 'Legacy' prefix) are deprecated. Please use the ORCv2 IRCompileLayer instead [-Wdeprecated-declarations]
compileLayer_(objectLayer_, SimpleCompiler(TM_)) {
^
/usr/lib/llvm-9/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h:66:3: note: 'LegacyIRCompileLayer' has been explicitly marked deprecated here
LLVM_ATTRIBUTE_DEPRECATED(
^
/usr/lib/llvm-9/include/llvm/Support/Compiler.h:277:23: note: expanded from macro 'LLVM_ATTRIBUTE_DEPRECATED'
decl __attribute__((deprecated(message)))
^
/home/muhd/opensource/glow/lib/LLVMIRCodeGen/GlowJIT.cpp:233:49: warning: 'LegacyCtorDtorRunner' is deprecated: ORCv1 utilities (utilities with the 'Legacy' prefix) are deprecated. Please use the ORCv2 CtorDtorRunner utility instead [-Wdeprecated-declarations]
LegacyCtorDtorRunner ^
/usr/lib/llvm-9/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h:100:3: note: 'LegacyCtorDtorRunner' has been explicitly marked deprecated here
LLVM_ATTRIBUTE_DEPRECATED(
^
/usr/lib/llvm-9/include/llvm/Support/Compiler.h:277:23: note: expanded from macro 'LLVM_ATTRIBUTE_DEPRECATED'
decl __attribute__((deprecated(message)))
^
4 warnings generated.
[653/653] Linking CXX executable tests/ConcatBench
muhd@mud:/opensource/build_Debug$

also, could you point to the right direction where i can build a subset of the project? or i have to use debug/release everytime to ensure everything goes nicely

[antimora]

@muhdmud, thanks for looking into building the feature. Looking forward to using it. Meanwhile could you please use PyTorch/Glow forum to ask build/code questions. Getting ticket alerts but they're getting unrelated to the main issue.

[muhdmud]

oh okay sure!

[antimora]

@mciprian13 @jfix71, according to https://pytorch.org/docs/master/nn.functional.html#log-softmax, there is a warning regarding the numerical stability if operation is decomposed into Log and Softmax. Would this be a concern for inference mode as well?

@muhdmud You can find some general guidelines here.
The main steps would be in this case:

1. You should define a new node LogSoftmax and define a lowering logic here to decompose it into Log + Softmax which are already supported by Glow.
2. Add logic in ONNXModelLoader to load a LogSoftmax node based on the proto description.
3. Add some unit tests in OnnxImporterTest

As a workaround, is it okay to export ONNX model as decomposed nodes (Log+Softmax)?

[mciprian13]

@antimora Not sure why decomposing it brings numerical instability. You should search for an implementation and if that requires the node to be implemented atomically then you should define a new node LogSoftmax and add a special implementation for that.