One: [luci] Shape inference not working for unknown shapes

[binarman]

Luci-based tools do not infer shapes for many operators if output shape is not specified in model.

This is happening because
1) GraphBuilder in importer (see code) sets node shape_status to NOSHAPE
2) shape inference checks shape_status and skips inference for nodes with NOSHAPE set (see code)

So, my main question is: was It implemented intentionally like this?

Additional question: what is the difference betweeen UNDEFINED and NOSHAPE statuses?

[binarman]

I found this during working on Where operator. Shape inference is not executed for both "no shape" and "scalar shape" cases (see recipes in #2741 )

[binarman]

+cc @seanshpark

[seanshpark]

was It implemented intentionally like this?

Yes, there was issues (like #1610) where shape was not set in the model file and I had to solve this somehow.
And there are unit tests that don't set shape informations. I could revise them, but this itself is good material for robust codes so I left them as-is and had to think about the solution.

Historically, shape/dtype as attribute to the CircleNode was introduced after we are working with this.
This made some design changes that I am still not confident about.

2741 is somewhat new case for _luci_ to handle cause output shape cannot be defined.

From current implementation, as ofm has no shape and shape inference won't work, exporting will store with no shape.
In our real in-house models there can exist this kind of cases.

You can searh from my past issues about this (https://github.com/Samsung/ONE/issues?q=is%3Aissue+author%3Aseanshpark+is%3Aclosed+shape+)

If I didn't understand corretcly about this issue, please point me again. As I wrote, I am still unconfident with current shape situations...

[binarman]

Here circle example
example.zip

[seanshpark]

Here circle example

The files seems fine to me. Output Identity may have no shape. Runtime should work as dynamic shape inference at run time.

[binarman]

The files seems fine to me. Output Identity may have no shape. Runtime should work as dynamic shape inference at run time.

Indeed. TFLite runtime works perfectly with unknown shapes.
It look like it actually doesn't care about any internal(not input and constants) shapes. I tried to set random shapes for tensors and tflite ignored them =)

[binarman]

Thank you!
I'll close this issue then, just one last thing I want to clarify:
Why we need both UNDEFINED and NOSHAPE shape statuses?

[seanshpark]

Why we need both UNDEFINED and NOSHAPE shape statuses?

Good question :)

• When a node is created, it's UNDEFINED. Unit tests don't do anything. Transformation Passes with new nodes may not do anything. UNDEFINED are for these. Nodes with this will try to run shape inference.
• NOSHAPE is when imported node don't have shapes so I wanted to preserve this state. I've tried to do some shape inference but not all cases can be solved. NOSHAPE will be exported as having no shape.

[binarman]

I see, so UNDEFINED is mostly for testing purposes.
Closing this issue, since all questions answered and there are nothing to do with shape inference for now.

Thank you very much!