Glow: Slice node with non-constant start value

[alannnna]

Right now Glow requires slice nodes to have constant starts. What's the feasibility of making start a (non-constant) NodeValue? This is the op spec I have in mind - notably, size is specified instead of ends, so the SliceNode's output would have a static shape.

My use case is running something with a dynamic start but a constant shape via the ONNX model loader. If the above is possible, I would also like to expose it as a custom Glow op in the ONNX loader (so that I can specify start and size instead of the ONNX spec's start and ends).

[jfix71]

Hi @alannnna, seems totally doable, since the shape would still be known statically. Unfortunately changing Slice will mean backends will all need to be updated (and private backends will break), but this could be fine. Another downside here is that depending on the backend, start may only be supported if it's a Constant, however we cannot tell that in our current isOpSupported. However this is already the case for other ops (e.g. some backends may only support Convolutions if Weights are Constant).

[alannnna]

Awesome, thanks. I've found a not-great workaround for now but plan to look into this in the upcoming months.

[nickgg]

I've been thinking a lot about this kind of thing (although not for slice specifically) and I think in general Glow would benefit from a Scalar type for situations like this (another simple example is Splat which is limited to Float due to how we express the parameter). This area is something on my list to think about this year :)

For Slice, I think it might make sense to create a new operator (e.g. Slice2 or whatevs) which has this behaviour and migrate backends to it adhoc until we no longer need the old version. @jfix71 thoughts?

[alannnna]

@nickgg Just curious, how would a Scalar type be used in the Slice case?

When you say creating a new Slice2 operator, does the "operator" refer to the Glow graph node (something like Slice2Node)?

[nickgg]

@alannnna The way I was thinking was to change the start (or maybe offset) to be an input NodeValue like you suggested. I'm really just suggesting that we build the concept of a Tensor that may only have one element as an intrinsic concept of the Glow Node "language" rather than enforce it adhoc in Node verifiers.

The other advantage a Scalar input has is Type, which doesn't matter for SliceNode since offsets will always be positive integers, but does matter in other places (such as SplatNode which can only create Float tensors currently).

Yes I meant creating a new Glow graph Node, maybe with a slightly more descriptive name (such as DynamicSliceNode?). It's going to be important to support both kinds of Slice for awhile since we wont be able to update all backends (particuarly private backends) at the same time.