One: [compiler] Assert when type inferencing for dynamic output tensor

Created on 6 Jul 2020 · 25Comments · Source: Samsung/ONE

An assert is seen when testing for TFlite recipes under the Unique operation (please refer pull request #2829 for details).

Specifically, the assert happens at CircleTypeInferenceRule.cpp, when type checking for possibly a dynamic output tensor. The details are shown as follows:

++ /home/***/ONE/build/compiler/luci/tester/luci_readtester 
/home/***/ONE/build/compiler/luci/tests/Unique_000.circle
[INFO] Circle is '/home/***/ONE/build/compiler/luci/tests/Unique_000.circle'
[luci] NodeFinder INPUT(0) = 0x5555f086f2a0
[luci] NodeFinder OUTPUT(1) = 0x5555f086fdf0
[luci] NodeFinder OUTPUT(2) = 0x5555f086ffe0
--- graph dump begin -------------------------------------------
Name: main
In #0 { name: ifm, shape: [ 4 ] }
Out #0 { name: output_ofm, shape: [ ] }
Out #1 { name: output_ofm_idx, shape: [ 4 ] }

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT32)
%2 = circle.CIRCLEUNIQUEOUT(unique: %1)
%4 = circle.CIRCLEOUTPUT(from: %2)

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT32)
%3 = circle.CIRCLEUNIQUEOUT(unique: %1)
%5 = circle.CIRCLEOUTPUT(from: %3)

--- graph dump end ---------------------------------------------
--- FixInterGraph main-------------------------
--- ValidateGraphProp main---------------------
--- post_import_graph done -------------------------------------
[luci] shape: ifm
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: []
[luci] shape: ofm
              own_shape: [] -> infer: []
luci_readtester: /home/***/ONE/compiler/luci/service/src/CircleTypeInferenceRule.cpp:505: virtual loco::DataType {anonymous}::TypeInferenceAlgorithm::visit(const luci::CircleOutput*): Assertion `output_dtype == loco::dtype_get(node->from())' failed.
/home/***/ONE/compiler/luci/tests/readverify.sh: line 32: 17054 Aborted                 (core dumped) "${VERIFY_BINARY_PATH}" "${TESTCASE_FILE}.circle"

A quick look at the recipe reveals that the input tensor type is FLOAT32, and there are two output tensors, namely

ofm - type FLOAT32, shape unknown as it is dynamic
ofm_idx - type INT32, shape the same as that of input

I am not sure if I may have missed anything in the recipe. In order to understand the nature of the issue, I am not sure whether the output_dtype in the above case refers to ofm or ofm_idx. Could anyone please help me with this? Thank you in advance.

help wanted

Source

dr-venkman

All 25 comments

I see your current draft has code like this

  loco::DataType visit(const luci::CircleUniqueOut *node) final
  {
    return loco::dtype_get(node->input());
  }

Please fix this to return correct type for two outputs.

I assume you copied from existing multi-output node, then you should have

auto index = node->index();

Use this index to distinguish which node you refer. 0 for first one, 1 for second output.

Correct output dtype depends on the node type, this case Unique.

seanshpark on 6 Jul 2020

👍1

Thank you, it is clear that the first output will follow the same type as that of input.

For the second output, however, the type comes from the model/recipe itself. Is there a way to reference that in the code? I tried fetching the node with at(1)->node();, and then retrieving its data type as luci::dtype_get(node_obj). However, this approach did not work as expected. Am I missing a step here?

dr-venkman on 6 Jul 2020

the type comes from the model/recipe itself.

Can you explain more about this? Its hard for me to understand what this means...

seanshpark on 6 Jul 2020

from the document https://www.tensorflow.org/api_docs/python/tf/unique,

Returns | A tuple of Tensor objects (y, idx).
-- | --
y | A Tensor. Has the same type as x.
idx | A Tensor of type out_idx.

where out_idx is An optional tf.DType from: tf.int32, tf.int64. Defaults to tf.int32.

I think looking at the source files may help how to solve this.

I assume you created a .pbtxt file. can you copy-paste this file?
I also assume you created a .tflite file. can you copy-paste the dump result of this file from _tfldump_?

seanshpark on 6 Jul 2020

Your recipe in draft has

operand {
  name: "ofm_idx"
  type: INT32
  shape { dim: 4 }
}

which seems to be the out_idx. is this correct?

seanshpark on 6 Jul 2020

I tried fetching the node with at(1)->node();,

Can you post current WIP code at your draft #2829?

seanshpark on 6 Jul 2020

I tried fetching the node with at(1)->node();,

Can you post current WIP code at your draft #2829?

I did so currently. Apologies for amending to the commit, please find the two modified files below, so that viewing the diffs is easier:

compiler/luci/service/src/CircleTypeInferenceRule.cpp
compiler/luci/lang/include/luci/IR/Nodes/CircleUniqueOut.h

dr-venkman on 7 Jul 2020

Your recipe in draft has
operand {
  name: "ofm_idx"
  type: INT32
  shape { dim: 4 }
}
which seems to be the out_idx. is this correct?

That is indeed correct. Please let me know if I miss anything in this regard.

dr-venkman on 7 Jul 2020

the type comes from the model/recipe itself.

Can you explain more about this? Its hard for me to understand what this means...

I meant that while the type of ofm can be inferred from input, the type of ofm_idx is by default tf.int32, and can be overridden by the operation itself (as tf.int64 for instance). So, in the former case, the type inferencing is straighforward and follows from input. In the case of the latter, I assumed that I have to fetch the ofm_idx Node and access it's type that comes directly from the recipe. Kindly let me know if my understanding is correct.

dr-venkman on 7 Jul 2020

Current draft code

  loco::DataType visit(const luci::CircleUniqueOut *node) final
  {
    if (node->index() == 0)
    {
      return loco::dtype_get(node->input());
    }
    assert(node->index() == 1);
    return loco::dtype_get(node->out_idx());
  }

~seems OK.~

~What is your error exactly with which recipe file (or all recipe fails?) ?~

CircleUniqueOut IR needs review.

seanshpark on 7 Jul 2020

Current draft code

  loco::DataType visit(const luci::CircleUniqueOut *node) final
  {
    if (node->index() == 0)
    {
      return loco::dtype_get(node->input());
    }
    assert(node->index() == 1);
    return loco::dtype_get(node->out_idx());
  }

seems OK.

What is your error exactly with which recipe file (or all recipe fails?) ?

Thank you for verifying the changes. Please find the output below:

++ /home/***/ONE/build/compiler/luci/tester/luci_readtester 
/home/***/ONE/build/compiler/luci/tests/Unique_001.circle
[INFO] Circle is '/home/***/ONE/build/compiler/luci/tests/Unique_001.circle'
[luci] NodeFinder INPUT(0) = 0x56072fadf2a0
[luci] NodeFinder OUTPUT(1) = 0x56072fadfdf0
[luci] NodeFinder OUTPUT(2) = 0x56072fadffe0
--- graph dump begin -------------------------------------------
Name: main
In #0 { name: ifm, shape: [ 4 ] }
Out #0 { name: output_ofm, shape: [ ] }
Out #1 { name: output_ofm_idx, shape: [ 4 ] }

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT64)
%2 = circle.CIRCLEUNIQUEOUT(unique: %1)
%4 = circle.CIRCLEOUTPUT(from: %2)

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT64)
%3 = circle.CIRCLEUNIQUEOUT(unique: %1)
%5 = circle.CIRCLEOUTPUT(from: %3)

--- graph dump end ---------------------------------------------
--- FixInterGraph main-------------------------
--- ValidateGraphProp main---------------------
--- post_import_graph done -------------------------------------
[luci] shape: ifm
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: []
[luci] shape: ofm
              own_shape: [] -> infer: []
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 1) >= this->size() (which is 1)
/home/***/ONE/compiler/luci/tests/readverify.sh: line 32: 18026 Aborted                 (core dumped) "${VERIFY_BINARY_PATH}" "${TESTCASE_FILE}.circle"

It seems like a out of range error, which makes me wonder if the Node creation is correct, or if I am accessing the right index.

dr-venkman on 7 Jul 2020

Read the comments I've added to your draft

seanshpark on 7 Jul 2020

Read the comments I've added to your draft

Thanks a lot for your inputs, I have pushed the necessary changes to the pull request. Regarding testcases, I have managed to clear all but one of them, namely the test.recipe under Unique_000 still fails at the assert condition. I will look at it in detail and update accordingly.

dr-venkman on 7 Jul 2020

👍1

Maybe close this issue?

seanshpark on 7 Jul 2020

I was looking at the logs, and noted the following:

The overall test result shows only one failure:


99% tests passed, 1 tests failed out of 73

Total Test time (real) =  55.14 sec

The following tests FAILED:
     25 - mir2loco_test (Failed)

The above errors have no dependence on the files modified under the pull request, in my modest opinion. Please correct if otherwise.

However, when I check the logs for Unique_000, I got the following:

++ /home/***/ONE/build/compiler/luci/tester/luci_readtester /home/***/ONE/build/compiler/luci/tests/Unique_000.circle
[INFO] Circle is '/home/***/ONE/build/compiler/luci/tests/Unique_000.circle'
[luci] NodeFinder INPUT(0) = 0x5555f086f2a0
[luci] NodeFinder OUTPUT(1) = 0x5555f086fdf0
[luci] NodeFinder OUTPUT(2) = 0x5555f086ffe0
--- graph dump begin -------------------------------------------
Name: main
In #0 { name: ifm, shape: [ 4 ] }
Out #0 { name: output_ofm, shape: [ ] }
Out #1 { name: output_ofm_idx, shape: [ 4 ] }

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT32)
%2 = circle.CIRCLEUNIQUEOUT(unique: %1)
%4 = circle.CIRCLEOUTPUT(from: %2)

%0 = circle.CIRCLEINPUT()
%1 = circle.UNIQUE(input: %0, idx_out_type: INT32)
%3 = circle.CIRCLEUNIQUEOUT(unique: %1)
%5 = circle.CIRCLEOUTPUT(from: %3)

--- graph dump end ---------------------------------------------
--- FixInterGraph main-------------------------
--- ValidateGraphProp main---------------------
--- post_import_graph done -------------------------------------
[luci] shape: ifm
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm_idx
              own_shape: [4] -> infer: [4]
[luci] shape: ofm
              own_shape: [] -> infer: []
[luci] shape: ofm
              own_shape: [] -> infer: []
luci_readtester: /home/***/ONE/compiler/luci/service/src/CircleTypeInferenceRule.cpp:505: virtual loco::DataType {anonymous}::TypeInferenceAlgorithm::visit(const luci::CircleOutput*): Assertion `output_dtype == loco::dtype_get(node->from())' failed.
/home/***/ONE/compiler/luci/tests/readverify.sh: line 32: 17054 Aborted                 (core dumped) "${VERIFY_BINARY_PATH}" "${TESTCASE_FILE}.circle"

The specific testcase looks consistent with the other recipes, which passed. I am not clear whether this should be counted as a failure. Could you please let me know?

dr-venkman on 8 Jul 2020

Can you give two output_dtype and loco::dtype_get(node->from()) values?

seanshpark on 8 Jul 2020

CircleOutput has index(). Can you give this value too?

seanshpark on 8 Jul 2020

Can you give two output_dtype and loco::dtype_get(node->from()) values?

I tried the following, and it seemed to pass:

venkat@venkat-PMBSB09A-Samsung-DeskTop:~/projects/pubfork_ONE/ONE$ build/compiler/luci/tester/luci_readtester build/compiler/luci/tests/Unique_000.circle 
[INFO] Circle is 'build/compiler/luci/tests/Unique_000.circle'
output_dtype = 7 , loco::dtype_get(node->from()) = 7
output_dtype = 10 , loco::dtype_get(node->from()) = 10

dr-venkman on 8 Jul 2020

I tried the following, and it seemed to pass:

What have you changed? or what was the problem?

seanshpark on 8 Jul 2020

CircleOutput has index(). Can you give this value too?

The value is 1. Overall, the test ran successfully, as I gather. I wonder why the log indicates otherwise.

dr-venkman on 8 Jul 2020

I tried the following, and it seemed to pass:

What have you changed? or what was the problem?

Nothing, as far as I recall. However, when I run ./nncc test, I get the error in the log file as shown above. Note that there is no debug log preceding it. Hence the confusion as to whether the tests passed successfully.

dr-venkman on 8 Jul 2020

I get the error in the log file as shown above.

Have to deleted the log file and tried again?

seanshpark on 8 Jul 2020

The following tests FAILED:
25 - mir2loco_test (Failed)

This project is not used in this year. You can ignore this for now.

seanshpark on 8 Jul 2020

The following tests FAILED:
25 - mir2loco_test (Failed)

This project is not used in this year. You can ignore this for now.

Thanks for the clarification, I wondered if it had any dependency to my pull request.

dr-venkman on 8 Jul 2020

I get the error in the log file as shown above.

Have to deleted the log file and tried again?

This worked for me, surprisingly. Now all testcase logs show passed result. Thanks a lot, will keep it in mind next time :).

dr-venkman on 8 Jul 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Improvement error handling

KimDongEon · 4Comments

Compiler FE: conversion of pytorch to TF with Ops

seanshpark · 3Comments

[onert] Improve performance of WICPlanner

periannath · 3Comments

Compiler FE: Enable ReLU6 Op

binarman · 3Comments

[infra/Android] Using gold linker for android build

periannath · 3Comments