One: [base_loader] Add GenModel test cases to improve Code Coverage

[wateret]

Here's the list of functions in base_loader.h::loadOperation that are never tested. We can make GenModel tests(which uses CircleGen) to cover them.

• [ ] loadElementwiseActivation(op, subg, ir::operation::ElementwiseActivation::Type::RELU, 1.f, -1.f);
• [ ] loadSquaredDifference(op, subg);
• [ ] loadOperationTo(op, subg);
• [ ] loadOperationTo(op, subg);
• [ ] loadReduce(op, subg, ir::operation::Reduce::ReduceType::ANY);
• [ ] loadOperationTo(op, subg);
• [ ] loadOperationTo(op, subg);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::CAST);
• [ ] loadOneHot(op, subg);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::ABS);
• [x] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::COS); #3775
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::SIN);
• [ ] loadReduce(op, subg, ir::operation::Reduce::ReduceType::PROD);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::LOG);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::ROUND);
• [ ] loadOperationTo(op, subg);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::LOGICAL_NOT);
• [ ] loadElementwiseBinary(op, subg, ir::operation::ElementwiseBinary::ElementwiseBinaryType::LOGICAL_OR);
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::ZEROS_LIKE);
• [x] loadTile(op, subg); #4253
• [ ] loadOperationTo(op, subg);
• [ ] loadBatchMatMul(op, subg);
• [x] loadLogSoftmax(op, subg); #4286
• [ ] loadElementwiseUnary(op, subg, ir::operation::ElementwiseUnary::Type::QUANTIZE);

Howto?

Please refer to the PRs above(the most recent one).

[glistening]

@wateret Alternatively, we may add nnpkgtc by creating models for each uncovered operators. It has several advantages than GenModel (at least I think):

i) models could be used to measure memory and performance because it is nnpackage.
ii) input and expected value are easy to create. Just run tflite2nnpkgtc.sh.

I usually create tflite by a) using flatc with hand-created json or b) splitting an op from existing model.

[s-barannikov]

It has several advantages than GenModel

There is one disadvantage:

• if a test fails, it does not necessarily mean that the problem is in the loader, it can be anywhere in the runtime pipeline.

If the intention is to just improve coverage, then it's OK I guess :)

[hyunsik-yoon]

Interesting this is
I think we could write CircleGen::save(filename) method, which can write a circle file.
So CircleGen -> model2nnpackage -> some test using nnpackage could be also possible.

[glistening]

There is one disadvantage:

* if a test fails, it does not necessarily mean that the problem is in the loader, it can be anywhere in the runtime pipeline.

@s-barannikov I did not understand. Both methods are same in this point. Error during running model from CircleGen or nnpkgtc.

---

Also ,during creating CircleGen test for sparcity, I've found several disadvantages.
It requires re-invent wheels (which means more development time and maintenance cost for that code).

• We need to write code using flatbuffer-generated. It is almost same to frontend and which has not been necessary in runtime.
• We need to write code for executing model like in GenModelTest.h

In addition, as I mentioned, we need to write expected values manually or copy and paste.
It requires several days at least for me, which requires several hours in nnpkgtcs.

Main advantage of CircleGen is that the tests can be managed in c++ source.

• They can be managed in ONE repo in one place.
• History can be managed.

Meanwhile, for nnpkg-tcs or models (.tflite or .circle), we provided alternative model server for providing binary resources. It is hard to review and keep the history.