One: Compiler FE: update nnpackage test in CI

Created on 10 Jun 2020  路  5Comments  路  Source: Samsung/ONE

In our CI, nnpackage test runs the scripts like this.

...
./infra/scripts/docker_build_nncc.sh
./infra/scripts/docker_collect_nnpkg_resources.sh
...

But, these scripts are outdated and need updating. Let's update nnpackage test script!

Related : #2068

Most helpful comment

nnpackage-test consists of a few stages.

build stage

build compiler stage

  1. execute ./infra/scripts/docker_build_nncc.sh
    In this script, nncc is built with default preset(20191215 as of now) using docker image nnfw/nnas. And then tar the built binary to nncc-package.tar.gz.
    output : nncc-package.tar.gz(compiler module executables)
  2. execute ./infra/scripts/docker_collect_nnpkg_resources.sh
    nncc is built one more time with pre-defined REQUIRED_UNITS. This build results in graphdef(.pb), input, golden data(.h5), info(.info) file. And then tar the results to test-resources.tar.gz.
    output : test-resources.tar.gz(test resources)

    build runtime stage

execute ./infra/scripts/docker_build_cross_arm_runtime_release.sh. Runtime is built for nnpackage test.
output : test-suite.tar.gz

generate nnpackage

execute ./infra/scripts/test_make_nnpkg.sh. It generates nnpackage with compiler build's output.
output : nnpackage testsuite

test run

execute ./infra/scripts/test_arm_nnpkg.sh. It tests to validate our nnpackage.

Why is there an error in the new preset?

To generate nnpackage, tf2circle, info, input and golden data are needed.

  1. tf2circle : It is deprecated. This can be replaced by tf2tfliteV2(or tflchef) + tflite2circle + circle2circle.
  2. info : info is still needed.
  3. input and golden data : Though originally created through nnkit, it became difficult due to version control issues. Maybe it can be replaced by luci-interpreter or tflite-interpreter(python).

Winter is comming..

All 5 comments

nnpackage-test consists of a few stages.

build stage

build compiler stage

  1. execute ./infra/scripts/docker_build_nncc.sh
    In this script, nncc is built with default preset(20191215 as of now) using docker image nnfw/nnas. And then tar the built binary to nncc-package.tar.gz.
    output : nncc-package.tar.gz(compiler module executables)
  2. execute ./infra/scripts/docker_collect_nnpkg_resources.sh
    nncc is built one more time with pre-defined REQUIRED_UNITS. This build results in graphdef(.pb), input, golden data(.h5), info(.info) file. And then tar the results to test-resources.tar.gz.
    output : test-resources.tar.gz(test resources)

    build runtime stage

execute ./infra/scripts/docker_build_cross_arm_runtime_release.sh. Runtime is built for nnpackage test.
output : test-suite.tar.gz

generate nnpackage

execute ./infra/scripts/test_make_nnpkg.sh. It generates nnpackage with compiler build's output.
output : nnpackage testsuite

test run

execute ./infra/scripts/test_arm_nnpkg.sh. It tests to validate our nnpackage.

Why is there an error in the new preset?

To generate nnpackage, tf2circle, info, input and golden data are needed.

  1. tf2circle : It is deprecated. This can be replaced by tf2tfliteV2(or tflchef) + tflite2circle + circle2circle.
  2. info : info is still needed.
  3. input and golden data : Though originally created through nnkit, it became difficult due to version control issues. Maybe it can be replaced by luci-interpreter or tflite-interpreter(python).

Winter is comming..

Can we close this or are there more things to do?

@seanshpark I have to update CI script so still in progress:)

What I am going to do

For seamless new test landing, I have to implement some new scripts first.

  1. Revise docker_collect_nnpkg_resources.sh

In this script, common-artifacts which generate nnpkg resources will be built.

  1. Revise nncc-tc-to-nnpkg-tc.sh

Since tf2nnpkg interface has been chagned, it should be revised.

3. Change tf2nnpkg default version

If these changes are landed simultaneously, nnpackage test will not complain:)

http://npuci.mooo.com:8080/job/nnfw/job/master/job/push-nnpackage-test/1864/
Done:)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

seanshpark picture seanshpark  路  3Comments

mhs4670go picture mhs4670go  路  3Comments

hasw7569 picture hasw7569  路  4Comments

kishcs picture kishcs  路  3Comments

periannath picture periannath  路  3Comments