Openwhisk: tests package needs some cleanup

[rabbah]

The tests suite currently include api gateway tests which belong in the CLI test suite since they are just actions (related to https://github.com/apache/incubator-openwhisk/issues/1959).

The cli.tests package maybe should be renamed since there are API tests not CLI (wsk) tests. Renaming this package will require changes in the CLI tests suite though.

There are some dated tests (sequence migration that are not needed anymore).

[rabbah]

The next phase is to remove and consolidate these suites:

tests/src/test/scala/system/basic/WskBasicJavaTests.scala
tests/src/test/scala/system/basic/WskBasicNode6Tests.scala
tests/src/test/scala/system/basic/WskBasicNode8Tests.scala
tests/src/test/scala/system/basic/WskBasicNodeDefaultTests.scala
tests/src/test/scala/system/basic/WskBasicPythonTests.scala
tests/src/test/scala/system/basic/WskBasicSwift3Tests.scala
tests/src/test/scala/system/basic/WskRestActionTests.scala
tests/src/test/scala/system/basic/WskRestBasicJavaTests.scala
tests/src/test/scala/system/basic/WskRestBasicNode6Tests.scala
tests/src/test/scala/system/basic/WskRestBasicNode8Tests.scala
tests/src/test/scala/system/basic/WskRestBasicNodeDefaultTests.scala
tests/src/test/scala/system/basic/WskRestBasicPythonTests.scala
tests/src/test/scala/system/basic/WskRestBasicSwift311Tests.scala
tests/src/test/scala/system/basic/WskRestBasicSwift41Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodeJavaTests.scala
tests/src/test/scala/system/basic/WskRestUnicodeNode6Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodeNode8Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodePython2Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodePython3Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodeSwift311Tests.scala
tests/src/test/scala/system/basic/WskRestUnicodeSwift41Tests.scala

Which don't cover all the runtimes but should - plus they include some tests which should be the responsibility of the runtime repository.

I propose that we have one basic _system_ test suite (like the unicode test) which iterate through all the runtimes, based on the runtimes manifest (indirectly received via GET https://api-host).

This will require that we have a canonical test artifact in every language under tests/dat/actions (or come up with a way to extract the hello example from the docs).

@akrabat @csantanapr FYI

[csantanapr]

What we need it is a minimum integration test for all runtimes deployed based on manifest.
To cover E2E flow what the unit tests don't cover in their individual repos.

[mgencur]

At this point I'd suggest the following changes to the test suite so as to achieve a better logical grouping of the tests:
1) move tests from system.basic.WskRestBasicTests to respective test classes (in the same package) for Actions, Rules, Packages etc. because the single class is a mix of tests that should be in the specific classes
2) move some tests from whisk.core.cli.test package to the system.basic package as they're using REST and are system-level tests anyway:
- WskRestBasicUsageTests - move tests to system.basic.{Action, Package, ...} except for two methods which are related to web actions, move these to whisk.core.cli.test.WskWebActionsTest
- pull test cases from WskActionSequenceTests and WskRestActionSequenceTests into a system.basic.WskSequenceTests, they're all using REST
- merge WskEntitlementTests and WskRestEntitlementTests into a single non-abstract class, and move this class to system.basic package
(it's been more than a year and there's still no other implementation than REST)
- move BaseApiGwTests, ApiGwRestBasicTests, ApiGwRestTests to a newly created package _system.api_
- since at this point the only remaining class in whisk.core.cli.test is WskWebActionTests, let's rename the package to _whisk.core.web_

[rabbah]

This is great! Thanks for tackling it.

The abstract classes exist to allow the CLI to reuse these test suites. Arguably the cli should use more unit test but there are very few of those today.

Nonetheless flattening the type hierarchy is still valid in some cases.

[rabbah]

For the sake of a more efficient review, id suggest either several prs or one pr with each commit refactoring one test suite.

You’ll need to be mindful of dependent changes that may be necessary in the cli repo.

[rabbah]

We've done a fair bit on this.