Magento2: [2.1.0-rc3] Jasmin JavaScript Tests Broken

[Vinai]

Steps to reproduce

1. Install Magento from the git 2.1.0-rc3 tag
2. Run bin/magento setup:static-content:deploy
3. Copy packages.json.sample to packages.json
4. Change required version of grunt-contrib-jasmine to ^1.0.0(workaround for issue #5044)
5. Run npm install
6. Copy Gruntfile.js.sample to Gruntfile.js
7. Run grunt spec

Note: I also followed these steps on the develop branch with identical results.

Expected result

Execution of jasmine based JavaScript tests in dev/tests/js/jasmine/tests

Actual result

$ grunt spec
Running "spec" task

Running "connect:blank" (connect) task
Started connect web server on http://0.0.0.0:8000

Running "jasmine:blank" (jasmine) task
Testing jasmine specs via PhantomJS


log: JQMIGRATE: Logging is active
>> Error: Script error for: Magento_Ui/js/grid/controls/bookmarks/view
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/component/core
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/component/manip
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/component/provider
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/component/traversal
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/registry/events
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError
>> Error: Script error for: Magento_Ui/js/lib/registry/storage
>> http://requirejs.org/docs/errors.html#scripterror at
>> http:/localhost:8000/requirejs/require.js:141 defaultOnError
>> http:/localhost:8000/requirejs/require.js:537 onError
>> http:/localhost:8000/requirejs/require.js:1681 onScriptError

Warning: PhantomJS timed out, possibly due to an unfinished async spec. Use --force to continue.

Aborted due to warnings.

In short, the following modules can't be loaded by require.js:

>> Error: Script error for: Magento_Ui/js/grid/controls/bookmarks/view
>> Error: Script error for: Magento_Ui/js/lib/component/core
>> Error: Script error for: Magento_Ui/js/lib/component/manip
>> Error: Script error for: Magento_Ui/js/lib/component/provider
>> Error: Script error for: Magento_Ui/js/lib/component/traversal
>> Error: Script error for: Magento_Ui/js/lib/registry/events
>> Error: Script error for: Magento_Ui/js/lib/registry/storage

Checking the files that where generated by setup:static-content:deploy these JS files are indeed not present.
Looking further, they also don't exist under app/code/Magento/Ui/view/*/web/js/*.
It seems that the JavaScript code was refactored without updating the tests :(
I didn't check in detail, but it seems as if the modules in question might have simply been renamed or moved. That would mean it probably would not be a lot of work to get the tests green again.

The tests should either be deleted (which would be a real shame) or updated to match the current production code.

The tests in question that need to be updated are:

dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/grid/controls/bookmarks/view.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/component/core.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/component/manip.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/component/provider.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/component/traversal.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/registry/events.test.js
dev/tests/js/jasmine/tests/app/code/Magento/Ui/base/js/lib/registry/storage.test.js

I really like the way the jasmine specs can be run, so it would be great to have the Magento core include a working test runner and a working test suite.

[Vinai]

Unfortunately it doesn't seem to be as simple as a rename. If I switch off the tests listed above, the remaining 721 specs run, but of those 115 fail. Seems like the testsuite is currently falling into disuse.
Will they be picked up again?
I think someone who is familiar with the classes under test should have the testsuite in good shape again relatively quickly.

[guz-anton]

Hi @Vinai ,
It's good point to gap our solution. Thanks.
We will do fixes under MAGETWO-54768. Just a notice.
Please be in touch.

[Vinai]

Do you have an update on this issue? Is it planned to be integrated? It would be so nice to be able to run JS tests again using the spec runner as intended.

[VladimirZaets]

Hi @Vinai. We have fixed JS tests, you can see this in current mainline.
Thanks for your report.